Sexual dimorphism is a widespread phenomenon and contributes greatly to intraspecies variation. Despite a long history of active research, the genetic basis of dimorphism for complex traits remains unknown. Understanding the sex-specific differences in genetic architecture for cranial traits in a highly dimorphic species could identify possible mechanisms through which selection acts to produce dimorphism. Using distances calculated from three-dimensional landmark data from CT scans of 402 baboon skulls from a known genealogy, we estimated genetic variance parameters in both sexes to determine the presence of gene-by-sex (G × S) interactions and X-linked heritability. We hypothesize that traits exhibiting the greatest degree of sexual dimorphism (facial traits in baboons) will demonstrate either stronger G × S interactions or X-linked effects. We found G × S interactions and X-linked effects for a few measures that span the areas connecting the face to the neurocranium but for no traits restricted to the face. This finding suggests that facial traits will have a limited response to selection for further evolution of dimorphism in this population. We discuss the implications of our results with respect to the origins of cranial sexual dimorphism in this baboon sample, and how the genetic architecture of these traits affects their potential for future evolution.