Many animal species display striking color differences with respect to geographic location, sex, and body region. Traditional adaptive explanations for such complex patterns invoke an interaction between selection for conspicuous signals and natural selection for crypsis. Although there is now a substantial body of evidence supporting the role of sexual selection for signaling functions, quantitative studies of crypsis remain comparatively rare. Here, we combine objective measures of coloration with information on predator visual sensitivities to study the role of crypsis in the evolution of color variation in an Australian lizard species complex (Ctenophorus decresii). We apply a model that allows us to quantify crypsis in terms of the visual contrast of the lizards against their natural backgrounds, as perceived by potential avian predators. We then use these quantitative estimates of crypsis to answer the following questions. Are there significant differences in crypsis conspicuousness among populations? Are there significant differences in crypsis conspicuousness between the sexes? Are body regions “exposed” to visual predators more cryptic than “hidden” body regions? Is there evidence for local adaptation with respect to crypsis against different substrates? In general, our results confirmed that there are real differences in crypsis conspicuousness both between populations and between sexes; that exposed body regions were significantly more cryptic than hidden ones, particularly in females; and that females, but not males, are more cryptic against their own local background than against the background of other populations. Body regions that varied most in contrast between the sexes and between populations were also most conspicuous and are emphasized by males during social and sexual signaling. However, results varied with respect to the aspect of coloration studied. Results based on chromatic contrast (“hue” of color) provided better support for the crypsis hypothesis than did results based on achromatic contrast (“brightness” of color). Taken together, these results support the view that crypsis plays a substantial role in the evolution of color variation and that color patterns represent a balance between the need for conspicuousness for signaling and the need for crypsis to avoid predation.