Cumulative cultural change requires organisms that are capable of both exploratory individual learning and faithful social learning. In our model, an organism's phenotype is initially determined innately (by its genotypic value) or by social learning (copying a phenotype from the parental generation), and then may or may not be modified by individual learning (exploration around the initial phenotype). The environment alternates periodically between two states, each defined as a certain range of phenotypes that can survive. These states may overlap, in which case the same phenotype can survive in both states, or they may not. We find that a joint social and exploratory individual learning strategy—the strategy that supports cumulative culture—is likely to spread when the environmental states do not overlap. In particular, when the environmental states are contiguous and mutation is allowed among the genotypic values, this strategy will spread in either moderately or highly stable environments, depending on the exact nature of the individual learning applied. On the other hand, natural selection often favors a social learning strategy without exploration when the environmental states overlap. We find only partial support for the “consensus” view, which holds that individual learning, social learning, and innate determination of behavior will evolve at short, intermediate, and long environmental periodicities, respectively.