Historically, explanations for the evolution of floral traits that reduce self-fertilization have tended to focus on selection to avoid inbreeding depression. However, there is growing support for the hypothesis that such traits also play a role in promoting efficient pollen dispersal by reducing anther-stigma interference. The relative importance of these two selective pressures is currently a popular topic of investigation. To date, there has been no theoretical exploration of the relative contributions of selection to avoid the genetic costs of self-fertilization and selection to promote efficient pollen dispersal on the evolution of floral traits. We developed a population genetic model to examine the influence of these factors on the evolution of dichogamy: the temporal separation of anther maturation and stigma receptivity. Our analysis indicates that anther-stigma interference can favor dichogamy even in the absence of inbreeding depression. Although anther-stigma interference and inbreeding depression are the key forces driving the initial evolution of dichogamy, selection to match the timing of pollen dispersal to the availability of ovules at the population level becomes a more potent force opposing the further evolution of dichogamy as the extent of temporal separation increases. This result may help to explain otherwise puzzling phenomena such as why dichogamy is rarely complete in nature and why dichogamy tends to be associated with asynchronous flower presentation.