The evolution of different spore size classes, or heterospory, is a fundamental reproductive innovation in land plants. The appearance of heterospory is particularly notable during the Devonian, when most known origins of the trait occur. Here we provide a perspective on the evolution of heterospory during this time interval, particularly from the late Early Devonian through the Middle Devonian (Emsian to Givetian Stages; 408–383 Ma), which shows an unusually high concentration of heterospory origins. We use theoretical considerations and compilations of fossil and extant spore sizes to suggest that the basic features of most heterosporous lineages, large spores and gametophytes that mature within the spore wall, are difficult to evolve in combination, because large spores disperse poorly but small spores cannot support a functional gametophyte developing within their walls; evolving spores between 100 and 200 microns in diameter appears to represent a particularly important barrier for the evolution of heterospory. We then discuss why this barrier may have been lower in the Devonian, noting that the appearance and spread of heterospory is coincident with the emergence of peat-accumulating wetland habitats. We suggest that more widespread wetland habitats would have generally lowered barriers to the evolution of heterospory by reducing dispersal limitation in larger spores. Ultimately, we suggest that the initial evolution of heterospory may be explained by major changes in sedimentology, thought to have been driven by plant evolution itself, that increased the diversity of terrestrial depositional environments and led to a greater number of habitats where large spores could be successful.