Agricultural land-use legacies remain persistent determinants of plant-community composition, despite extended periods of recovery in postagricultural forests of New York. We surveyed the composition of vascular plant species for 33 paired residual (unconverted) and secondary (postagricultural) forest sites across three physiographic regions in New York to assess the recovery of residual forest species in secondary successional stands. There were 169 vascular plant species recorded among the 33 study sites. The residual stands exhibited slightly greater mean (± SE) species richness (26.8 ± 1.7 species per site) than secondary-interior (21.4 ± 1.2, P = 0.001) and secondary-edge (24.1 ± 1.6, P = 0.02) zones. Plant communities differed across the three physiographic regions (P = 0.001) and by stand history within regions (P = 0.004). Of the vascular plant species occurring at our sites, 22 (13%) were indicators of residual forest stands and 27 (16%) were indicators of secondary, postagricultural forest stands. Within 10 to 25 m of the historic field-forest ecotones, the relative frequencies of many residual woodland indicator species fell to 0–20% of the maxima they attained in the residual stands. Ant-dispersed (myrmecochorous) taxa represented 23% of residual and 0% of secondary forest indicators and exhibited lower migration rates (P < 0.001) than plants possessing other diaspore dispersal modes (anemochory, zoochory, barochory). In addition to plant dispersal mode, we found pH and forest-floor depth to be important determinants of plant species. Our results indicate that impoverishment of plant communities in postagricultural forests represent a regionally pervasive phenomenon with important resource management implications in the eastern USA. Residual woodlands remain important source populations for recovery of herbaceous species in postagricultural landscapes. However, given the slow historic recovery rates that we and others have observed in postagricultural secondary forests, in addition to other contemporary factors potentially inhibiting dispersal and establishment (e.g., herbivory, pollinator disruptions, invasive species), it seems likely that active restoration efforts are needed to promote full recovery of diverse, native temperate-forest herb communities.