Habitat fragmentation can intensify consumer attack on plants if herbivores stay longer, eat more extensively, or exhibit higher density at low resource availability; it may also reduce attack if consumers are dispersal limited and therefore fail to forage on smaller or more distant patches. Here we test these alternatives with a large-scale grassland fragmentation experiment, comparing vertebrate and invertebrate herbivory on seedlings of an oak species of central North America (Quercus ellipsoidalis) experiencing recruitment difficulties across its range. All seedlings suffered significant herbivore-based defoliation regardless of experimental context, confirming the influence of consumer pressure on recruiting oaks on contemporary landscapes. However, vertebrates and insects responded differently to fragmentation. Vertebrate grazing was unaffected by patch size or isolation – remnants were either periodically visited by foraging deer or colonized by small mammals that occasionally attacked seedlings. In contrast invertebrate consumers significantly reduced seedling growth via the effects of defoliation, but these effects decreased on smaller and more isolated patches. This was associated with a negative influence of patch size on the diversity of herbaceous vegetation, with reduced plant diversity correlated with the reduced abundance and diversity of several generalist insect herbivore groups. Our results demonstrate fragmentation has the potential to affect the identity and intensity of herbivore attack on oaks, although whether the reduced insect attack we observed in isolated remnants is sufficient to increase juvenile survivorship remains to be tested.