Heathlands in the northwest of Spain have been traditionally used by domestic herbivores as a food resource. However, their abandonment in the past decades has promoted a high incidence of wildfires, threatening biodiversity. Sheep and goats exhibit different grazing behavior, affecting rangelands dynamics in a different way, but the botanical and structural composition may also affect such dynamics. The aim of this article was to compare the grazing effects of sheep and goats on three different heathland types: previously burned grass- or gorse (Ulex gallii Planchon)-dominated and unburned heather (Erica spp.)-dominated shrublands. Two grazing treatments (sheep or goats) were applied in each vegetation type in a factorial design with two replicates (12 experimental plots). A small fenced area was excluded from grazing in each plot (control treatment). The experiment was carried out from 2003 to 2006, and the grazing season extended from May to October–November. Plant cover, canopy height, and phytomass amount and composition were assessed in each plot. Results showed that goats controlled shrub encroachment, phytomass accumulation, and canopy height more than sheep in either burned grass– and gorse– and unburned heather–dominated shrublands. It was accompanied by a higher increase of herbaceous species under goat grazing. Nevertheless, plant dynamics showed different trends between the three vegetation types studied. Grazing effects were more important in previously burned grass-dominated heathlands than in unburned heather-dominated shrublands. At the end of the experiment (May 2006), shrub cover, height, and woody phytomass were significantly higher in the ungrazed enclosures than in the grazed plots. Small ruminant grazing, especially with goats, is proposed as an efficient tool to reduce shrub encroachment and woody phytomass accumulation in heathlands, thus reducing fire hazard, although these grazing effects depend on heathland composition.