Rangelands with scattered trees are complex and dynamic systems with a long history in Europe. Generally referred to as “wood-pastures,” they are considered to have outstanding conservation value. Thorny shrubs are important for supporting the biodiversity of these wooded rangelands, as well as facilitating the regeneration of trees by acting as nurse species. We assess the direct effects of temporary shrub encroachment under the cover of mature sparse trees on overall plant and habitat diversity. We surveyed the herb layer of the main landscape features of a wood-pasture: open pasture, trees with a grass understory, trees with shrubs, and adjacent forest edges. The herb layer under trees with shrubs resembled that of forest edges more than open pastures and trees with grass. Trees with grass had a higher cover of ruderal species than trees with shrubs, while forest edges and open pastures had a lower cover of them. Forest species were absent from open pastures but were well represented in the other sites. The herb layer of trees with shrubs and forest edges had similar cover values, while trees with grass had a significantly lower cover of herbs than the other types. Trees with shrubs had higher species richness than any of the other three landscape features and had a much higher proportion of diagnostic species. We conclude that the scattered trees and shrubs of the studied silvopastoral system have additive facilitative effects on their understory, probably through modifying the microenvironment and grazing pressure, leading to the formation of temporary diversity hot spots with distinct vegetation. Thus maintaining a moderate level of shrub-encroachment under sparse trees is recommended for not only creating safe havens for tree recruitment but also increasing the overall species and habitat diversity of wood-pastures.