Japanese gardens emphasize on the aesthetic value of bryophytes, a major ground cover in moss gardens. Using medieval and early modern documents, this study analyzed the evolution of moss gardens and their aesthetic value by elucidating how moss usage changed over time. Twenty documents written from the 11th to the 19th centuries were surveyed. According to these documents, mosses were used as supplemental ground cover around the 11th century. After the 15th century, mosses were intentionally selected and planted in Zen gardens (gardens established at Zen temples) and Roji (gardens attached to tea ceremony rooms). Zen gardens provide monks with a feeling of being deep in the mountains. Roji particularly emphasizes Wabi-Sabi aesthetics, which embrace the opposite of conventional beauty, such as aging processes and tranquility. These concepts in Zen and Roji gardens are associated with bryophyte-covered landscapes. This is because these landscapes suggest a mountain-like ambiance, the passage of time, and tranquility in Japan. These associations suggest that the importance of bryophyte cover in gardens might have increased with the development of Zen and Roji gardens—specifically, mosses came to be used as major ground cover after the establishment of Roji. Moreover, given that Zen Buddhism underlies the establishment of Wabi-Sabi aesthetics, it might have been the impetus for the development of moss gardens.