As heron and egret colonies are sometimes abandoned due to human activity and subsequently newly established at nearby locations, colony locations per se cannot represent colony fates. Here, “colony lineages” were traced by connecting short-distance location shifts between colony sites in consecutive years to quantify the site fidelity of heron colonies themselves rather than the breeding-site fidelity of individuals. Site fidelity of 61 colony lineages recorded from 1963 to 2013 in and around Ibaraki Prefecture, Japan, was evaluated by persistence periods with survival analysis and shapes of historical trajectories by applying the correlated random walk analysis. While 36% (n = 22) of colony lineages did not show site fidelity and lasted only 1 year or indicated untangled shapes of historical trajectories, 64% (n = 39) of colony lineages exhibited site fidelity by reusing the same locations for multiple years or by staying within a small area with entangled shapes of historical trajectories. Even though suitable breeding sites were abundant in the study area and site fidelity of colony members was low, a high rate of site fidelity in colony lineages may be achieved when a few faithful pioneer herons and egrets are followed by many others.