The Hengduan Mountains, situated in the southeastern Tibetan Plateau, have undergone dramatic geological and climatic changes over the Pleistocene epoch. Several studies have revealed that the mountains served as a refugium during the ice age. The large white-bellied rat Niviventer excelsior is a rodent endemic to the Hengduan Mountains, which makes it an appropriate species for investigating the influence of glacial movements on the genetic structure of mammals. In this study, we sequenced the partial mitochondrial DNA control region from 72 N. excelsior specimens collected from 20 localities. The results revealed very high levels of haplotype diversity (h = 0.947) and nucleotide diversity (π = 0.101) in this species. No common haplotype was found to be shared in samples from all geographic regions. Demographic analyses suggested that N. excelsior populations had not been subject to either expansion or bottleneck. The phylogenetic relationships among the haplotypes have no correlation with their geographical origins, while topology revealed two major clades. We speculate that the populations of N. excelsior may have been restricted to two separate refugia during the Last Glacial Maximum (0.60–0.17 Mya), with one west and one east of the Shaluli Mountains. Between the two major refugia, there existed a more widely distributed network subrefugia, which conserved genetic variations in N. excelsior. These results indicated that complex topographic configuration in the Hengduan Mountains provided a network of refugia to maintain the high level of genetic diversity in Pleistocene glaciations.