The wild rhinoceros populations have declined drastically in the past decades because the rhinoceros are heavily hunted for their horns. Zoological institutions aim to conserve rhinoceros populations in captivity, but one of the challenges of ex situ conservation is to provide food sources that resemble those available in the wild. Considering that the mammalian gut microbiota is a pivotal player in their host's health, the gut microbiota of rhinoceros may also play a role in the bioavailability of nutrients. Therefore, this study aims to characterize the fecal microbiome composition of grazing white rhinoceros (WR; Ceratotherium simum) and greater one-horned rhinoceros (GOHR; Rhinoceros unicornis) as well as the browsing black rhinoceros (BR; Diceros bicornis) kept in European zoos. Over the course of 1 yr, 166 fecal samples in total were collected from 9 BR (n = 39), 10 GOHR (n = 56), and 14 WR (n = 71) from 23 zoological institutions. The bacterial composition in the samples was determined using 16S rRNA gene Illumina sequencing. The fecal microbiomes of rhinoceros clustered by species, with BR clustering more distantly from GOHR and WR. Furthermore, the data report clustering of rhinoceros microbiota according to individual rhinoceros and institutional origin, showing that zoological institutions play a significant role in shaping the gut microbiome of rhinoceros species. In addition, BR exhibit a relatively higher microbial diversity than GOHR and WR. BR seem more susceptible to microbial gut changes and appear to have a more diverse microbiome composition among individuals than GOHR and WR. These data expand on the role of gut microbes and can provide baseline data for continued efforts in rhinoceros conservation and health status.