The lily family Colchicaceae consists of geophytic herbs distributed on all continents except the Neotropics. It is particularly diverse in southern Africa, where 80 of the 270 species occur. Colchicaceae exhibit a wide range of ploidy levels, from 2n = 14 to 2n = 216. To understand where and how this cytogenetic diversity arose, we generated multilocus phylogenies of the Colchicaceae and the Colchicum clade that respectively included 85 or 137 species plus relevant outgroups. To infer the kinds of events that could explain the observed numbers in the living species (dysploidy, polyploidization, or demi-duplication, i.e. fusion of gametes of different ploidy), we compared a series of likelihood models on phylograms, penalized likelihood ultrametric trees, and relaxed clock chronograms that contained the 58 or 112 species with published chromosome counts. While such models involve simplification and cannot address the processes behind chromosomal rearrangements, they can help frame questions about the direction of change in chromosome numbers in well-sampled groups. The results suggest that dysploidy played a large role in the Colchicaceae, with the exception of Colchicum itself for which we inferred frequent demi-duplication. While it is known that triploids facilitate the fixation of tetraploidy and that plant species often include individuals of odd ploidy level (triploids, pentaploids), we hesitate to accept the phylogenetically inferred scenario without molecular-cytogenetic work and data from experimental hybridizations.