Studying the vulnerability of rural settlements is necessary for their revitalization and sustainable development. In this study, a basic organizational framework and evaluation system that considers the natural environment, social development, and other factors in terms of exposure, sensitivity, and adaptability were developed for the vulnerability of rural settlements. Qixingguan District of Bijie City was considered as a case study, and a geographic information system spatial analysis function and geographic probe model were applied to determine the spatial distribution and characteristics of rural settlement vulnerability in karst mountains and their influencing factors. The results demonstrated that rural settlements here have a multicore distribution pattern where the vulnerability is moderately low overall but has significant spatial heterogeneity. There is a considerable positive spatial correlation between vulnerable rural settlements and weakly negative correlations between exposure and adaptability and between sensitivity and adaptability, which can be attributed to the interaction among natural, human-made, and social factors. The primary factors influencing the vulnerability of rural settlements here are stone desertification and soil erosion. These results have important theoretical and practical value for enhancing the stability of rural human-land systems in karst mountains and their long-term protection.