Spermatogenic failure is believed to be a major cause of male infertility. The establishment of a testis organoid model would facilitate the study of such pathological mechanisms and open the possibility of male fertility preservation. Because of the complex structures and cellular events occurring within the testis, the establishment of a compartmentalized testis organoid with a complete spermatogenic cycle remains a challenge in all species. Since the late 20th century, a great variety of scaffold-based and scaffold-free testis cell culture systems have been established to recapitulate de novo testis organogenesis and in vitro spermatogenesis. The utilization of the hydrogel scaffolds provides a 3D microenvironment for testis cell growth and development, facilitating the reconstruction of de novo testis tissue-like structures and spermatogenic differentiation. Using a combination of different strategies, including the use of various scaffolding biomaterials, the incorporation of the living cells with high self-assembling capacity, and the integration of the advanced fabrication techniques, a scaffold-based testis organoid with a compartmentalized structure that supports in vitro spermatogenesis may be achieved. This article briefly reviews the current progress in the development of scaffold-based testis organoids while focusing on the scaffolding biomaterials (hydrogels), cell sources, and scaffolding approaches. Key challenges in current organoid studies are also discussed along with recommendations for future research.