Germ cells are the common cells of origin for the two different types of gametes, sperm and eggs. In vertebrates so far examined, the sex of germ cells is determined by gonadal somatic cells. However, influenced by the somatic cells, how germ cells adopt their sexual fates by intrinsic factors has long been unclear in vertebrates. We recently identified forkhead box L3 (FOXL3) as a germ cell-intrinsic factor involved in the sperm–egg fate decision in the teleost fish medaka (Oryzias latipes). On the basis of the results obtained by the analysis of foxl3/FOXL3 expression and loss-of-function mutants, we review when and how germ cell sex is regulated non-cell-autonomously and cell-autonomously. We then discuss the fact that the germline sex determination pathway is genetically distinct from other essential gametogenic pathways such as meiotic entry and the establishment of germline stem cells. Another extraordinary finding in the foxl3 mutant is that functional sperm can be produced in the ovary, which provides a new notion that gametogenesis can proceed regardless of the sex of the surrounding somatic cells once the sexual identity of germ cells is established in medaka.