A late Pliocene (3.36–3.06 Ma) exposure of the Tangahoe Formation on the North Island of New Zealand preserves close fossil relatives of many extant seabird clades. Here, we report an extinct member of the little penguin (Eudyptula Bonaparte, 1856) lineage from the Tangahoe Formation—the smallest extinct crown penguin yet known. Eudyptula wilsonae n. sp. is based on the nearly complete skulls of an adult and a fledged but immature individual. Both skulls show more slender proportions than modern little penguins and precede genome-derived estimates for the divergence between Eudyptula minor minor Forster, 1781 (endemic to New Zealand) and Eudyptula m. novaehollandiae Stephens, 1826 (native to Australia and recently established in New Zealand). This raises the possibility that the fossil taxon represents a lineage directly ancestral to extant little penguins. Our results support a Zealandian origin for little penguins, with subsequent Pleistocene dispersal to Australia and a more recent Holocene range expansion of Eudyptula m. novaehollandiae back into New Zealand.