We describe a partial skeleton of a new stem group representative of the Sphenisciformes from the mid- Paleocene Waipara Greensand in New Zealand, which represents the best-preserved and most complete Paleocene penguin found so far. Sequiwaimanu rosieae, n. gen. et sp., is the fourth penguin species from the Waipara Greensand, which previously yielded two species that were assigned to the taxon Waimanu, in addition to leg bones of an unnamed giant penguin. Among other features, the new species is characterized by an articular facet for the furcula on the apex carinae of the sternum, which is unknown from other sphenisciforms. We perform detailed comparisons with the species assigned to Waimanu and show that the type species Waimanu manneringi differs in tarsometatarsus morphology from its putative congener ‘W.’ tuatahi, which is here assigned to the new taxon Muriwaimanu. Sequiwaimanu rosieae exhibits a more derived morphology than Muriwaimanu tuatahi, but its exact affinities to W. manneringi are unresolved owing to the incompletely known osteology of the latter species. With S. rosieae being more closely related to the crown group than M. tuatahi, shared characteristics of the two taxa are likely to be plesiomorphic for sphenisciforms. Although the skeletal morphology of these sphenisciform stem species shows some similarities to plotopterids (i.e., wing-propelled diving seabirds from the North Pacific Basin) in some characters, the stem group sphenisciforms from the Waipara Greensand are clearly distinguished from plotopterids.