During the first 35 years of the Green Revolution, Chinese grain production doubled, greatly reducing food shortage, but at a high environmental cost. In 2005, China alone accounted for around 38% of the global N fertilizer consumption, but the average on-farm N recovery efficiency for the intensive wheat—maize system was only 16–18%. Current on-farm N use efficiency (NUE) is much lower than in research trials or on-farm in other parts of the world, which is attributed to the overuse of chemical N fertilizer, ignorance of the contribution of N from the environment and the soil, poor synchrony between crop N demand and N supply, failure to bring crop yield potential into full play, and an inability to effectively inhibit N losses. Based on such analyses, some measures to drastically improve NUE in China are suggested, such as managing various N sources to limit the total applied N, spatially and temporally matching rhizospheric N supply with N demand in high-yielding crops, reducing N losses, and simultaneously achieving high-yield and high NUE. Maximizing crop yields using a minimum of N inputs requires an integrated, interdisciplinary cooperation and major scientific and practical breakthroughs involving plant nutrition, soil science, agronomy, and breeding.