In a rice (Oryza sativa L.)-wheat (Triticum aestivum L.) rotation system, a study was conducted to determine the effects of different fertilization regimens (no fertilization, replacement of a portion of chemical fertilizer with composted pig manure, chemical fertilizer only, and straw return combined with chemical fertilizer) on the weed communities and wheat yields after 4 and 5 yr. The impact of the long-term recurrent fertilization regimen initiated in 2010 on the composition and diversity of weed communities and the impact of the components and total amount of fertilizer on wheat yields were assessed in 2014 and 2015. Totals of 19 and 16 weed species were identified in experimental wheat fields in 2014 and 2015, respectively, but the occurrence of weed species varied according to the fertilization regimen. American sloughgrass [Beckmannia syzigachne (Steud.) Fernald], water starwort [Myosoton aquaticum (L.) Moench], and lyrate hemistepta (Hemistepta lyrata Bunge.) were adapted to all fertilization treatments and were the dominant weed species in the experimental wheat fields. The greatest number of weed species were observed under the no-fertilization treatment, in which 40% of the weed community was composed of broadleaf weeds and the lowest wheat yields were obtained. With fertilizer application, the number of weed species was reduced, the height of weeds increased significantly, the density of broadleaf weeds was significantly reduced, the biodiversity indices of weed communities decreased significantly, and higher wheat yields were obtained. Only the chemical fertilizer plus composted pig manure treatment and the chemical fertilizer-only treatment increased the density of grassy weeds and the total weed community density. The treatment with chemical fertilizer only also resulted in the highest density of B. syzigachne. Rice straw return combined with chemical fertilizer yielded the lowest total weed density, which suggests that it inhibited occurrence of weeds. The different fertilizer regimens not only affected the weed species composition, distribution, and diversity, but also the weed density. Our study provides new information from a rice-wheat rotation system on the relationship between soil amendments and agricultural weed infestation.