Temporal and spatial variability in herbivory can be influenced by plant genotype, environmental conditions and their interactions. However, few studies correctly control for the relative influence of these factors. Here, I report results from a reciprocal common garden experiment designed to tease apart the effect of local environmental variation and plant genotype on the abundance of four insect species: Dasineura lupini (Felt), D. lupinorum (Gagne), Epinotia infuscana (Walsingham) and Orgyia vetusta (Boisduval). Full-sib/half-sib families of Lupinus arboreus Sims were made within three different natal populations; replicates from each lupine family then were transplanted back into common gardens located in each of the three parental populations. In two separate years I measured how insect density varied between local environments, within a population of related lupine and among the three populations of lupine. For each insect species, local environment influenced density substantially and microsite variation within environments explained a significant amount of variation. Two insect species (E. infuscana and O. vetusta) congregated on plants originating from the same parental population more than lupine from other populations, while the other two insect species showed this pattern only in specific environments. Even though these insects were differentially abundant on lupine from the three natal populations, they rarely discriminated among individual genotypes within a population. Thus, insect density was affected by environmental factors unique to each site and partly by genotypes originating from the same natal origin, but rarely by fine-scale differences among genotypes from within one parental population.