Predicting the host range for herbivores has been a major aim of research into plant–herbivore interactions and an important model system for understanding the evolution of feeding specialization. Among many terrestrial insects, host range is strongly affected by herbivore phylogeny and long historical associations between particular herbivore and plant taxa. For small herbivores in marine environments, it is known that the evolution of host use is sculpted by several ecological factors (e.g., food quality, value as a refuge from predators, and abiotic forces), but the potential for phylogenetic constraints on host use remains largely unexplored. Here, we analyze reports of host use of herbivorous amphipods from the family Ampithoidae (102 amphipod species from 12 genera) to test the hypotheses that host breadth and composition vary among herbivore lineages, and to quantify the extent to which nonpolar secondary metabolites mediate these patterns. The family as a whole, and most individual species, are found on a wide variety of macroalgae and seagrasses. Despite this polyphagous host use, amphipod genera consistently differed in host range and composition. As an example, the genus Peramphithoe rarely use available macrophytes in the order Dictyotales (e.g., Dictyota) and as a consequence, display a more restricted host range than do other genera (e.g., Ampithoe, Cymadusa, or Exampithoe). The strong phylogenetic effect on host use was independent of the uneven distribution of host taxa among geographic regions. Algae that produced nonpolar secondary metabolites were colonized by higher numbers of amphipod species relative to chemically poor genera, consistent with the notion that secondary metabolites do not provide algae an escape from amphipod herbivory. In contrast to patterns described for some groups of phytophagous insects, marine amphipods that use chemically rich algae tended to have broader, not narrower, host ranges. This result suggests that an evolutionary advantage to metabolite tolerance in marine amphipods may be that it increases the availability of appropriate algal hosts (i.e., enlarges the resource base).