Aphids are major pests of African indigenous vegetables. Information on the genetic diversity and the role of host crop and environmental differentiation in their diversity in East Africa is scanty. The knowledge on genetic diversity is a critical component in the development of sound and sustainable integrated pest management strategy, from detection to control. A portion of the mitochondrial cytochrome c oxidase subunit I (COI) gene was used to characterise the species of aphids on amaranth and nightshades at different agro-ecological zones of Kenya and Tanzania. Aphid samples were collected in localities growing the vegetables in low, mid and high altitude agro-ecological zones. Total DNA was isolated and amplified using universal barcoding primers targeting the 5′ end of the COI barcode region. Nucleotide sequences of the COI barcode, using the Basic Local Alignment Search Tool model, found high homology to four species of aphids: Myzus persicae, Aphis fabae, Aphis craccivora and Macrosiphum euphorbiae. Three subspecies of the A. fabae were also detected. Intraspecific diversity depicted M. euphorbiae having the lowest value, while A. fabae showed the highest diversity. Interspecific diversity between A. fabae and A. craccivora was the lowest while between A. craccivora and M. persicae it was the highest. The phylogenetic tree showed each species clustering together irrespective of the host crop or site where collected. Principal component analysis and haplotype network analyses confirmed these results. Low genetic diversity revealed by COI suggests that the environment or host crop contribute less to the genetic diversity of aphids in both countries.