A decrease in the amount of crop damage by Japanese macaques (Macaca fuscata) is often defined as the primary official goal for conflict resolution. Recent government-led countermeasures have resulted in a decline in the amount of such damage, at least per the government statistics; however, residents rarely recognize this decline as a success of conflict management. We hypothesized that this unexpected perception of residents is because crop damage has changed not only quantitatively but also qualitatively, thereby affecting their perceptions of damage. We verified this hypothesis in terms of temporal variations influencing the psychological pressure on residents—that is, the geographical locations where severe crop damage occurred and the characteristics of macaque troops including their attitude toward residents—determined using multivariable analyses with questionnaire surveys and geographical information in Yamagata Prefecture, northern Japan. The results showed that the areas with severe crop damage had expanded to urban agriculture land, with a higher density of human population, and that the troop characteristics had changed substantially to facilitate this expansion. These findings support our hypothesis and indicate that the current attention of decision-makers is being, often and intensively, directed toward only the apparent total amount of damage.