Plant–pollinator interactions are essential for both actors and are an important ecosystem service for natural and agricultural systems (Klein et al., 2007; Vanbergen and Insect Pollinators Initiative, 2013). However, climate change may put this critical interaction at risk (González-Varo et al., 2013; Scheffers et al., 2016). For example, the spring ephemeral Corydalis ambigua Cham. & Schltdl. is flowering earlier in mountain habitats due to earlier snowmelt, but the pollinators (Bombus hypocrita and B. hypnorum koropokkrus), whose emergence is associated with soil temperature, are not always synchronized with flowering commencement, resulting in lower seed production (Kudo and Ida, 2013). Here I review methods used to assess how climate change affects this interaction with a focus on flowering phenology of plants and emergence or floral visitation phenology by pollinators.

Environmental alterations associated with climate change include the following: seasonally distinct temperature increases with greater winter warming; geographically distinct temperature increases that are greatest in the Arctic followed by other northern regions; altered precipitation resulting in changed annual means and altered seasonal variation; increasing CO₂ and N₂O; and greater frequency of extreme events (IPCC, 2014). The unpredictability and geographic variation of these abiotic consequences of climate change make it very difficult to make global predictions regarding the effect on species. These changes are expected to affect the timing of developmental events (phenology) of organisms because some environmental cues (CO₂, ozone, temperature, and precipitation) are changing (Hughes, 2000; Hayes et al., 2012; Kumar et al., 2012). There is growing evidence that many species are shifting their phenologies, including plants and their pollinators, which could result in mismatching between the timing of flowering and pollinator foraging (Bertin, 2008; Hegland et al., 2009; Chambers et al., 2013; Monahan et al., 2016). Furthermore, species distributions are changing as habitat suitability is altered by these changes (Hickling et al., 2006; Franzén and Öckinger, 2012). Thus, the actors of plant–pollinator interactions are facing spatial and temporal shifts in their abiotic environment and are responding with biotic shifts in spatial (distribution) and temporal (phenology) dimensions. Because of the complexity associated with temporal and spatial variation in the effects of and the subsequent responses to climate change, creative approaches are needed to quantify and comprehend the consequences for plant–pollinator interactions.

Here I review some approaches used to assess the impact of climate changes on plant–pollinator interactions, with a focus on phenological changes. My goal is to present a wide diversity of approaches, particularly creative or unique methods. I included many studies that focus on either plants or pollinators (without explicitly assessing interactions) because these studies also have unique and useful methods. As part of this review, I very briefly discuss limitations or strengths associated with different general approaches. My aim is to provide a resource of approaches with suggestions as to what is still needed and to provide inspiration for further development.

MATERIALS AND METHODS

To identify papers used in this review, I used the search engine Web of Science (1988 to present) using the following sets of key words: “climate change” and “pollinator^*” (371 references); “climate change” and “flower^* phenology” (357); “climate change” and “plant–pollinator^* (134)”; “climate change” and “plant–pollinator^* and “networks” (32). Using Web of Science, I also did a forward search on two highly cited papers (Memmott et al., 2007 and Burkle et al., 2013). From these search results, I read the abstracts to determine if they were potentially a good fit for this review. After reading the full text of over 500 potential papers, I identified a subset where the authors clearly assessed impacts of climate change. Papers were excluded if they only discussed implications of their results for climate in the conclusion section of the paper. From this set, I identified 323 papers for more careful consideration, and from their citations, more than 50 additional papers were included.

This review is organized into four major sections based on the primary taxonomic focus (plant or pollinator) or type of interactions (network or not). I have separated general interaction studies, which can range from one plant species and its pollinator to the entire community of plants and pollinators, from network studies, which are community interaction studies but typically take a more holistic approach toward their questions and statistical approaches (e.g., Olesen et al., 2008; Encinas-Viso et al., 2012). Within each major section, results are organized by the types of approaches used (for an overview see Fig. 1). For three of these major sections, tables provide further details of discussed examples, as well as unique aspects of additional studies beyond those discussed in the text ( Appendices S1 (apps.1700012_s1.docx),  S2 (apps.1700012_s2.docx), and  S3 (apps.1700012_s3.docx)). Within each of these tables, the studies are organized into the same subsections as the main text.

PLANT-FOCUSED STUDIES

The measurement of flowering phenology includes the onset, end, and duration of flowering, abundance of flowers, and the flowering community (CaraDonna et al., 2014). Depending on the method chosen, these different aspects of flowering phenology can be quantified.

Historic records—Herbarium specimens are available for a wide diversity of species and are potentially available across a species' range; these are increasingly accessible electronically (e.g.,  www.idigbio.org). Other types of historical records include naturalists' notes, dated photos, and long-term observations (Primack and Miller-Rushing, 2012).

Herbarium specimens and current observations are often combined with temperature data to determine if phenology is changing in relation to temperature ( Appendix S1: Table A (apps.1700012_s1.docx)). For example, herbarium specimens and field observations of Hungarian terrestrial orchids enabled assessment of phenology of species for which conservation concerns have limited the number of more recent herbarium specimens (Molnár et al., 2012). Molnár et al. (2012) included phylogenetic relatedness and several life-history traits to determine which predict the responsiveness of flowering phenology to temperature changes ( Appendix S1: Table A (apps.1700012_s1.docx)).

To determine whether herbarium specimens are a valid approach for assessing change in flowering phenology in response to climate, two formal analyses have compared herbarium specimens to long-term field observations (Robbirt et al., 2011; Davis et al., 2015). Comparison of temperature (3-mo mean) and peak flowering time using herbarium specimens (1848–1958) of an orchid revealed the same relationship (slope) with mean temperatures as recorded observations of peak flowering from 1975–2006 in one site (Robbirt et al., 2011). Comparing these approaches for plant communities, Davis et al. (2015) compared estimates of flowering onset from herbarium specimens (late 1800s to mid-1900s) with several field observation data sets of flowering onset (1852–2013) for 20 species near Concord, Massachusetts, USA. Analyses of the slope of individual species from a regression of temperature and flowering onset from specimens or field observations (as the response variable) did not reveal any significant differences. However, the estimates did differ when examining all of the slope estimates collectively for many species, which was attributed by Davis et al. (2015) to limited data from the long-term field observations.

Some limitations are associated with the use of herbarium specimens, particularly the inability to assess all aspects of the phenology. The assumption is made that the flowering on the herbarium specimen reflects peak flowering of that population and year, but if the collectors obtained specimens at either the onset or end of the flowering period, this assumption is not met (Primack et al., 2004). Primack et al. (2004) classified specimens at peak flowering for a location if over 50% of the flowering buds on a specimen were open when collected, which provides a method to standardize interpretation for herbarium specimens. If herbarium samples are from a broader region, where seasonal differences could make comparisons inaccurate, Lavoie and Lachance (2006) suggest making an adjustment for the collection date (to estimate flowering peak) using location differences in snowmelt dates to account for seasonal differences. While there are some limitations, creative approaches incorporating herbarium specimens can be a strong asset, as demonstrated in these examples and those in  Appendix S1: Table A (apps.1700012_s1.docx), which all found that at least some species are now flowering earlier.

Field observations—Recorded observations of flowering phenology, often collected by one person over many years, vary in the level of detail but typically involve observations of the entire plant community ( Appendix S1: Table B (apps.1700012_s1.docx)). Fitter and Fitter (2002) recorded the date of flowering onset for 557 species over a 47-yr period in south-central England. Focusing on the 385 species that were present in the data set for at least 23 yr, they assessed the extent of phenological shifts in 10-yr increments to detect when changes have been the greatest. Inclusion of temperature (monthly means) in a further analysis enabled them to determine which month was the most significant predictor of deviation in first flowering day (Fitter and Fitter, 2002). As illustrated with this example and others ( Appendix S1: Table B (apps.1700012_s1.docx)), these long-term observation data sets have been a valuable resource for documenting the degree of shifting of flowering onset over the time that the climate has been changing, but there is limited availability of this type of data. Furthermore, often only flowering onset is recorded, limiting assessment of any temporal shift in duration and abundance of flowers, which are likely key for pollinator interactions (for creative analysis of this type of data and discussion of the limitations, see Diez et al., 2012).

Because the rate of temperature and precipitation change is greatest in alpine and arctic environments (IPCC, 2014), researchers have focused on these habitats. Ecologists have been studying the dynamics of plant reproductive biology and plant–pollinator interactions (see below) in these same habitats, resulting in very detailed long-term data sets that are now being analyzed to understand the impacts of climate change ( Appendix S1: Table B (apps.1700012_s1.docx)). Of note are the long-term study plots established in 1973 by Inouye (2008) in dry to wet mountain meadows at the Rocky Mountain Biological Laboratory (Crested Butte, Colorado, USA). Inouye (2008) obtained a very comprehensive record of the community flowering phenology and abundance by counting the number of flowers opened and number of ramets every two to three days over the growing season for over 35 yr. Snowmelt date was also tracked over this time, and this cue for flowering onset for many alpine species changes with the warming climate ( Appendix S1: Table B (apps.1700012_s1.docx)). These very detailed data (1973–2012) provide evidence of phenological shifts for many species and which aspects of phenology (onset, peak, and last flower) shifted in response to the changing climate (CaraDonna et al., 2014). CaraDonna et al. (2014) with further analysis quantified the extent of changes in coflowering between species over a 39-yr period.

Fig. 1.

A conceptual overview to general approaches used for addressing some of the major questions regarding the effects of climate change on phenology of plants, pollinators, and their interactions. The organization of this figure is reflected in the text and  Appendices S1 (apps.1700012_s1.docx),  S2 (apps.1700012_s2.docx), and  S3 (apps.1700012_s3.docx). The questions for the plant-focused studies are as follows: (1) Are plants flowering earlier in response to earlier spring/summer?; (2) Is the peak and duration of flowering changing in response to temperature or precipitation changes?; (3) Are there distinctions in how species that flower in different seasons respond phenologically to climate changes?; (4) What are the specific environmental cues underlying the observed phenological changes? Do cues differ across a species' range?; and (5) Are related species more likely to have similar phenological responses to climate change? The questions for the pollinator-focused studies are: (6) Are pollinators emerging or migrating earlier in response to earlier spring/summer?; (7) Is the peak and duration of activity changing in response to temperature or precipitation changes?; and (8) Are there changes in diapause? If so, what are the impacts on activity and survivorship? The questions for the interactions-focused studies are: (9) Are plants and their pollinators responding to the same extent in phenological changes to the same environmental changes? Is there any evidence of limited or no overlap in phenology between plants and pollinators resulting in mismatching?; (10) Are early- and later-flowering species responding differently to the changing climate? Has the altered flowering phenology of plants resulted in seasonal resource gaps for pollinators?; and (11) Does earlier shifting of phenology increase exposure to extreme spring frost events? The questions for the network studies are: (12) Is there redundancy among species interactions within networks? Are specialized species (i.e., those with limited links) more likely to be negatively impacted with climate changes?; and (13) Are related species more likely to decline with climate change?

Extensive collections of data for plants and animals have been organized by phenological networks in many countries where the data are often collected by volunteers using particular protocols and for specific species (e.g.,  www.naturescalendar.org.uk). Using statistical and modeling approaches, researchers have been able to assess the impact of changing temperature on flowering onset, as well as address other questions using these data sets ( Appendix S1: Table B (apps.1700012_s1.docx)). To develop a statistical approach to assess the relative phenology changes in 276 U.S. National Parks and Preserves across the contiguous United States and Alaska, Monahan et al. (2016) used species-specific spring indices for three nonnative indicator species (a cloned lilac cultivar and two honeysuckle species commonly found in these locations). Spring indices included first leaf and first bloom indices, which are based on the number of accumulated growing degree-days. The authors used variable time interval lengths (10, 20, and 30 yr) of historic temperatures across the parks, with the spring indices to determine the spatial patterns of recent variability compared to historic variability in spring onset in the parks and preserves (Monahan et al., 2016). While these studies provide insights to changes in the phenology of these varieties, the use of nonnative indicator species for monitoring by the phenological network sampling groups could limit the application of the results from these models to native communities.

Many long-term studies rely on a diversity of participants to collect phenological data. This approach has the positive broader educational impact of including citizen scientists, but data quality could be an issue in these cases (Gordo and Sanz, 2009). Use of methods for removal of biased data along with sufficient training of the volunteers can improve quality (Gordo and Sanz, 2009; Tooke and Battery, 2010; Schwartz et al., 2012; Havens and Henderson, 2013; Gonsamo and D'Odorico, 2014). Digital photography provides a visible record of phenology, allowing removal of potential bias (Crimmins and Crimmins, 2008). A formal assessment of the accuracy of trained citizen scientists compared to a professional ecologist found that some phenological stages and species were not correctly recorded, but overall volunteers correctly identified the phenological stage 91% of the time (Fuccillo et al., 2015). Formal assessment of volunteers' data could inform project organizers as to the common errors being made and what type of further training may be necessary. Citizen scientists are likely to be key for long-term data collection, with a concomitant educational benefit that is invaluable for them when they actively quantify data likely to reveal the impact of climate change; thus, the development of training approaches and further tools that can later verify data will be essential.

While a complete assessment of flowering phenology would quantify the onset, peak, and duration of flowering, the above approaches often only assess flowering onset. To validate if flowering onset is sufficient to estimate the flowering phenology for entire populations, Miller-Rushing et al. (2008) modeled the impact of population size and sampling frequency using the data sets of recorded observations from Gothic, Colorado, and Concord, Massachusetts, USA. They found that flowering onset was a fair predictor of flowering peak in the Colorado data set. Changes in population size affected estimates of flowering onset, with declining population size delaying onset but increasing population size advancing the beginning of flowering. Changes in sampling frequency affect the ability to detect changes in flowering onset. Miller-Rushing et al. (2008) recommend avoiding the confounded effects of population size and sampling frequency when obtaining estimates of flowering onset by quantifying the entire flowering distribution.

A newer effort by a group of cooperative entities led by the National Ecological Observatory Network (NEON) has been designed to collect data for 30 yr across 47 sites representing multiple habitats, using standardized protocols to quantify plant phenology and environmental information (Elmendorf et al., 2016). Projects such as NEON have advantages including rigorous experimental design and extensive types of data, which should enable clear tests of hypotheses with very reliable results and robust conclusions. While substantial funding is required to sustain the network, the products of projects from NEON may indicate the best methods for quantifying phenology changes that could be reapplied to smaller-scale studies.

Experimental—Experiments in which the abiotic conditions that are expected to shift with climate change are manipulated in natural or experimental communities and growth chambers can demonstrate effects on phenology ( Appendix S1: Table C (apps.1700012_s1.docx)). Alternatively, researchers have used natural variation across an environmental gradient, sometimes combined with reciprocal transplants (Etterson, 2004a; Li et al., 2016; see next subsection). Using formal genetic designs, evolutionary responses and phenotypic plasticity of responses to abiotic changes have also been determined ( Appendix S1: Table C (apps.1700012_s1.docx); Etterson, 2004a, 2004b).

Looking at one of these studies' approaches in greater detail, Xia and Wan (2013) conducted a four-year study in a semiarid temperate steppe habitat of Inner Mongolia, China, experimentally altering temperature and nitrogen. Temperature was increased during the growing season using infrared radiators for treatments that differed in the timing of warming (daytime only, nighttime only, continuous warming, or no warming [control]). Nitrogen was added to half of the control and continuous warming plots. Both the rate of warming and addition of nitrogen were designed to simulate the known changes in this location. Xia and Wan (2013) assessed the reproductive phenology of eight focal species. For forbs, this included: unopened buds, open flowers, postanthesis flowers, initiated fruits, expanding fruit, and dehisced fruit. Their assessment of phenology involved fitting their phenology scores (based on observations) into the Richard growth equation (Xia and Wan, 2013), enabling a more comprehensive evaluation of climate change. The realistic simulation of the environmental changes enables clear assessment of the implications of their results. The broader type of phenology data collected (flowers and fruits) allowed for more inclusive assessment of the reproductive phenology. Because the effects of year and year by warming treatment interaction on phenology were both significant, longer-duration studies should be implemented when possible.

The duration of a phenological study may limit a researcher's ability to correctly measure effects of any treatment. The response of the plants to warming can be delayed for years with an accumulative effect expressed later, and variation among years also limits our ability to detect a treatment effect (Hoffmann et al., 2010). Plant phenology responses to long-term warming can decline as other factors limit further response (Kremers et al., 2015; Barrett and Hollister, 2016;  Appendix S1: Table C (apps.1700012_s1.docx)). The results of these studies suggest that long-term investigations are essential for understanding the consequences of experimental warming. Other environmental factors (e.g., drought), to my knowledge, have not been experimentally altered in long-term studies. Furthermore, the biological implications of the results of these experimental approaches depend on how well the specific study simulates the expected environmental changes.

A unique approach by Thomann et al. (2015a, 2015b) used stored seeds (1992) compared to recently collected seeds (2010) from a region that has seen a pollinator decline. These seeds were used in two common garden experiments to compare flowering phenology and plant mating system traits (Thomann et al., 2015a, 2015b;  Appendix S1: Table C (apps.1700012_s1.docx)). Except for potential problems with lower germination for older seeds and confounding maternal effects, this approach could provide unique insights regarding how floral traits may change with pollinator declines. For example, Thomann et al. (2015b) found that selection occurred for earlier flowering but not for longer duration for flowering. The potential confounding with maternal effects can be estimated by specific experimental designs or limited by growing plants in the same conditions (Shaw and Byers, 1998; Thomann et al., 2015b). Many botanic gardens (such as Chicago Botanic Garden's Dixon National Tallgrass Prairie Seed Bank) actively store seeds for long-term conservation and research, which could be used for studies of this type.

In northern regions, temperature changes are expected to be greatest in the winter; however, fewer studies have focused on winter compared to other seasons (Kreyling, 2010; IPCC, 2014). Warming winters may have specific impacts, including insufficient vernalization for flowering (Kreyling, 2010). Mu et al. (2015) used chambers with different coverings to create seasonally asymmetric warming (greater warming in the winter), resulting in lower flowering frequency ( Appendix S1: Table C (apps.1700012_s1.docx)).

A strength of these experimental studies (above and  Appendix S1: Table C (apps.1700012_s1.docx)) is that they typically quantify more aspects of flowering phenology (onset, peak abundance, duration), which are useful to determine the potential for negative impact with pollinators.

Environmental gradients—Studies involving observations or transplants across an environmental gradient can determine how the different environmental conditions affect plants, and can represent both present and future abiotic conditions (Etterson, 2004a). In the absence of long-term temporal data on flowering phenology, spatial gradients can be used to quantify how flowering phenology may change with the future climate ( Appendix S1: Table D (apps.1700012_s1.docx)). Studies with long-term observations across spatial gradients enable detection of environment-dependent phenology changes ( Appendix S1: Table D (apps.1700012_s1.docx)). Experimental spatial and temporal approaches can assess how distribution and phenological shifts will impact the potential for plant–pollinator interactions to be retained (Morton and Rafferty, 2017).

A unique approach involved reciprocally transplanting soil blocks (100 cm² with 30–40 cm depth), with the grassland community intact, into four common garden sites along an altitudinal gradient (3200–3800 m) in the Qilian Mountains, China (Wang et al., 2014). For comparison, Wang et al. (2014) moved soil blocks containing the plant community within the altitude for the four sites. Using environmental sensors, they tracked air and soil temperature along with soil moisture in the four common garden sites. This experimental design enabled comparisons of the impact of increased temperature (moved to lower elevation), decreased temperature (moved to higher elevation), and no short-term temperature change on flowering onset of six focal species that included early and later seasonally flowering species (Wang et al., 2014). The authors developed a conceptual model for considering the relative phenological responses of the plants to warming and cooling. Species from the early season responded more to cooling, while the later season species responded more to warming (Wang et al., 2014). For further studies using these experimental plots, see  Appendix S1: Table D (apps.1700012_s1.docx).

Phylogenetic considerations—Whether a species is constrained in how its phenology changes in response to climate change may depend on its evolutionary history, so some researchers have included a phylogenetic analysis in their investigation ( Appendix S1: Table E (apps.1700012_s1.docx)). A phylogenetic assessment of multiple data sets, with global representation, concluded that related species were likely to have a similar response to climate change (Mazer et al., 2013; for details on this study see  Appendix S1: Table E (apps.1700012_s1.docx)). Mazer et al. (2013) suggested that predictions could be made for unstudied species based on results from related species.

POLLINATOR-FOCUSED STUDIES

This subsection highlights some methods that could be integrated into a broader study on plant–pollinator interactions; it also includes some methods and concerns discussed in the entomological literature to bring them to a botanical audience. For an overview of field methods to measure bee diversity, see Westphal et al. (2008), in which the authors used multiple methods in a diversity of habitats and compared the results from different techniques.

Historic records—Pollinator specimens in museum collections can be used to characterize the relative rates of changes in species diversity and richness. To test if ecological or species traits predict changes in a species' abundance, Bartomeus et al. (2013) combined trait data with abundance changes using bee specimens from museums and university collections, as well as a database, to better inform how pollinator diversity could be changing over time (for details, see  Appendix S2 (apps.1700012_s2.docx)). Other studies have used museum collections of insects along with plant data (see the section on Interaction-Focused Studies below).

Field observations—The first spring activity of honeybees are cleansing flights (to excrete feces), which have been quantified by beekeepers in Poland. Comparing 25 yr of these flight records with temperature data, Sparks et al. (2010) concluded the bees were starting their flights a month earlier, which over time is associated with increased temperatures occurring earlier in the season.

Many long-term butterfly observations are from national monitoring networks and monitoring programs at research sites (see “Field observations” in  Appendix S2 (apps.1700012_s2.docx)). To determine which types of butterflies are more likely to shift their arrival time, Diamond et al. (2011) used the mean first date of appearance across transects obtained from the UK Butterfly Monitoring Scheme (weekly observations started in 1976;  http://www.ukbms.org/), along with insect species traits suggested in other studies to affect butterflies' response to climate change. Many of these are the same traits that Bartomeus et al. (2013) identified, namely diet breadth (number of host species), dispersal ability, voltinism, overwintering stage, and range size. While Diamond et al. (2011) found evidence for the date of first appearance advancing in butterflies, some of the results were not as expected. For example, butterfly species that appeared earlier exhibited narrower diets, which is unexpected as host plants may be limited at that time (Diamond et al., 2011). Assessments of which species traits are more often associated with either changes in species abundance (Bartomeus et al., 2013) or shifts in phenology (Diamond et al., 2011) can produce testable predictions for researchers looking for mismatching occurring between plants and their pollinators. Other butterfly studies have combined long-term data sets with habitat characteristics to provide further context to the species-specific phenological responses to changing temperature ( Appendix S2 (apps.1700012_s2.docx)).

Given the resources necessary for long-term monitoring, some have proposed the use of distribution records in which location and date have been recorded. Bishop et al. (2013) compared estimates of phenological parameters (mean date of the seasonal activity flight time and seasonal duration of activity) for butterflies (univoltine species only) assessed from distribution records (1995–2009) collected in Britain to observation records from the UK Butterfly Monitoring Scheme. The distribution records predicted the mean flight date correctly for 22 of 30 species, but the records were less successful in predicting flight duration (Bishop et al., 2013). Thus, if direct observations are not available, this approach using distribution records may provide some insights at least for first appearance date.

Experimental—Several experimental approaches have been used to assess if the earlier phenology of pollinators is due to faster development in the life stages of bees with warmer temperatures, if diapause is shortened or lost in response to warmer winter temperatures, and whether social changes in bees result from warmer temperature ( Appendix S2 (apps.1700012_s2.docx)).

In Northern Europe, where bees are typically in diapause during the winter, some species are now skipping diapause, or have a shortened diapause and now establish new late season colonies. If bees skip diapause, they may not survive the winter. If they have a shortened diapause, they may not survive the extreme seasonal temperature variation. To evaluate Bombus terrestris audax's cold tolerance, Owen et al. (2013) used experimental treatments consisting of exposure to low and very low temperatures, sometimes with changes in diet (pollen and nectar vs. nectar only); these treatments were designed to test physiological mechanisms that insects use for cold tolerance. They found that queens were more cold tolerant than workers, who would only survive a mild winter without diapause (Owen et al., 2013).

These studies quantify the response of bees by testing the bees at controlled but biologically relevant temperatures. This complements field studies where temperatures are less controlled, making it more difficult to draw conclusions (Owen et al., 2013;  Appendix S2 (apps.1700012_s2.docx)).

Environmental gradients—Because long-term observations are usually rare, researchers have used gradients as a substitute for time (as has been done with plants). To obtain phenological data, two studies used elevation gradients across different mountain ranges in Spain. The butterfly communities were monitored with frequent visits to 10–40 sites (De Arce Crespo and Gutiérrez, 2011; Illán et al., 2012;  Appendix S2 (apps.1700012_s2.docx)). Although these studies found an overall pattern of a delay in mean flight date with higher elevation, this pattern was not consistent for many of the butterfly species. Particularly, the seasonally later flying species were mostly synchronized in mean flight date across elevations. The lack of change with elevation was proposed to be due to the limited season at higher elevation and selection for emergence at lower temperatures (Illán et al., 2012). The use of a reciprocal transplant approach could assist in resolving the cause behind species' variable responses. These studies illustrate the limitations in using gradients as a replacement for long-term observations for butterflies.

INTERACTION-FOCUSED STUDIES

This section provides a review of approaches used in studies of plants and their pollinators. The studies differ in whether the focus is on the interaction(s) explicitly, or whether the emphasis is on the dynamics of the two groups with less direct assessment of their interactions. Studies using a network approach (the entire plant–pollinator community with an emphasis on the collective dynamics of the entire set of interactions) are in a separate section because the questions and analysis can be very different. Therefore “plant–pollinator interactions” is a more general term that can refer to a range of interactions from between one plant species and one pollinator species, to the entire community of plants and pollinators. Additional studies with further details are included in  Appendix S3 (apps.1700012_s3.docx) and organized by the general approach, as are the following subsections.

Historic records—To determine if mismatching is occurring between the presence and abundance of plants and their pollinators, researchers have combined museum and herbarium specimens with climate information and statistical models ( Appendix S3 (apps.1700012_s3.docx)). Although the focus of the study by Scheper et al. (2014) was to determine if bee species diversity and abundance has declined after agriculture intensification during the 20th century, their methods of establishing a historic record of interactions are broadly applicable to studies of plant–pollinator interactions over time. Given the concerns of bee species declines in the Netherlands, Scheper et al. (2014) used a combination of historic collections of bees from multiple museums, a long-term observational data set (European Invertebrate Survey–Netherlands [ http://www.eis-nederland.nl]), and current floral seasonal diversity in the agricultural landscape. To determine the historic record of plant–pollinator interactions using bee museum specimens, pollen was removed and mounted in glycerine jelly, identified to plant taxa using a reference collection, and the relative abundance of the identified pollen on each specimen was calculated (Scheper et al., 2014). From this information, the authors determined the interactions of both plants and bee visitors and any change over time. They determined the change in the bees' (abundance and range) population dynamics over time using the monitoring data for the Netherlands. These data were combined in a set of models to determine which predictor best explained the decline of bees. The authors concluded that the change in the bees' interactions with plants has led to the decline in bee abundance (Scheper et al., 2014). Researchers are using a diversity of creative methods with historical specimens to obtain information about past interactions.

Field observations—Studies differ in the extent of the community considered (from a few focal species to most of the plant and pollinator community), study duration, environmental cues measured, and consideration of consequences from phenology shifts ( Appendix S3 (apps.1700012_s3.docx)). To explore if butterflies (Pieris rapae) and Prunus tree species would respond the same to a series of different environmental cues, Doi et al. (2008) designed a statistical approach. They used the flowering phenology data set collected by the Japan Meteorological Agency from 1953 to 2002 at the Nobeyama Solar Radio Observatory (Nagano, Japan); this data set is based on the date when five to six flowers are open (“unfolding”), a date when ∼80–100% of the flowers are open (“full flowering”), and “flowering speed” (difference in days between full flowering minus unfolding). The Japan Meteorological Agency also recorded the arrival date of the butterfly Pieris rapae for the same years. For climate data, the authors used air temperature and total precipitation from the same location. In their analyses, they examined the relationships between phenology of the plants or butterflies as the response variable with temperature and precipitation as predictor variables, using 13 different time-length windows from the proceeding 365 d to the phenological event. This allowed Doi et al. (2008) to test for the species-specific response to environmental cues of different duration. Differences in environmental cues for phenology, as found in this study, could lead to mismatching in timing between plants and their pollinators.

As an explanation for the decline of some long-tongued bee species in the Rocky Mountains of Colorado, USA, Miller-Struttmann et al. (2015) proposed and tested for functional mismatches. Using museum specimens of bees (from 1966–1969) and recently collected bees in the same area, the authors measured tongue length to determine if it changed within species over time. To determine if the bumblebee species composition had changed due to competition from immigrating subalpine species, they resampled the bumblebee community. To assess if flower depth changed, they compared herbarium specimens and historic data sets to their measurements in the current populations, where some of the same plots were sampled as previously (Miller-Struttmann et al., 2015). This study illustrates how the use of historical and current data, combined with rigorous statistical analysis, can reveal mechanisms behind the changes in bee species.

Experimental—Some approaches have altered the timing of flowering and then observed plant–pollinator interactions and the consequences for plant reproduction ( Appendix S3 (apps.1700012_s3.docx); Rafferty and Ives, 2011). Other investigations have altered the pollinator community and assessed the consequences ( Appendix S3 (apps.1700012_s3.docx); Brosi and Briggs, 2013), while still others have altered drivers of climate change (carbon dioxide and nitrous oxide), which may directly affect flowering phenology and floral traits, and thus plant–pollinator interactions ( Appendix S3 (apps.1700012_s3.docx); Hoover et al., 2012).

Altering flowering—A comparison of a historic data set (1935– 1945) to a more recent data set (1977–2007) revealed that some plant species of the prairies and woodlands of Wisconsin, USA, have shifted to earlier flowering onset (i.e., historically advanced) but others have not changed (Rafferty and Ives, 2011). Using 14 of these plant species (including six historically advanced species), Rafferty and Ives (2011) proposed to test whether the impact of earlier flowering on interactions with their pollinators differed between these groups (historically advanced vs. no change). The authors advanced and delayed the flowering onset of these species by raising seedlings in warmer (with supplemental lighting) or cooler greenhouses. To determine if the differences in flowering onset would alter their attractiveness to pollinators, they quantified the sucrose content and volume of nectar. Using these plants with experimentally delayed, advanced, or unchanged flowering (of historically advanced or unchanged species), arrays were established in an arboretum with prairie and woodland habitats to assay pollinator visitation rate and diversity over the season. This experimental design allowed the authors to determine if specific species were constrained in their ability to alter flowering onset as a result of a lack of visitation by their pollinators. For example, if no pollinators visited, the plants would not set seed, thus flowering onset would be constrained by pollinators. They found that visitation to early flowers was higher for the historically advanced species, suggesting species that have advanced their phenology are less constrained by pollinators (Rafferty and Ives, 2011).

Altering pollinators—It has been proposed that pollinator networks have enough redundancy so that loss of species will result in functional replacement by other species. To test if the loss of pollinator species (a Bombus species) will impact plant reproduction (of Delphinium barbeyi (Huth) Huth) or if other species of Bombus will fill in the gap due to functional redundancy in the Bombus community, the most abundant Bombus species were temporally removed via netting from field plots in the Rocky Mountains (Colorado, USA) (Brosi and Briggs, 2013). To determine floral fidelity of the bumblebees, the authors assessed the Bombus species behavior and pollen load in plots with and without bumblebee removal. To determine if the pollen deposited on plant stigmas changed with bumblebee removal, they bagged unopened flowers of D. barbeyi, and then bags were removed at the start of the bumblebee observations. In the field, stigmas were removed and prepared for counting and identification of the pollen. Seeds of these flowers were counted. With this approach, the authors could detect a significant impact due to short-term removal of the most abundant Bombus species. In the removal plots, remaining Bombus species visited a greater diversity of plant species, which increased the species diversity of pollen deposited on stigmas and lowered seed production of D. barbeyi (Brosi and Briggs, 2013).

These experimental studies (above and in  Appendix S3 (apps.1700012_s3.docx)) illustrate that carefully designed manipulations and assays have the potential to provide insights that are not possible without manipulations.

Environmental gradients —Because species interactions may depend on the abiotic environment, studies across environmental gradients or contrasting microhabitats may provide essential insights on predicting how plant–pollinator interactions could be altered with climate change ( Appendix S3 (apps.1700012_s3.docx); Maron et al., 2014). Studies differ in whether manipulations are used or historical data are included with current observations; reciprocal transplants enable assessment of local adaptation ( Appendix S3 (apps.1700012_s3.docx)).

Often observers of plants record the seasonal phenology of pollinator activity. However, without observations of pollinator activity made separately from the plants, it cannot be determined if they are responding to the same or different cues. To separately quantify pollinator activity from that of plants, Forrest and Thomson (2011) used nest traps for bees and wasps in 14 sites across an elevation gradient in the Rocky Mountains (Colorado, USA) (2007–2010, see paper for different trap designs). To verify that insects are responding to local conditions and that results were not reflecting local genotype responses, they reciprocally transplanted nest traps between two of the sites, moving the nest traps in the autumn to expose the insects to winter temperatures of the other site. To assess the effect of snowmelt at one of the higher-elevation sites, they positioned nest traps at different heights. For all of the nest traps, they quantified emergence of pollinators (into vials that were frequently checked) and air temperature (recorded hourly at the nests using dataloggers). To monitor the flowering phenology, at each of the sites they established and frequently monitored three to four belt transects. Plant phenology was quantified by counting the number of open flowers on the same days the nest traps were checked. Their experimental design allowed the authors to build thermal threshold models of the bees and wasps, as well as plants. Overall, Forrest and Thomson (2011) concluded that the insects had higher temperature requirements than the plants, but plants were more likely to respond to warming temperature by flowering earlier. Pollinator phenology studies would benefit by more direct methods to determine the cues for the pollinators, as demonstrated by this study.

Spring climate—Extreme weather events are expected to increase in frequency with climate change, including increasing spring temperature variation resulting in later spring frosts (IPCC, 2014). Frosts are also more likely to occur as protective snow melts earlier or is not present when earlier-flowering spring plants may be particularly sensitive to extreme temperatures (Inouye, 2000; Augspurger, 2013;  Appendix S3 (apps.1700012_s3.docx)).

CaraDonna and Bain (2015) designed an approach to experimentally test if frost sensitivity is different between vegetative vs. reproductive structures of plants and if there are seasonal differences among plants. Using pairs of congeneric species (one seasonally earlier-flowering species and one seasonally later-flowering species) from subalpine habitat of the Rocky Mountains in Colorado, USA, healthy leaves and flowers were randomly collected from multiple plants for temperature assays. Using temperatures similar to the minimum temperature after snowmelt in early spring at this site (0°C to −10°C), they tested samples in a temperature chamber. For all trials, they started at 5°C, then lowered the temperatures to the target (over a 6-h period), held the target temperature for 2 h, and increased the temperature in 1 h. Frost damage was assessed visually (wilting and discoloring) and compared to untreated cut leaves or flowers. Flowers were more sensitive than leaves to the warmer test temperatures but both showed damage at the lower temperatures. The analyses comparing the results of early- vs. later-flowering species were variable; thus, whether earlier-flowering species are more sensitive to frosts cannot be concluded (CaraDonna and Bain, 2015). This experiment nicely complements field studies ( Appendix S3 (apps.1700012_s3.docx)) in which long-term observational data sets that include frost events have provided insights to the impacts on flower and pollinator visitation.

NETWORK STUDIES

A more comprehensive approach to plant–pollinator interactions is evaluation of the dynamics of plant–pollinator networks. Changes in phenology can impact the structure of these networks (Olesen et al., 2008; Encinas-Viso et al., 2012). To address the extent of specialization present in the plant–pollinator interactions of a network, Petanidou et al. (2008) used a data set consisting of four years of recorded year-round observations from a nature reserve in Daphni, Greece. In part, they compared annual networks (one year of data) vs. a network from the complete four years of data to assess the network properties. Petanidou et al. (2008) found that specialized interactions changed over time, suggesting that interactions in pollinator networks can be temporally plastic. If this is generally true, then the extent of specialized interactions may be rare, and effects from habitat fragmentation to climate changes could have less of a negative impact than expected. This approach of network assessment needs to be applied to additional network data sets and longer-term data sets for which the climate has changed over the monitoring period.

Because species can respond to climate change temporally through their phenology or spatially by changing geographic distributions, plant–pollinator networks need to be assessed on these scales. In a review on spatial and temporal dynamics of plant–pollinator networks, Burkle and Alarcón (2011) also conclude that the interactions and other structural aspects of networks could be more variable (plastic) across these scales than previously thought.

Relative to the other types of studies in this review, fewer studies on pollinator networks have explicitly examined the effects of climate change.

Field observations—Two complementary investigations (Memmott et al., 2007; Burkle et al., 2013) addressed changes in networks based on the same historic observations, but these studies provided different insights. Robertson (1929) reported flowering and pollinator visitation from 1884 to 1916 in central/western Illinois, USA, for 429 plant species and 1420 pollinators in prairies and woodlands. To explore how phenological shifts in plants and pollinators can impact the pollinator community, Memmott et al. (2007) used Robertson's data set, along with the expected climate changes for this region of Illinois and estimates of the phenological responses of species due to climate changes in northern regions of the United States and United Kingdom. They simulated the climate change effect via randomly advancing phenology (1, 2, or 3 wk) for plant taxa and pollinator taxa at two different activity levels for the pollinator. Then, using the pollinator network derived from Robinson's data, they determined the potential impact of these simulated changes on temporal gaps in food supply for the pollinators. While this is an interesting approach, it would be good to test this method using current data, but this is difficult because <0.1% of the historical prairie habitat remains in Illinois; the loss of habitat is likely to have already caused a decline in these network interactions.

In a further use of Robertson's (1929) data set, Burkle et al. (2013) focused on the spring flowering community of the woodlands and their pollinators in central/western Illinois, including more recent observations (1970–1971 by Marlin and 2009–2010 by Burkle and Knight in 14 sites; Burkle et al., 2013). They compared the network from Robertson's data with the network from their recently collected data, calculating the gains and loss of interactions and taxa. Because bee taxa were extirpated in many of the locations, they compared the traits of the bees to determine if particular characteristics (i.e., historic diet breadth, nesting habit, sociality, historical phenological overlap) were associated with loss from sites. To determine if the bee extirpations could be attributed to phenology or were due to changes in diversity, they created a null model. After each run with the model, the authors determined whether interactions remained after phenological change. Further analysis of this community determined if pollinator diversity and fidelity for one of the common plants (Claytonia virginica L.) had been altered with land-use changes, using aerial photographs to quantify changes from 1968 to 2005 (Burkle et al., 2013). Overall, their different approaches for comparing networks over time enabled the authors to determine the extent of changes and attribute some to specific causes (i.e., phenological and land-use changes). As many habitats are impacted by land-use alteration and other influences along with climate changes, approaches that can tease apart the different causes of the loss of plant–pollinator networks will be particularly useful.

Experimental—The effect of species richness on specialization in plant–pollinator networks was tested using experimental climate treatments in 15 grasslands positioned along an elevation gradient in the German Alps (Hoiss et al., 2015). Snowmelt date was altered by adding or removing snow, or drought was imposed using rainout shelters (4 × 4-m coverings for 43 d, with a canister and funnel to measure the amount of water excluded). These treatments were designed to simulate extreme climate events where air temperature at the soil surface and moisture probes quantified the treatment effect (but the addition of snow did not significantly delay snowmelt). The plant–pollinator networks were quantified multiple times over the season. In part, Hoiss et al. (2015) found that species diversity was negatively influenced by drought and higher elevations, particularly for pollinators. While the advance of snowmelt date did not impact species diversity, it increased the degree of specialization within a network. A limitation of their study, as discussed by the authors, is that the experimental plots were limited in area (4 m²), which could underestimate specialized interactions.

Phylogenetic considerations—Given the concerns about the impact of species loss on mutualism networks, Rezende et al. (2007) evaluated the phylogenetic relationships of species in mutualistic networks to determine whether extinctions are random or are more likely to occur among related species. Using published networks, they simulated extinction cascades by removal of one species (starting with a specialist or the least linked species), followed by assessment of all of the remaining species; if a species was without interactions, it went extinct. The resulting network was evaluated for the decrease in species diversity and phylogenetic relatedness. When compared to null models, related species with similar roles within the networks experienced higher than expected extinctions if evolutionary relationships did not structure these networks (Rezende et al., 2007).

If climate changes impact a pollinator's phenology such that interactions with plants are negatively affected (i.e., interactions are lost or weakened with few visits), are related plant species more likely to be negatively affected compared to unrelated species with similar floral traits? Using a linear model approach with phylogenetic information (i.e., phylogenetic linear mixed models [PLMNs]), in which the strength of interactions between plants and pollinators is the response variable, Rafferty and Ives (2013) developed an approach to test if the phylogenetic relationships can predict changes in interactions. Using their data from a field study, Rafferty and Ives (2013) constructed two types of models—one based on plant traits and the other on phylogenetic relationships. These models were applied to their data assessing the effects of phenology and floral traits on the pollinator community. Plant traits were key in predicting pollinator responses; if a plant's traits are not known, the plant's phylogenetic relationships can be used to determine which species are susceptible to changes in pollinator composition (Rafferty and Ives, 2013). While this phylogenetic approach can be very rigorous and very useful, the application to this specific data set may not have been the best example, given the limited and nonrandom set of plants.

To determine whether multiple factors (including phylogenetic relationships) imposed constraints on the structure of plant–pollinator networks, Vázquez et al. (2009) developed a conceptual model that used a matrix statistical approach to include several potential constraining factors. They found a weak phylogenetic signal in a plant–pollinator network based on data from a desert ecosystem, but only plants (and not pollinators) affected the network structure (Vázquez et al., 2009).

Given the real potential of species loss with climate change, particularly when combined with other aspects of environmental degradation, and considering further nonrandom extinctions due to phylogenetic relationships, phylogenetic analysis should be incorporated into studies of how networks or communities respond to environmental changes.

DISCUSSION

I have presented an overview of the diversity of approaches used to study the impacts of climate change on the phenology of plants, pollinators, and their interactions. Here, I will first highlight some of the strengths and limitations of approaches (for details on specific studies see  Appendices S1 (apps.1700012_s1.docx),  S2 (apps.1700012_s2.docx), and  S3 (apps.1700012_s3.docx)). As the strengths and limitations of the general type of methodological approach are often not taxon specific, I will combine this discussion across the main sections (taxon) and organize it by the subsections (study type). I will then discuss research directions that are understudied in the climate change literature but critical for a more comprehensive evaluation of the impacts on plant–pollinator interactions.

Limitations and concerns

Historic records—Evaluations of climate change impacts are greatly strengthened by clear assessments of how phenology (and other traits) have changed over time. Thus, herbarium specimens of plants and museum specimens of pollinators are vital for obtaining this historical perspective. Strengths of using these specimens to reconstruct historic dynamics include: a long time period of available specimens predating climate change; many specimens with collection date and geographic location; high diversity of taxa represented in the collection; the collection encompasses the geographic/habitat range of taxa; and repeated collections at a site allow for community studies. For plants, the phenological stage and flower/fruit abundance can be determined from these specimens. Bee specimens can be used to determine the following information: their social caste, sex of individuals, and pollen collected for species that can be identified to taxa. This biological history can be combined with climate variables typically available from governmental agencies for many years, but older records may be less frequent and cover fewer locations.

One limitation of using these specimens for evaluation of phenological responses to climatic change in plants is that they may only represent a snapshot of time; for example, while we can see the stage (e.g., flowering, fruiting) for the specimen, it does not tell us about all of the flowering phenological stages for the other individuals or populations. Similarly, for pollinators, the start and duration of their activity in a location may not correspond to dates of collected specimens. Sampling effort for specimens is largely unknown. Thus, they represent an imperfect historical record, which researchers have nonetheless been able to successfully use, often combined with creative approaches (e.g., Bartomeus et al., 2013; Robbirt et al., 2014; Matthews and Mazer, 2015).

Field observations—The strengths and limitations of field observations vary across studies because they depend on the design and protocol for the specific study. Some of the potential strengths of field observations include the following: long-term observations that often predate climate change environmental effects; comprehensive seasonal collection of plant and pollinator phenology; and given the multiple environmental factors that are changing, long-term records of how species are responding to multiple factors. When combined with long-term environmental data, these studies can provide insights into how natural communities are responding to environmental changes.

Limitations of field observations include the following issues: first flowering day or first arrival is often the only phenology measurement; sampling effort is not verified (which is an issue for quantifying plant–pollinator interactions for communities and networks); observations sometimes occur on nonnative species (by some national phenology groups); when observations are the only approach used, determining the drivers and mechanisms behind phenotypic changes can be difficult to interpret (Rafferty et al., 2013; Forrest, 2015); pollinator emergence may not be monitored independently of flowering phenology (Forrest and Thomson, 2011); and extensive time and/or effort is required for more detailed phenology, particularly for community and network studies. Many studies have focused phenology assessment on the start of seasonal activity. However, as climate change includes longer summer and fall seasons, phenological shifts in the response to these later seasonal events would be missed (Diez et al., 2012).

The sampling concerns below apply to observation/monitoring and other approaches (experimental and environmental gradients) whenever plant–pollinator interactions are being estimated. Although it is desirable to assess pollinator networks in multiple environments to obtain a comprehensive assessment of the impact of the environment on the network, obtaining a complete sample of the network requires substantial effort. A four-year study was designed to determine the extent of sampling and observations needed to determine the network's structure (Chacoff et al., 2012). After intensive effort, their sample included 80% of the pollinators, but was missing many of the interactions. The authors concluded that most published networks are incomplete, suggesting that greater sampling efforts are needed as well as approaches to verify the completeness of the networks (Chacoff et al., 2012). To further quantify species interactions, methods beyond direct observations may need to be considered. The assessment of the pollen on the captured pollinators, where pollen can be identified via comparison to a reference collection or via DNA barcoding (Bell et al., 2017), is one complementary approach. The use of digital video recorders could also supplement the observations (Gilpin et al., 2017).

Another sampling concern with assessment of plant–pollinator interactions is that data from single visits by a pollinator are often used for constructing pollinator networks or to assess the relative effectiveness of a pollinator; however, this may be a poor approach and potentially overestimates the degree of specialization of plant–pollinator interactions (King et al., 2013).

Experimental—While researchers may debate the problems with observations vs. experimental approaches to assess climate change impacts, I tend to see them as complementary, where experiments can test response mechanisms posed by results from long-term observations. Experimental approaches enable direct tests of specific environmental variables and response variables, while the data from observations can inform hypotheses and experimental treatments.

Strengths of experimental studies include the following: detailed and comprehensive data on phenology, and clear cause and effect due to the specific treatment(s). Limitations of experimental studies include that they are often conducted for a limited time that could be inadequate to detect a response. For example, a formal comparison of long-term observations and short-term warming experiments that assessed flowering onset or leafing out concluded that the experiments were underestimating the changes in the observations (Wolkovich et al., 2012). While Wolkovich et al. (2012) were unable to find a specific factor in their data set that could explain this difference, they recommend collection and analysis of further environmental data, although comparison of studies using different timeframes could be problematic.

Environmental gradients—Overall observational or experimental studies that incorporate environmental gradients are particularly advantageous for assessment of climate change impacts. As climate changes, the responses of species occur on temporal and spatial scales, which can be incorporated into studies across environments over time.

Strengths of environmental gradient approaches include the following: when combined with longer-term studies, they can incorporate spatial and temporal scales; they can assess multiple natural environments; they can potentially use environmental gradients (spatial variation) for temporal responses (e.g., Lessard-Therrien et al., 2014); reciprocal transplants within a gradient can be used to determine whether a population will respond to future climate conditions, as well as the extent of evolutionary potential and local adaptation (Etterson, 2004a, 2004b; Morton and Rafferty, 2017).

The main limitation of environmental gradient approaches is that spatial variation is not always equivalent to temporal variation. Thus, it is not always possible to use an environmental gradient study to replace the lack of historical data (Illán et al., 2012).

Research needs—Further research would be particularly valuable in the following areas.

Community plant–pollinator interactions and network studies—Both plant and pollinator communities have been found to be changing in their phenologies and seasonal overlap (CaraDonna et al., 2014 [see section on Plant-Focused Studies, above]; Petanidou et al., 2014 [see  Appendix S2 (apps.1700012_s2.docx)]). Because the dynamics of plant–pollinator interactions is a community-level process, studies are needed that assess community changes such as temporal dynamics of coflowering and pollinator communities (e.g., Benadi et al., 2014; Kudo, 2014 [see  Appendix S3 (apps.1700012_s3.docx)]).

Floral traits for pollinator attraction and climate change—While the overlap in timing of phenological events is vital for successful plant–pollinator interactions, other floral characteristics are critical for attracting pollinators. Some of the key traits for attracting pollinators are directly altered by climate change. For example, nectar production (estimated via microcapillary tubes) and sugar concentration (estimated via handheld refractometer) were affected by warming treatments in the field or climate chamber (Mu et al., 2015; Takkis et al., 2015). Flower display and size decreased with drought, warming, and snowfall in field studies (Saavedra et al., 2003; del Cacho et al., 2013; Burkle and Runyon, 2016). A metanalysis of many long-term Arctic warming studies found that flowering was earlier with warming, but the effect on the number of flowers was not consistent across species (Barrett and Hollister, 2016 [see  Appendix S1: Table C (apps.1700012_s1.docx)]). For some species, floral volatiles (volatile organic compounds collected by portable volatile collection systems and measured by gas chromatography) were reduced by drought associated with climate change (Burkle and Runyon, 2016, 2017). Ultraviolet reflection of flowers (i.e., bullseyes), which is a key visual cue in bees, is altered with changing irradiance associated with atmospheric changes and climate change. Koski and Ashman (2013, 2015) found geographic patterns of floral bullseyes in ultraviolet reflection of fresh and pressed flowers. Because floral traits are key for attraction, quantification of changes in these traits will enable a more comprehensive assessment of impacts on plant–pollinator interactions in communities where plants potentially compete for fewer pollinators. The ecophysiological perspective that integrates plant attraction traits and pollinator foraging traits is reviewed elsewhere by Scaven and Rafferty (2013).

Consequences of changing phenology for plant reproduction—Given the expected changes in phenology with the altered climate, the next step is to determine if there are consequences for the actors. Here I will focus on examples of the types of studies needed to assess the consequences for reproduction given climate change and potential declines in pollinator service.

Herbarium specimens used to assess flower phenology may also be used to determine if fruit production is declining. For example, Molnár et al. (2015) quantified fruit production for 150 yr in herbarium specimens of orchid species.

A few of the studies tracking flowering phenology also quantified reproduction (Kudo et al., 2004; Li et al., 2016 [see  Appendix S1: Table D (apps.1700012_s1.docx)]). For example, in an unusually warm year in Japan, spring ephemerals flowered earlier and bee-pollinated species had lower seed production compared to other years. However, fly-pollinated species flowered earlier but did not exhibit reduced seed production (Kudo et al., 2004). Including estimates of seed production enables phenotypic selection analysis of changing phenology and floral traits (e.g., Forrest and Thomson, 2010).

Other studies investigating climate change that combined plant–pollinator interactions with reproduction used the following approaches: sites with differing climates were used to determine the impact of specific environmental conditions (Sletvold and Ägren, 2015); pollinator-exclusion and pollen-addition treatments were used along with open-top chambers to determine effects on flower longevity and reproduction (Arroyo et al., 2013); long-term observations were combined with pollen addition and pollinator exclusion/enrichment studies (Thomson, 2010 [see  Appendix S3 (apps.1700012_s3.docx)]); and pollinator observations were combined with quantification of seeds (Gezon et al., 2016; Rafferty et al., 2016 [ see Appendix S3 (apps.1700012_s3.docx)]).

Pollen limitation and climate changes—Because seasonally earlier-flowering species are expected to be more likely to shift their phenology in response to climate change compared to later-flowering species, the extent of pollen limitation may differ during the season. To address whether this occurs, earlier- and later-flowering plants were compared in a warming treatment vs. ambient temperatures (Totland and Eide, 1999; Forrest and Thomson, 2010). Seed production and mass were quantified in both studies, enabling the authors to test if earlier flowering leads to pollen limitation or lower seed quality (Totland and Eide, 1999). Although the assessment of pollen limitation is typically studied via addition of outcross vs. natural pollen, pollen limitation is likely to be overestimated if plants naturally receive a mixture of self and outcrossed pollen (Thomson, 2010). Thus, a combination of approaches is recommended to better assess if plants are pollen limited.

Consequences of changing plant–pollinator interactions on plant mating systems—Given the potential lack of pollinators as a result of population declines or temporal mismatching, increased selfing may evolve over time (Levin, 2012). Several studies have used different experimental approaches to test if floral traits changed, as expected for greater selfing (Bodbyl Roels and Kelly, 2011; Jones et al., 2013; Spigler and Kalisz, 2013; Van Etten and Brunet, 2013; Thomann et al., 2015a, 2015b [see  Appendix S1: Table C (apps.1700012_s1.docx)]; Zhang et al., 2015). To assess if floral traits would evolve toward increased selfing, Bodbyl Roels and Kelly (2011) conducted a five-generation selection experiment, in the presence or absence of bumblebees. Similarly, to determine if treatments resembling the changes found with climate change would favor plants with traits associated with selfing, Van Etten and Brunet (2013) used the genetic variation among populations to examine the response of floral traits to different moisture and temperature treatments. Although not all traits responded as expected, both studies found a decrease in relative anther and stigma separation in the absence of bees or with drier treatments. Whether floral traits are changing toward increased rates of selfing needs to be quantified in additional studies.

Demographic impacts of shifting phenology—The consequences of shifting phenology and mismatching are potentially wide ranging; poor reproduction and altered mating systems for plants, as well as reduced pollinator longevity, may result in demographic effects for both plants and pollinators (Fagan et al., 2014). However, studies assessing demographic consequences are rare.

Species distribution shifts in response to climate change—Although this review did not include studies on species distribution shifts, pollinators and plants may differ in their range shifts in response to climate changes. Thus, approaches that include spatial and temporal responses to climate changes are critically needed to assess impacts on mutualisms. Long-term observational community studies across environmental gradients have found that plants and bees respond temporally or spatially to these changes (Pyke et al., 2016 [see  Appendix S3 (apps.1700012_s3.docx)]). For assessment of plant phenology across larger spatial scales, satellite imagery could be a useful approach (Ding et al., 2016).

Winter and spring climate—Changes in winter temperatures, which are changing at a faster rate, also have specific impacts; however, this change has been relatively less studied (Kreyling, 2010). Extreme late frost events during the spring are predicted, while seasonally early species are expected to flower earlier (Inouye, 2000; Augspurger, 2013). Further assessments are needed of the impact of extreme frost events, lack of snowpack, and alteration of insect diapause on plant–pollinator interactions (Mu et al., 2015 [see  Appendix S1: Table C (apps.1700012_s1.docx)]; Fründ et al., 2013 [see  Appendix S2 (apps.1700012_s2.docx)]; and others in the Spring Climate group of studies [see  Appendix S3 (apps.1700012_s3.docx)]).

Impacts of other drivers of climate change—Many studies of climate effects on plant–pollinator dynamics have focused on temperature and aspects of precipitation; however, other factors that are drivers of climate change (such as CO₂ and N₂O) have been less well studied, although they are also known to impact multiple aspects of plants reproduction. These drivers, along with other anthropogenic impacts (e.g., habitat fragmentation, invasive species), may interact to further affect plant–pollinator interactions as well as other multitrophic interactions (Tylianakis et al., 2008; Schewieger et al., 2010; González-Varo et al., 2013). Thus, it is essential to assess climate change impacts in different environmental contexts (Harrison and Winfree, 2015).

Restorations and interactions given climate change impacts—Given the global loss of natural habitat, there are many regions of the world where habitat restoration projects are underway. The inclusion of species interactions is increasingly being considered during habitat restorations (Winfree et al., 2015). Given the changes in flowering phenology with climate change, the availability of floral resources during the season needs to be included in restoration design. Considering the potential of changing plant phenologies to result in resource gaps for pollinators, Memmott et al. (2010) tested the impact of different seed mixtures (with or without wind-pollinated species) that are expected to differ in seasonal food availability for bumblebee pollinators. The mixtures were planted along field margins, the plant–pollinator networks were determined, and the impact of climate changes on the networks was simulated. Although this study found differences in the gaps of food availability for the bees, very few species were left with no food resources. This study provides an assessment of how climate change affects bumblebees' diet and how restoration projects can be designed to protect the pollinator communities against the effects of climate change (Memmott et al., 2010). This approach can be more widely applied to managed habitats for conservation efforts with climate change, thus assisting the restoration of this key interaction.