Exploring Landscape Change in Mountain Environments With the Mountain Legacy Online Image Analysis Toolkit

Since 1996, Mountain Legacy Project (MLP) researchers have been exploring change in Canada's mountain environments through the use of systematic repeat photography. With access to upwards of 120,000 systematic glass plate negatives from Canada's mountain west, the MLP field teams seek to stand where historic surveyors stood and accurately reshoot these images. The resulting image pairs are analyzed, catalogued, and made available for further research into landscape changes. In this article we suggest that repeat photography would fit well within the Future Earth research agenda. We go on to introduce the Image Analysis Toolkit (IAT), which provides interactive comparative image visualization and analytics for a wide variety of ecological, geological, fluvial, and human phenomena. The toolkit is based on insights from recent research on repeat photography and features the following: user-controlled ability to compare, overlay, classify, scale, fade, draw, and annotate images; production of comparative statistics on user-defined categories (eg percentage of ice cover change in each image pair); and different ways to visualize change in the image pairs. The examples presented here utilize MLP image pairs, but the toolkit is designed to be used by anyone with their own comparative images as well as those in the MLP collection. All images and software are under Creative Commons copyright and are open access for noncommercial use via the Mountain Legacy Explorer website. The IAT is at the beginning of its software life cycle and will continue to develop features required by those who use repeat photography to discover change in mountain environments.


Introduction
In this article we draw on 20 years of experience rephotographing historic Canadian mountain images for the Mountain Legacy Project (MLP) to show that repeat photography can be a critical interdisciplinary method for observing and attributing change in mountain environments. The article gives a sampling of the wide range of studies from many different disciplines that have successfully used repeat photography methodology in scholarly inquiry. Using examples from our own rephotographic work in the mountains of western Canada, we introduce the Image Analysis Toolkit (IAT), software designed by MLP to assist with visualizing, annotating, and quantitatively describing changes and similarities between photographs.
Today many research groups recognize the importance of an interdisciplinary approach in attempting to observe, understand, and explain some of the interactions of Earth systems. Chief among these is Future Earth, a research platform committed to international engagement around global sustainability (Future Earth 2014b). Several of the questions they pose in their research agenda may be a good fit with repeat photography methodology. For example, Future Earth (2014a: 15) asks, ''How has the Earth system, with its ecosystems and societies, changed in the past, and what can this tell us about current responses to environmental change?'' As part of their Dynamic Planet theme, Future Earth researchers focus on knowledge about physical, ecological, and social mechanisms that underpin environmental change. They ask how new perspectives can be obtained by including historical dimensions of environmental change, and are specifically looking for methods and technologies to access, organize, and integrate various types of interdisciplinary knowledge (Future Earth 2014a: 14-15). We contend that repeat photographic methods, including the IAT introduced here, could play a role supporting Future Earth and similar research initiatives in exploring such perspectives. At its core, repeat photography involves determining where a previous photographer stood when she or he took the original image, going to that place, repeating the photograph, and looking for changes and similarities within the image pair. Webb et al (2010), in their seminal work on repeat photography, review use of these techniques in a wide range of studies that evaluate ecological, geomorphological, and cultural change across landscapes large and small.
Repeat photography, together with software tools to help visualize and quantify changes within and between images, can play a role in looking at mountain ecosystems, land use, and/or cultural practices and help relate the past to current environmental change. Such images and software can also be used to illustrate patterns of change across mountain landscapes at a range of scales. Here we introduce and provide a preliminary assessment of the IAT, a new image visualization application created by MLP to help practitioners use photographs-either their own or those in the MLP collection-in a diverse range of mountain research and development pursuits.
Using repeat photography to explore change in mountain landscapes Repeat photography typically works with land-based oblique images, those taken looking out from a mountaintop or promontory and thereby rendering the landscape as continuously variable across the image plane. These are images taken from a human-eye viewpoint rather than the bird's-eye view of aerial photography and remotely sensed satellite imagery. Vast numbers of oblique images exist for mountainous regions of the world because climbers and enthusiasts have been inspired to document ascents and remarkable mountain landscapes almost since the invention of practical photography in the mid-19th century. Indeed, the first use of repeat photography as a scientific tool was in 1888 by Bavarian mathematician Sebastian Finsterwalker studying mountain glaciers in the Tyrolean Alps (Webb 1996: 30).
Today repeat photography is used extensively to document change in a wide variety of ecological, geological, fluvial, and human phenomena. White and Hart's (2007) repeat photography in the Canadian Rockies touches on all of these phenomena. Collection types vary significantly, from one-off images gathered in archives to large, systematic collections such as those collected to track changes in desert plant communities near Tucson, Arizona (Hastings and Turner 1965;Webb et al 2007) and the MLP (Delaney 2008;Trant et al 2015).

Challenges in using repeat photography image pairs
Apart from the issues of working with the continuously variable representation of landscapes within oblique photographs, there is the challenge of finding image collections that are both comprehensive and systematic. Systematic collections are those taken with similar or consistent imaging techniques according to consistent rules (eg triangulated views from mountain summits), and are sufficiently comprehensive across a landscape to be useful. Finding historical images to repeat, especially in isolated areas, can be challenging. Moseley (2006), looking at historical landscape change in remote northwestern Yunnan province, China, collected photos from a wide range of sources, including National Geographic Magazine. Kull (2005), in examining regional land use change in the Madagascar Highlands, describes issues around obtaining spatially representative images, noting that collections often include many images taken from the same placeusually a place that is easily accessed. N€ usser (2000) used repeat photography to look at sustainability, land use, and landscape change in the Nanga Parbat region of Pakistan. He notes the valuable role it can play when combined with other analysis techniques (eg historical reviews, in situ interviews, and ground truth analysis).
Repeat photographers are sometimes asked, ''Why not just use remotely sensed data, such as aerial photographs and satellite imagery?'' Depth of time and increased subject detail are 2 reasons to consider repeat photography of landscapes. In the Canadian mountain west, for example, by 1945 only one quarter of Canada had air photo coverage. Only in 1957 was the entire country covered (Harris 1990). Thus, many of these early air photos go back only 60 years, and they do not provide the detail found in historic glass-plate images taken by earlier mountain surveyors. If we rephotograph the oblique survey images used in creating maps of the Canadian cordillera, in some cases we can look 125 years into the past. Figure 1 gives an example of both time depth and detail. As can be seen, the passage of time-93 years in this case-is writ large on the landscape.
In writing about the history and processes behind the use of photography in the creation of early Canadian maps, Dyce (2013: 74) says, ''Both map and photograph represent reality through the rules of perspective-they are reduced-scale versions of what they portray, where the relative distances in the smaller versions correspond to the larger distances found in reality.'' Thus the oblique image gives us a perspective view-a representation of reality close to what the human eye sees.
In their closing chapter, Webb et al (2010: 310) suggest that repeat photography, a decidedly low-tech tool in the evolving high-tech world of remote image capture, has a compatible and complementary role to play in acquiring spatial data alongside that of the most sophisticated remote systems, ''in fact, far exceeding those systems if site-specific information or a long-term perspective is desired.'' Another key challenge is the ability to align images. Gat et al (2011) gives an excellent overview of the necessary techniques. Features are matched between image pairs and then the correct scaling, translation, shear, and rotation between the pairs takes place. Figure 1 is an example of an image pair that has been aligned via this technique.
Visualizing change in image pairs also has its challenges. Years of working with MLP researchers and their photographic collections reveal that the ability to view aligned images on a computer screen side by side at the same size and scale underlies many other useful functions. Certainly, the ability to overlay one image on top of another and fade (adjust opacity) between the 2 is an important visualization technique. Hoffman et al (2010) details the use of adjusting opacity as a method to count numbers of a particular species of plant that survived, died, and recruited in specific plots over the years between the images.
The ability to create categories (eg snow / ice, rock / scree, coniferous forest, etc) and apply them to images has featured prominently in many mountain rephotography studies (Rhemtulla et al 2002;Zier and Baker 2006;Roush et al 2007;Hendrick and Copenheaver 2009;Taggart-Hodge et al 2015). In studies like these, such categorization is usually handled within a geographic information system (GIS) or some type of graphics software package (eg Photoshop). The work can be laborious and time consuming. Automated categorization of mountain landforms is beginning to figure prominently in computer vision (Jean et al 2015), and segmentation of oblique mountain images into habitat categories looks promising.
In many mountain-based research projects, some type of simple quantitative description has been applied to categories-counting pixels, calculating percentage of the images under each category, comparing the categories in both images to see if/where change has occurred. Typically, the pixel counts for each category assembled in the GIS or graphics package are retrieved and then analyzed in some type of statistical software package.
It is relatively straightforward to count and average pixels in aligned images, but actually visualizing what, for example, an area of rock and scree in an historical image may have changed into in a modern image presents some challenges. The change may not be uniform, or may be on a very small scale, or it may be that new categories need to be developed to describe the change. Moreover, this type of quantification is relative to the images only and has been very difficult to apply directly to real-world geographic coordinates.
Nonorthorectified look-down imagery in steep mountain landscapes can distort features like cliffs, high ridges, and steep mountain slopes. Oblique photos can provide insights into such terrain, giving a perspective view that can be further used to explore or illustrate causes of landscape change. That said, it is important to note that oblique images might not show an entire landscape-components can be hidden from the camera's view. Also, scale will vary within an image-a pixel in the foreground can represent a geographic area much smaller in size than a pixel in the background. This can pose problems if practitioners want to use these images to speak about change in the broader landscape. However, several studies (Rhemtulla et al 2002;Hendrick and Copenheaver 2009) indicate that if appropriate criteria are used when selecting image pairs for analysis, and if validation of category distribution over an entire set of images is performed (Fomin et al 2015), then issues with representation and scale can be at least partially mitigated. New developments in a technique known as monoplotting (Bozzini et al 2012;Stockdale et al 2015) show promise in allowing georeferencing and orthorectification of oblique-angle photographs. These techniques will allow for more accuracy in calculating category distribution within and between individual image pairs as well as large image collections like those available to the MLP.

The MLP
The MLP-based at the University of Victoria, British Columbia, Canada-utilizes Canadian mountain images taken from the 1860s to the 1950s. It is the largest collection of systematic and comprehensive historical mountain images in the world. Figure 2 illustrates the extent of the historic collection and indicates where the majority of the repeat images have been taken.
A technique developed in Canada in the 1880s, known as photo-topographic surveying, married panoramic images with altitude and survey angles to create the topographic information produced on the maps of the time. Within a region, surveyors worked from locations, known as stations, on peaks and high points. They took panoramic photos on glass plates from each station, ensuring that at least 2 of the other stations in the region were visible from photographs taken at any given station.
Since 1996, MLP's interdisciplinary mountain researchers have been in the peaks of western Canada rephotographing these historic images from the same stations using high-resolution digital cameras. MLP teams are then able to compare present mountain landscapes with the past and, through interpretation and analysis, explore the dynamics of ecological change and note the direct and indirect influence of human activity on the landscape.
A large selection of image pairs is available to anyone with a modern web browser and Internet connection (http://explore.mountainlegacy.ca). All images are available under Creative Commons copyright and are free for noncommercial use (Trant et al 2015). Basic metadata are available for each pair, as is at least one visualization technique: a wiping vertical slider that allows the user to hide/reveal the modern aligned image underlying the historic one.
Watching people use this visualization technique to engage with the images, MLP researchers began asking these questions: ''What tools do mountain study researchers and community practitioners from diverse disciplines need to access, explore, and analyze image pairs? Can images outside the Mountain Legacy collection also be explored?'' The IAT is the response, and, as more mountain studies practitioners engage with the tool, it continues to develop.

The IAT
In designing the IAT, MLP researchers worked with 4 central aims: Focus on oblique photographs-as Delaney (2008: 76) says, ''From the earliest days of the photograph to the present, its value has been recognized by the scientific community to help them document and understand the natural world.'' Much of the functionality in the IAT is based on observing practitioners from a broad range of disciplines, organizations, and communities working with photographs to better understand multifaceted mountain phenomena; Widely available access-allow anyone with an Internet connection and a modern web browser to see, manipulate, compare, and analyze their own mountain photographs from anywhere in the world and/or use images in the Mountain Legacy collection. Allow anyone to download the toolkit and use it without an Internet connection, thus making it usable in remote field locations. Comparative visualization-aligning and showing the images side by side as well as overlaid on one another. Allow for targeted, aligned window, polygon, and area views from one image onto the other in a manner similar to the Klett et al (2004) time-reveal window. Classification and interpretation-allow practitioners to apply their own expertise when exploring the images. Practitioners should be able to classify and quantify, Although some IAT components can be used on individual images, the toolkit is designed to display 2 or more images and afford practitioners different methods of visualizing, annotating, and quantitatively describing changes (and similarities) between the photographs. Although there is software available to georeference oblique images (see for example Stockdale et al 2015), the focus of this IAT version is the photographs themselvesall quantitative descriptions and visualizations are relative to the images, not to a digital elevation model or map. IAT was developed with the MLP in mind, but its use is not restricted to the Canadian mountain west. It is lightweight (approximately 45 KB), uses ubiquitous technology, has minimal dependency on other software, and is a single file that can readily be downloaded and used with any imagery and by practitioners not affiliated with the MLP.
Tables 1 and 2 break down IAT concepts into basic and advanced processes. Table 1 describes basic image manipulation and file management. Table 2 outlines the more advanced functionality around categorizing images, visualizing the categories, and calculating relative percentages in and between categories.
Many of the tools in the basic category can be applied to individual images or to image pairs that cannot be properly aligned. Almost everything in the advanced category requires aligned image pairs in order to give meaningful visualizations (Figures 3 and 4 are examples).

Initial toolkit assessment-the Mountain Legacy perspective
The questions MLP researchers ask cover a wide range of mountain research and development topics, and all inquiry has involved the use of repeat photography. The IAT was not in place when these projects were undertaken, but the research questions, methods, and results were formative in developing the toolkit. Moreover, had these same projects been undertaken today, the IAT could have played an important role in all of them. Table 3 presents an overview of representative studies undertaken by MLP, describes each study's influence on toolkit development, and suggests how the IAT might be used in future mountain research of this type.
All of the mountain-based repeat photography studies noted in Table 3 above, along with many of the studies in the literature noted here (eg Hendrick and Copenheaver 2009;Webb et al 2010;Fomin et al 2015;Stockdale et al 2015), especially those that are quantitative in nature, use several different software packages to achieve their goals. GIS and graphic packages such as Photoshop figure prominently. Certainly, these are powerful multipurpose Display images in their own viewing pane side by side, above one another, or one at a time. Swap left and right (or above and below) images.

Enhancement
Change color images to black and white (grayscale).
Zoom and pan Scaling-from pixel to any zoomed-out level.
Dragging the image(s) about in the viewing area.

Annotation tools
Creating polygons in a layer over the image(s). Freehand scribing of regions in a layer over the image(s). Line drawing on a layer over the image(s). Label creation in a layer over the image(s).

Fade
Controlled fade between images underlying a selected polygon or region. For example, imagine the practitioner has scribed out a region that covers a glacier on the left-hand image. A fade can be applied such that that same area from the right-hand image emerges to overlay the left and vice versa.
Basic file manipulation Import / export / save Browse to import images, classification categories, and masks. Drag and drop images in the left and right viewing panes. Drag and drop saved categories and masks in the viewing panes. Save any practitioner-created classification categories. Save classification category masks as PNG files.
tools, but sometimes practitioners may benefit from using a tool that is more directly focused on visualizing change in the images. The lightweight, web-delivered nature of the IAT means that practitioners do not need specialized GIS training to use the toolkit for creating and applying classification categories. Nor do they need to move between software applications to calculate classification cover statistics. That said, the IAT is also a good choice to use alongside GIS and other packages that deal with spatial data. For example, classification covers designed and applied in the toolkit can easily be imported to the R statistical computing environment.
The IAT is a new application, initially introduced in late 2015. Toolkit development continues, and in 2016 it was adopted as the primary analysis platform for a project looking at wildfire management strategies in at-risk mountain communities along the eastern slopes of the Rocky Mountains in the province of Alberta, Canada. A polygon or region can be assigned a selected category. A polygon or region of a given category is drawn onto the image(s). It overlays a semitransparent mask in the selected category color in a layer on top of the image(s).

Classification category analysis and visualization
Viewing and fading Creation of a moveable viewing window that allows the same view from righthand image to be overlaid on the left and vice versa. Fade between images underlying the specific viewing window. Fade between the left-and right-hand images in their entirety.

Alignment
Allows practitioners to align their own image pairs.

Classification category analysis
Count the number of pixels in each category in each of the left and right images.
Calculate the percentage of each category shown in each image. Count the number of pixels in each category that still occupy the same place in both images (no change). Calculate the percentage of each category that still occupies the same place in both images (no change). These calculations can be done for a single image (but cannot show change calculations between images), an entire image pair, a segment of a single image, or a segment of an image pair. Figure 3 gives a detailed example of an entire image pair.

Classification category visualization
Shows images without classification category image masks. Shows the categorized image masks with categories at full color saturation-1 color per category. Shows the categorized image masks with categories at semitransparent saturation overlaid on the original images.
-In any given image pair, category masks can overlay the image they are based upon, or the other image in the pair.

Single-category change visualization
Utilizes a third viewing pane to show detailed change for a given category over an image pair. The third pane displays a full color saturation image mask as follows (Figure 4 as an example): -The practitioner-defined classification category color is used to represent any areas of a category that remain the same in both images of a given image pair. -A substantially darker rendition of the same color is used to represent category areas that are present in the right-hand image but not in the left. -A moderately darker rendition of same color is used to represent category areas that are present in the left-hand image but not in the right.
Delivering change statistics in actual real-world ground cover as well as cover in the image itself; Delivering habitat-fragmentation statistics and nearestneighbor statistics; Tighter integration with the MLP Explorer website; Creating detailed alignment tools to help practitioners position and overlay their own image pairs from mountain regions around the world; Creating tiers of tools that correspond (in increasing levels of sophistication) to annotation, interpretation, and analysis.

Conclusions
In this article we presented an overview of repeat photography methods, showing their value in a diverse range of mountain-based studies and initiatives.
Uncovering and understanding patterns of physical, ecological, and social change-at a variety of scalesunderlies all repeat photography work. Understanding how the past brings us to the present, and how these patterns shape our decisions about the future, are key components in Future Earth's Dynamic Planet theme FIGURE 4 An example of the third viewing pane applied to the same image pair as shown in Figure 3. The classification category of snow / ice is visualized on a small area in the images. Where snow / ice areas are the same between the historic and modern image, the category color of light blue remains the same. Dark blue is used to represent snow / ice that is present in the right-hand image (2011 image) but not in the left (1918). Moderately dark blue is used to represent snow / ice that is present in the 1918 image, but not in the 2011 image.
(Future Earth 2014a: 14). Repeat photography offers an accessible approach for both professional and citizen mountain study practitioners to explore landscape change over time, especially in the mountains. What has to this point been largely an interpretive endeavor is given visualization and analytic powers with the IAT. It is possible not only to use historic images, but also to create reference images in areas of key scientific or community interest and track change over successive years using repeat photography and the IAT. Our aim in creating the IAT is to offer an affordable, accessible, and easy-to-use set of digital tools that can be used on single images, image pairs, or multiple images to help identify change in mountain environments around the world. IAT results can be useful in and of themselves, or can be joined with other research methods to more fully explore the dynamics behind observed changes.
A lightweight, full-feature web application, the IAT exists in a Creative Commons copyright space: http:// explore.mountainlegacy.ca. As such, it is an open and inclusive platform accessible to anyone with an Internet connection and web browser. To our knowledge, there is no other application that works with image pairs in this manner, and the IAT might therefore be considered innovative in communicating, engaging, and visualizing change. We offer it here as yet another instrument for inclusion in Future Earth's research and development platform. Exploring the use of repeat photography from a visual anthropological perspective as a powerful method to produce knowledge about place (Smith 2007(Smith , 2014. Using repeat photography to engage workshop groups around how long-term landscape changes influence conservation and restoration in remote mountain ecosystems (Falk 2014  As in the vegetation composition study above and including the following: Create visualization for a given classification showing what changes occurred between the historic and modern images. Advanced spatial statistics (eg fragmentation analysis, nearest neighbor analysis)-in development.
Advanced georeferencing and orthorectification of objects in the images-in development.
Use as parts of an integrated study that includes GIS, digital elevation models, and/ or 2D maps. Use to help visualize change between 2 or more images. For example, creation of masks at different transparencies that represent not just that an area has changed, but what type of change it has undergone. Use to help accurately determine field study sites.