The Long-Term Agroecosystem Research Network launched the LTAR Agricultural Performance Indicator Framework to evaluate how agricultural innovations perform relative to sustainable intensification goals in five domains: Environment, Productivity, Economic, Human Condition, and Social.
Here we describe our progress and plans for measuring the performance of agricultural innovations on rangelands.
We present a method for measuring outcomes of management innovations against site-specific benchmarks, which can be applied in grazinglands worldwide.
LTAR typically studies management on fine scales (ecological site, ranch); how to measure effects on broad scales (landscape, community) remains a persistent question.
LTAR's Agricultural Performance Indicator Framework will evolve with stakeholder engagement.
Introduction
Ranchers and land managers have long aspired to promote livestock production and natural resource conservation on rangelands,1 but most recognize that no management approach perfectly maximizes both outcomes. Overall, an accurate understanding of the co-benefits and tradeoffs among goals is needed for optimal decision-making about management options.2
Identifying the effects of rangeland management can be difficult, however, due to the complex patterns and processes found on rangelands. These lands support tightly coupled human and natural systems that vary greatly over space and time, exhibit highly complex responses to weather, climate, and management actions, and abide many competing demands for ecosystem services.3 Often, forces outside of a manager's control – global markets, fuel costs, precipitation, soil fertility, land tenure, urbanization, disease dynamics, and climate change – interact to determine whether a particular management action is successful in a given time or place.4,5 Adaptation to such conditions is a hallmark of rangeland management,6 but these conditions and adaptation to them present difficulties for researchers seeking to tease apart the effects of management versus the effects of the processes outside managers' control.7 The accurate assessment of management tradeoffs is confounded not only by such complexities, but also by the lack of a universal approach to evaluating the outcomes of rangeland management against social, economic, and environmental goals – goals which can at times be in competition with one another.
We contend that the effective evaluation of management tradeoffs and co-benefits can be achieved by measuring management performance against predetermined targets for multiple goals at coordinated agricultural research sites, and developing an indicator framework with stakeholders to communicate the performance of the management strategies. The 300+ scientists, producers, land managers, technicians, and data managers collaborating in the Long-Term Agroecosystem Research Network (LTAR) are advancing such an approach. Here we introduce the network's Agricultural Performance Indicator Framework and our progress in developing the framework for rangelands, including how we are building from the network's nationwide experimentation and grappling with issues of scale, developing a benchmark approach, and building capacity for tracking meaningful socioeconomic indicators. We discuss future plans, which include systematic engagement with stakeholders to co-develop indicators and benchmarks and an exploration of the many factors that influence the adoption of alternative management strategies.
LTAR and sustainable intensification
Several entities working on rangelands have developed indicators, but LTAR's focus on the sustainable intensification of agriculture sets it apart.8 Sustainable intensification is an array of goals and interventions intended to increase agricultural production while maintaining or improving human well-being and environmental quality. LTAR was founded to study and advance sustainable intensification, and currently represents 18 socio-agroecological regions across the U.S. in rangelands, croplands, and integrated crop-livestock systems (Fig. 1a).9,10 The historical timespan of experimentation and monitoring at LTAR locations averages 55 years, ranging from 19 to >100 years.10 As of 2018,LTAR regions supported ∼49% of the nation's cereal production, 30% of its forage production, and 32% of its livestock production.10
The Common Experiment is a principal network activity through which LTAR sites compare how local management systems promote or inadvertently impede sustainable intensification. At each site, the experiment contrasts at least one management system representing “business as usual” (a conventional management system of the region) with at least one “aspirational” management system hypothesized to advance the sustainable intensification of agriculture in regionally-appropriate ways.11 Notably, as the innovations of today become routine practice tomorrow, new innovations will emerge to continue the dynamic advancement of the Common Experiment. Along with the Common Experiment, the 18 networked sites also conduct common long-term measurements including meteorological, eddy flux, and core agricultural production and environmental indicators relevant to local and national stakeholders.12 Nearly 20 working groups have been formed to collaborate across sites and disciplines to develop data-driven solutions to challenges related to sustainable intensification ( https://ltar.ars.usda.gov/ltar-working-groups/).
LTAR's scientists have increasingly recognized that to achieve sustainable intensification, the complex relationships among human and natural systems must be understood and integrated into research and management.13 Accordingly, LTAR is expanding human dimensions research13 and has informally adopted a set of sustainable intensification goals, advanced by the USAID Feed the Future Innovation Lab for Collaborative Research on Sustainable Intensification,14 that spans five domains: Environment, Productivity, Economic, Human Condition,and Social (Fig.1b).The explicit inclusion of the latter three domains in LTAR's research parallels an ongoing shift in the scope of range science in North America15 and distinguishes LTAR from other long-term research networks such as the Long-Term Ecosystem Research network (LTER) and the National Ecological Observatory Network (NEON).13
LTAR's Agricultural Performance Indicator Framework
The LTAR network has launched the LTAR Agricultural Performance Indicator Framework to help agricultural scientists and all other stakeholders evaluate the tradeoffs and co-benefits of agricultural management and research approaches, in light of sustainable intensification goals in five domains (Fig. 1b).
The network is developing protocols for knowledge co-production with stakeholders to identify and refine the indicators to measure and how to measure them. We currently use a working definition of knowledge co-production put forth by the Stockholm Resilience Center and collaborators as the ‘iterative and collaborative processes involving diverse types of expertise, knowledge and actors to produce context-specific knowledge and pathways towards a sustainable future.’16 The network’s partners in knowledge co-production may include those who will directly use the indicators, and others who are directly or indirectly affected by decisions based on these indicators. Examples include producers, producer groups, food consumers, consumer groups, industry groups, agricultural business lenders, environmental groups, human health and nutrition institutions, and natural resource managers working for land management agencies. However, this short list will continue to expand with the goal of inclusivity and representation over time and space.17
The LTAR Agricultural Performance Indicator Framework is a work in progress. To date, about half of LTAR's working groups have identified the innovations for which performance indicators are needed, the audiences and stakeholders who would likely use or be affected by the indicators, how to best integrate a diversity of values and data into the indicators, and how to work with stakeholders to systematically prioritize an essential set of indicators. Currently, the innovations for which performance indicators are being developed and the working groups identifying them include a) on-farm/on-ranch management (Working Groups include Croplands Common Experiment, Grazinglands, Biology, Soils, Water Quality, Water Quantity, and Wind Erosion),b) farm-to-farm coordination (Manuresheds Working Group), c) standardized data production and interpretation (Phenology Working Group), and d) knowledge co-production (Human Dimensions Working Group). As the network continues to expand its expertise to broader scales of the food, fuel, and fiber system, focal innovations will be adapted appropriately,moving beyond agricultural production to include realms of consumption and public health.
Progress in establishing Agricultural Performance Indicators for rangelands
Building from the LTAR Common Experiment
Interdisciplinary scientists working at LTAR's six rangeland sites have met at least once per year in 2016-2021 to discuss how to best measure and communicate the effects of on-farm and on-ranch management discovered in the LTAR Common Experiment (letter “a” in the list above). The six rangeland sites are part of the LTAR Grazinglands Working Group and are represented by the authors of this article. Our efforts have been have been parallel with, but distinct from, those of the Croplands Common Experiment Working Group, with regular communication about potential harmonization of indicators.
Given the diversity of rangeland characteristics, management goals, and management treatments among the rangeland sites (Table 1), we recognized that specific common indicators were not always possible and that common indicator categories with site-specific indicators were a solution to this problem. We used three surveys to build consensus around a preliminary list of 14 performance “indicator categories” within five domains of sustainable intensification goals, designed so that each site can select one or more locally appropriate “site-level indicators”within each indicator category (Table 2). Responses to the three surveys and the synthesis of responses that led to the indicator categories were developed using local expertise and knowledge, input from formal or informal stakeholder groups at each site, and the wealth of studies published by each site and the network over decades to centuries10 (e.g.,from https://data.nal.usda.gov/publications/ltar; https://ltar.ars.usda.gov/publications/). The structure of our indicator framework is designed to be geographically flexible with site-specific performance indicators reflecting site-specific rangeland conditions and management goals, knitted together under a common umbrella of investigation and learning.
Goals are implicit in our agricultural performance indicator framework for rangelands (Table 2). In the Environment domain, for example, the indicator category “Biota” represents the goal of conserving biodiversity on rangelands. In the Social domain, the indicator category “Community security” represents the goal of preserving and strengthening ranching communities on U.S. working lands. Below we propose management performance be measured against locally derived benchmarks for the goals.
Stakeholder and partner insights have been, and will continue to be, instrumental to the development of our goal-oriented framework. The framework developed by the Feed the Future Innovation Lab14 has been a major influence on the LTAR Agricultural Performance Indicator Framework. Another key influence has been the concept of “multifunctionality of agriculture”, the idea that agriculture can provide environmental and social benefits beyond food and fiber production.18 We have benefited from decades of strong interagency collaborations among land management and agricultural research agencies that paved the way for systematic, standardized indicators of rangeland health.19-21 Other influential indicator frameworks include those developed by the Sustainable Rangelands Roundtable and the US Roundtable on Sustainable Beef.8
Grappling with issues of scale
Issues of scale arise in nearly every conversation about LTAR's indicators because understanding the spatial and temporal context of a given management approach is critical to evaluating its performance. Moreover, network scientists recognize that interventions designed to advance sustainable intensification can create problems for some agricultural stakeholders while providing benefits to others and that tradeoffs may vary with scale.22 For example, in the landscape of the Archbold Biological Station/University of Florida LTAR site, large wetland systems remove nutrients from river-canal systems. At a fine scale, potential drawbacks are eutrophication and invasive species, but at the watershed scale, the wetlands improve water quality and overall quality of life.23 To provide information for stakeholders who might focus on different scales, the LTAR network strives to develop meaningful indicators at multiple scales.
Table 1
Six LTAR sites investigating the sustainable intensification of rangelands in the LTAR Common Experiment.
Hierarchies of spatial scales in socio-agroecological systems are, by design, reductionist.24 Nonetheless, LTAR scientists have found such hierarchies useful when building a common indicator framework across multiple locations. For those working in rangelands, a commonly used spatial hierarchy includes ecological site concepts from the site to the ecological site through Land Resource Region25; we reference this hierarchy for a broad understanding across rangeland disciplines. The patch (site) is the finest-scaled unit (Fig. 2). At a broader scale, the ranch comprises the ecological sites, social systems, land uses, budgets, and infrastructure within the borders of a ranching enterprise.26 The rangeland landscape contains multiple ranches and interspersed land uses, and webs of social relations providing a sense of place and purpose for agricultural producers and consumers whose livelihoods are directly or indirectly dependent upon agroecosystems.26,27 At broader scales are regions (Land Resource Regions), which contain multiple landscapes, and continents, which contain multiple regions. We recognize that this spatial hierarchy reflects land systems more than food systems28. To expand, we seek to partner with scientists across the USDA and universities to expand our hierarchy into realms of human nutrition and consumer well-being.
Rangeland patterns and processes also exist on temporal scales ranging from minutes to epochs, but the typical rangeland management study is conducted for a short time, and typically in a single location. Consequently, reliable management recommendations can be found for specific places within a specific timeframe but extrapolating to many locations and longer timeframes is a widespread and persistent challenge.29 The LTAR Common Experiment is poised to disentangle spatio-temporal dependencies, because it facilitates the investigation of site- and ranch-scale management for multiple decades. Long-term research allows monitoring to continue beyond interruptions from external forces (e.g., drought) and helps capture effects as external factors fluctuate (e.g., markets, disease). This study design also allows measurement of the effects of dynamic adaptive management over time, as in the case of the Central Plains Experimental Range.30,31
Despite the design benefits of the long-term Common Experiment, most of its management systems are designed, implemented, and studied at fine spatial scales (e.g., pasture, ranch), and focused on outcomes for a short time frame (e.g., a 5 to 10-year planning horizon). We are grappling with how to identify the effects of fine-scale management on larger spatial scales such as community, region, or continent, and temporal scales such as decades to centuries.32 Standardized approaches to data production over landscapes and regions in concert with modeling will be central to cross-scale analysis for LTAR. Rangeland scientists focused on the Common Experiment are partnering with the LTAR Regionalization and Rangeland Soil Erosion Working Groups to advance these efforts.
The Regionalization group has defined regional boundaries33 through lenses of three domains of sustainable intensification: Environment, Production, and Rural Prosperity.10 This work will result in a data-driven refinement of the network's original regional boundaries (Fig. 1a), and will likely evolve into the five domains used in the LTAR Agricultural Performance Indicator Framework (Fig. 1b). The variables used to define the regional boundaries will allow network scientists to track trends in a “wall to wall” fashion across the contiguous U.S. (e.g., raster-based commodity production estimates, biodiversity estimates, and estimates of access to broadband internet).21,34 To integrate agricultural performance indicators with the “wall to wall” data products, we plan to identify a core set of site-level performance indicators common among all sites in the Common Experiment. These common indicators can be aligned with variables measured over large expanses to understand the effects of ranch-level management on regional- and national-level trends and to interpret regional and national trends using local findings and experiences.35
Table 2
Indicator categories and site-level indicators used to quantify performance of rangeland management systems at four rangeland sites in the LTAR Common Experiment. Future work and stakeholder participation may result in different indicators and increased duplication among sites. These indicators align mainly with a 5 to 10-year planning horizon for the ranch scale.
Figure 2.
Indicators of rangeland management performance in five domains of sustainable intensification goals. Management and performance indicators at fine scales (patch, pasture/herd, ranch) are the focus of the LTAR Common Experiment (Table 2). We will rely on standardized datasets and modeling approaches to identify the effects of fine-scale management on broader scales (landscape, region).
To predict the influence of ranch-level management on broader-scale impact in the Environment domain, we are fortunate to draw upon protocols and data systems built and tested for decades across the American West. Established rangeland ecosystem monitoring programs (e.g., Natural Resources Conservation Service (NRCS) National Resources Inventory, Bureau of Land Management (BLM) Assessment, Inventory, and Monitoring programs) use standardized methods on about 312 million hectares (770 million acres) of U.S. rangelands to monitor indicators of ecosystem attributes including soil and site stability, biotic integrity, and hydrologic function.19–21 The Rangeland Soil Erosion Working Group is applying a wind erosion model (AERO) developed at LTAR sites36 to BLM and NRCS field monitoring data which can assess how management systems or practices on public and private lands affect rangeland health. If LTAR rangeland sites use the standardized indicators as site-level indicators to assess performance of management treatments in the Common Experiment (e.g., ground cover in Table 2), we can extend our inferential footprints via modeling efforts such as those of the Rangeland Soil Erosion Working Group.
Ultimately, the network seeks an overarching modeling framework to predict the effects of fine-scale management across broader scales and among all five domains of sustainable intensification (Fig. 1b), much like the SEAMLESS Integrated Framework (SEAMLESS-IF) that forecasts impacts of agro-environmental policies and agro-technological innovations in the European Union.37 LTAR is partnering with other USDA entities (e.g., the National Agricultural Statistics Service; Agriculture Innovation Agenda) to quantify rates of adoption of the management systems under investigation in the Common Experiment, as we anticipate adoption patterns will be a key input into this overall modeling framework.
A typology of indicators will be important to organize thinking for this cross-scale effort. For instance, the World Health Organization, in its work on interventions for child health and development, uses an indicator hierarchy spanning from input indicators, to process indicators, to outcome indicators, to impact indicators.38 In the realm of sustainable intensification, it may be useful to consider impacts on a landscape or region to be the result of the aggregate of outcomes on many ecological sites and ranches in concert with external forces outside of any manager's control.4,5 Accordingly, in our hierarchical example of performance indicators for rangelands (Fig. 2), outcome indicators are provided for finer scales (patch, pasture/herd, ranch) and impact indicators are provided for broader scales (landscape, region).
Developing a benchmark method to assess management performance
We modified a benchmark method developed by the BLM and inter-agency collaborations among BLM, ARS, NRCS, LTAR, and others39,40 to evaluate the performance of rangeland management in the Common Experiment against predetermined sustainable intensification targets. We put forth this modified method for consideration among all LTAR scientists and partners, as well as the greater community of rangeland and pastureland ecologists and managers worldwide.
The step-wise method (Box 1) entails developing meaningful benchmarks for each site-level indicator adopted by each local LTAR site, conducting appropriate measurements to identify whether sampling units met the benchmarks, and ultimately performing a site-level and network-level assessment of the efficacy of “aspirational” production systems under investigation in the Common Experiment. Lessons learned from the assessment would be fed back into LTAR research, and into the knowledge co-development protocols of the stakeholder-driven Agricultural Performance Indicator Framework.
Here we focus on benchmarks and management outcomes at the ranch scale because our work in the Common Experiment aligns with this scale; however, the method can be applied at multiple scales (see Box 1). Further, this method can be reproduced across the world where parallel questions about management system performance and indicators arise. As benchmarks can be set by local stakeholders in knowledge co-production processes, we posit that this method can transcend some of the tradeoffs between “bottom-up”vs “top-down” indicator frameworks.41
To date, we have made more progress using the benchmark approach to measure the performance of on-ranch management against desired conditions in Environment and Productivity domains, which is logical, as the approach builds on established methodology used to assess impacts of management on rangeland health.39 Questions persist about how to set benchmarks in Economic, Human Condition, and Social domains. For example, one proposed indicator category in the Human Condition domain is “Physical safety of workers” (Table 2). This is clearly an important goal for agriculture, but how much and what type(s) of physical safety of workers is adequate? In addition, in the Economic domain, what is a meaningful target for “Net returns from operations”? Income as a percentage of inputs for normal business operations may underpin a meaningful benchmark, however, what proportion is sufficient for a high quality of life, and over what timescale, given fluctuations of external forces (markets, climate) and producers' willingness to prioritize returns other than financial in their ranching enterprises?42,43
Selecting indicators and benchmarks in Economic, Human Condition, and Social domains
To develop useful and meaningful performance indicators in the three socio-economic domains of sustainable intensification (Fig. 1b), a number of theories, methodologies and approaches are available, including social-ecological network analysis,44 innovation-adoption and economic studies, 4,45–47 and other sociological or ethnographic approaches.2,48,49 Paired social-ecological research in the study region may also engage stakeholders more directly, for example in participatory or “co-production” methodologies.50
Different scientific research methods have distinct goals. While some sociological and economic approaches are best able to describe larger-scale social or economic systems of rangeland communities,27,51 others follow an action-oriented or transdisciplinary tradition that initiates change within the research itself.52 In the latter, the science directly engages with multiple types of knowledge and experience and involves stakeholders in all stages of the research process,from research question development through data reporting, learning, and knowledge implementation.53,54 In recent years, the scientists in the Human Dimensions Working Group have infused the LTAR network with expertise and interest in this diversity of approaches.13
A wide variety of collaborative approaches seeking to connect knowledge and action52 are currently in use by LTAR's rangeland sites. All sites have formal or informal customer groups influencing the management investigated and how research is conducted in the Common Experiment. Several sites have conducted producer surveys to inform local experimentation.27, 55,56 The experiments at the Central Plains Experimental Range30, Northern Plains, and Jornada Experimental Range57 entail forms of collaborative adaptive management and translational science. Notably, such approaches can be resource-intensive processes which require special consideration of research ethics and are subject to human-subjects research regulation, and despite the contribution of the Human Dimensions Working Group, the LTAR network is not at present equipped to systematically expand these resource-intensive processes.13
Box 1 Stepwise method to assess management performance using benchmarks.
Identifying benchmarks of management performance and using indicators to assess whether the benchmarks were met helps to identify how management investments should be directed. Here we put forth a step-wise method for consideration across the LTAR Common Experiment – and beyond, in agroecosystems worldwide. We use an example from the LTAR-Jornada site, where scientists, producers, K-12 educators, and extension partners are partnering to compare how alternative supply chains originating from Southwestern ranches meet sustainable intensification goals.56 The LTAR Common Experiment at the Jornada site spans multiple phases, from cow-calf to finishing to meat quality to marketing, but the figure below represents only the experimentation in the cow-calf phase, where conventional cattle genetics (business as usual) are compared to heritage cattle genetics (aspirational). For purposes of illustration, only a subset of the site's indicators and benchmarks are shown. Tradeoffs presented are still hypothetical, as they are under active investigation.
Steps 1-2: LTAR-Jornada works with stakeholders to select site-level indicators within network-level indicator categories in the five domains of sustainable intensification (Table 2), and integrates local priorities into the overall stakeholder-driven LTAR Agricultural Performance Indicator Framework. Step 3: LTAR-Jornada sets management objectives and benchmarks for each site-level indicator using past research, data from standardized databases on ecological sites,58 stakeholder knowledge, and other sources. Step 4: Scientists and partners measure outcomes of business-as-usual and aspirational treatments under investigation within appropriate sampling units, producing new data and information for network-wide datasets (e.g., LTAR Data Inventory; Ag Data Commons). Step 5: Scientists measure how much or how many of the relevant sampling unit(s) meet pre-determined benchmarks under each treatment. Spider diagrams can be used for illustration. Step 6 (not pictured): Scientists, partners, and stakeholders assess why benchmarks were or were not met to inform future adaptive research and management. Step 7 (not pictured): The LTAR network aggregates the outcomes of each site's treatments relative to respective benchmarks, to evaluate the aggregate impact of widespread adoption of aspirational management approaches across U.S. rangelands, croplands, and integrated systems.
The core scientific expertise of the LTAR network is biophysical, as the USDA-Agricultural Research Service (ARS), the federal agency leading many LTAR network efforts, historically did not employ research social scientists (although this trend is changing), or have access to its own Institutional Review Board, and must comply with the Paperwork Reduction Act and other relevant regulations. University research and cooperative extension partners are critical collaborators, but permanent ARS social scientists will be needed to fully realize the LTAR network's vision of the Common Experiment and other LTAR research in human-natural systems studies. Overall, for greater balance in the information provided about the five domains, expanded investments in social science and economics personnel will be needed.13
Plans for the LTAR Agricultural Performance Indicator Framework and its rangeland indicators
The LTAR Agricultural Performance Indicator Framework is intended to provide usable and useful information and knowledge supporting decision-making about adopting innovations for sustainable intensification. However, LTAR scientists recognize many factors beyond information influence adoption, including structural policy, markets, and institutional contexts.11,27 Future steps must include coordinated research on all factors affecting decision-making around LTAR innovations. Such a coordinated approach will require further investment in innovation-adoption research, as well as investments in product delivery and educational campaigns run by experts in their fields.
As mentioned above, LTAR's rangeland researchers plan to identify a core set of ranch-level indicators common across all rangeland sites to link outcomes observed at the ranch scale to outcomes in similar variables at regional and broader scales. This work will progress in tandem with modeling efforts of interdisciplinary teams such as the Regionalization and Rangeland Soil Erosion Working Groups and will include insights on the degree of adoption of the innovations in the Common Experiment on rangelands.
These future steps will not occur in a research bubble, but instead, will be conducted in collaboration with stakeholders across the nation's land and food systems. We are building a stakeholder engagement program to shape the science grounding the indicator framework, drawing on the rich literature and experience on effective and equitable stakeholder engagement across power gradients.17 Our methods for stakeholder engagement around indicators will include: listening to civil society partners regarding goals, needs, and knowledge gaps; cooperative visioning of sustainability at multiple scales; and synthesis of stakeholder insights with literature, data, and ongoing research. Although the specifics are a work in progress, such knowledge co-production may be our best bet for simultaneously meeting environmental, economic, and social goals on rangelands of the future.
Declaration of Competing Interest
The authors certify they have no financial interest in the subject matter discussed in the manuscript. S.S, N.P.W, E.H.B., R.K.B., A.B.B., P.E.C., C.D.H.C., D.L.H., N.K., S.E.M., G.M., L.P., D.T., H.W., J.D.W., and B.T.B. are employees of USDA Agricultural Research Service, or are, or have been, supported by LTAR funds, and were associated with management decisions regarding the topic of this manuscript. A.B.B. and G.M. are Guest Editors for this Special Issue of Rangelands but were not involved in the review or decision process for this manuscript.
Acknowledgments
This research was a contribution from the Long-Term Agroecosystem Research (LTAR) network. LTAR is supported by the USDA. Thanks to LTAR leaders Teferi Tsegaye, Peter J.A. Kleinman, Tim Strickland, Phil Robertson, Mark Liebig, Hilary Swain, Dawn Browning, Zachary Hurst, Claire Freidrichsen, and Sieg Snapp for ensuring that indicators are a priority for the LTAR Network. The ideas discussed herein were developed at several meetings sponsored by the LTAR Rangelands Working Group in 2016-2021. We gratefully acknowledge Dr. John P. Ritten for providing expertise on indicators in the economics domain. Thanks to Mark Kautz for artwork in Figure 1 and Dr. Matthew M. McIntosh for the scales diagram in Figure 2.
