There are many techniques to deal with uncertainty when modeling data. However, there are many forms of uncertainty that cannot be dealt with mathematically that have to be taken into account when designing a biodiversity monitoring system. Some of these can be minimized by careful planning and quality control, but others have to be investigated during monitoring, and the scale and methods adjusted when necessary to meet objectives. Sources of uncertainty include uncertainty about stakeholders, who will monitor, what to sample, where to sample, causal relationships, species identifications, detectability, distributions, relationships with remote sensing, biotic concordance, complementarity, validity of stratification, and data quality and management. Failure to take into account any of these sources of uncertainty about how the data will be used can make monitoring nothing more than monitoring for the sake of monitoring, and I make recommendations as to how to reduce uncertainties. Some form of standardization is necessary, despite the multiple sources of uncertainty, and experience from RAPELD and other monitoring schemes indicates that spatial standardization is viable and helps reduce many sources of uncertainty.

Biodiversity, stakeholder, sampling, identification, detectability, distribution, remote sensing, concordance, complementarity, stratification, data quality

There are many sources of uncertainty in scientific research, some of which can be modeled mathematically, but some sources of uncertainty are considered non-probabilistic, and the best way to deal with these is controversial (e.g. Sniedovich 2014). Some uncertainties can be reduced by careful quality control during data collection, but most researchers attempt to reduce uncertainty by statistical analysis, usually through model-based inferences (Gitzen et al. 2012, Thornton et al. 2014). There is an enormous literature on model-based inference (e.g. Anderson 2008), but even the most complex of these models, with many alternative hypothesis, relate to only very simple systems with limited inputs and outputs. They are good for evaluating sources of uncertainty that can be modeled by probability distributions, and in essence are usually just attempts to parameterize a given generic model. For example, Anderson (2008) used Caley’s and Hone’s (2002) study of tuberculosis transmission in ferrets as an example of a multi-hypothesis study. However, all of the hypotheses presented related to ways that ferrets could contract bovine tuberculosis. These are very interesting questions, but a biologist interested in biodiversity questions might have to deal with uncertainty as to whether bovine tuberculosis is better avoided or treated, whether management costs outweigh the costs of no action, whether other species are more important than ferrets in transmission, whether control measures might be considered inhumane, whether climate change or changes in markets might make the question irrelevant, and many other sources of uncertainty that are difficult to put into a probabilistic framework. While it is reasonable to ignore such concerns in a short-term study designed to find a solution to an immediate pressing problem, researchers interested in conserving biodiversity over the next century do not have the luxury of being able to use such a focused and short-term approach (Haila et al. 2014).

Ecologists are generally most worried about uncertainty in relation to their field of research. Taxonomists worry about the correctness of identifications, modelers worry about the accuracy of parameter estimates in their models, sociologists are concerned with uncertainties about the contributions of different stakeholders, geneticists try to reduce uncertainty about gene flow, etc. However, biodiversity managers have to deal with all sources of uncertainty simultaneously, and the importance of different forms of uncertainty will vary depending on the objectives of management. I have therefore adopted a very broad concept of uncertainty, and make recommendations as to how they can be reduced or quantified by planning during implementation of field infrastructure and quality control during data collection.

Here we will describe some of the sorts of uncertainty that we had to take into account when developing the RAPELD system of biodiversity monitoring (Magnusson et al. 2005). We will use that system to illustrate the issues, but the same considerations are applicable to any in-situ biodiversity monitoring system. The RAPELD system is a standardized monitoring scheme developed to allow integrated analyses of biodiversity data collected in rapid assessments (RAP) and long-term (LTER [PELD in Portuguese]) studies (Costa and Magnusson 2010; Magnusson et al. 2013). Sampling in the RAPELD system is based on spatially standardized transects and plots, reducing uncertainty about spatial interpolations and extrapolations. The system can be adapted to different sized areas of interest, but, because it is modular, statistically valid comparisons can be made between studies that originally used different combinations of spatial modules.

The basic sampling unit in the RAPELD system is a 1 km transect combined with one or more permanent plots that are usually 250 m long in the largest dimension. That is, the system was designed for the relatively large landscapes managed by most municipal, state and federal governments, and is often not appropriate for the small-scale landscapes studied by many academic biologists. Uncertainty about the scale at which users would apply the results was the prime reason for designing a modular system.

Although there is large variation in the sizes of RAPELD modules, most users use a standard 25 km² grid with 30 uniformly distributed plots for intensive studies in long-term ecological research sites located near major research institutions, and standard 5 km² (5 km × 1 km) modules with 10 uniformly distributed plots for RAP studies or long-term studies distributed over large areas (http://ppbio.inpa.gov.br/inventarios/modular). Uniformly distributed plots are 250 m long and the center line follows the altitudinal contours (http://ppbio.inpa.gov.br/instalacao/parcelas), a design that generally allows greater precision of models that relate biodiversity parameters to environmental variables. Plots for special strata, such as streams and riparian zones, are distributed in proportion to their occurrence in the landscape.

Most biologists specialize in a limited range of taxa (e.g. vascular plants) or processes (e.g. pollination), but decision makers have to take into account the needs of many different stakeholders, who may be interested in subjects as varied as the effect of large carnivores on domestic animals, bacterial metagenomics, ecosystem carbon storage and traditional uses of biodiversity. Reduction of these varied interests to a production-line mathematical model with limited inputs and outputs is usually not feasible, especially when a major uncertainty is whether we are addressing the right question (Haila et al. 2014). Therefore, investing monitoring in a limited number of questions, however important they may be at the moment, is not an efficient strategy.

A major difficulty, perhaps the major difficulty, with the interpretation of data collected in monitoring exercises is that the biologists have focused on their favorite group and not collected the data in such a way that it can be integrated with information generated on other biological groups and presumed environmental drivers. Different taxa provide different information about the distribution of biodiversity, and there is often heated discussion about the appropriate group to study (Magurran and McGill 2011). However, the sad reality is that we generally just base our decisions on convenience. There are few groups that have been surveyed over wide areas for which we are reasonably confident that most individuals have been correctly identified. These are usually only birds and vascular plants, though some groups of butterflies and mammals are reasonably well known in some areas. There is strong evidence that they are not sufficient to represent all biodiversity (Caro 2010), but they will continue to do so until we move out of our comfort zones within the internet cloud.

When we started our studies, discussing possible sampling designs only resulted in endless discussions as to which design optimized for a particular question was the “right” design. We found that the only way to obtain integrated data collection was to provide standardized infrastructure that could be used by most researchers to answer a wide variety of questions. Henle et al. (2006) present a European example of this approach, and Olsen et al. (2012) give several examples from the USA. Most researchers had not adequately budgeted for field infrastructure and were happy to use what was available. As this infrastructure had been designed to allow integration, data collection resulted in integrated studies almost as a side effect (Costa and Magnusson 2010, Magnusson et al. 2013).

There is a great difference between planning for an individual study of a limited range of organisms and planning a monitoring system for a wide range of taxa over very large areas. We did not appreciate this at the beginning, and it only became obvious to us as we saw what worked for a wide range of researchers over large areas, and what was mainly useful for specific studies. This dichotomy has been recognized by many researchers responsible for nationwide monitoring of biodiversity (e.g. Johnson 2012, Olsen et al. 2012), but has only recently been included in reviews of best monitoring practice from a statistical perspective (e.g. Buckland et al. 2011, Connolly and Dornelas 2011, Reynolds 2012), most of which had previously concentrated on idiosyncratic planning of projects with a common source of funding (e.g. Likens and Lindenmayer 2011). Below I will provide a short overview of key sources of uncertainty and how standardized field infrastructure can be used to help avoid or quantify uncertainty during the establishment and running of large-scale monitoring schemes.

Individual researchers tend to consider their study site to be primordially of interest in relation to their current research question. However, that piece of land may have a multitude of other values for the local people (Silvius et al. 2004). When planning where to install long-term research sites, we found that many different stakeholders were interested in the same site. Data generated might be used by international organizations, such as the International Long Term Ecological Research program and United Nations agencies, federal agencies, such as ministries of science and environment, regional bodies, such as State Government planning agencies, individuals operating regionally, such as university professors, park administrators and firms specialized in bioprospecting, and those interested in a small patch within a site, such as students, community groups and volunteers. In general they follow a political hierarchy (e.g. Magnusson et al. 2013: 58-59), but the categories do not always have clear boundaries. For example, individual volunteers may contribute to nation-wide projects, such as Christmas bird counts.

Stakeholder roles also depend on their position in the hierarchy. Multinational bodies try to influence decisions by changing national policies. Most funding agencies for long-term monitoring are national, but international organizations may provide short-term funding. Federal and state governmental agencies generally try to manipulate people´s behavior through the legal system. Academics are involved in planning and analysis, and most of the long-term monitoring has to be done by local people or students. All of these categories have fuzzy boundaries, and the relative interest of these groups is likely to change depending on unpredictable factors, such as employment opportunities, market demand and global climate change. Nevertheless, a monitoring scheme has to take into account the different roles of stakeholders. Probably the most difficult aspect of developing the RAPELD system was ensuring that different actors in different levels of the political system would be satisfied with their role, and the roles of other actors (Magnusson et al. 2013).

Biological relationships with distance are not linear (Landeiro and Magnusson 2011, Magurran 2011, Rosenzweig et al. 2011). It is possible to scale up from local data to larger areas, but only if data are collected in spatially standardized arrays, and this may create conflicts with organizations and volunteers who collect the data (Turnhout and Boonman-Berson 2011). Relationships depend heavily on the sampling scale (Baccaro et al. 2012, Rosenzweig et al. 2011, Chisholm et al 2013). Evaluations of the effects of scale of sampling are generally difficult with idiosyncratic sampling, but can be achieved with a few sites with standardized infrastructure. Where coverage is inadequate, geostatistical techniques may help to define priorities for locations of new sampling sites (Lin et al. 2008). Modular designs allow flexibility in answering local questions, while permitting different stakeholders to adjust the system to their questions (Magnusson et al. 2013). Some RAPELD sites, such as that in the Virua National Park, have been used both for local studies of interest to park managers (e.g. Pontes et al. 2012) and included in cross-site comparisons for academic studies (e.g. Souza et al. 2012).

Different stakeholders have different human and financial resources, but very few have the capacity to undertake detailed studies over large areas. Therefore, we needed a system that would allow integration of a large number of stakeholders with different technological tools at their disposal. Our infrastructure is suitable for use by local people with no formal education, and their participation is often vital because much biodiversity is hidden from the eyes of casual visitors (Magnusson et al. 2013).

Students are the main researchers in most RAPELD sites, but many of the surveys carried out in the modules around the Santo Antônio hydro-electric dam were undertaken by parataxonomists who had been trained in another state. We found that there was a trade-off in sources of uncertainty. Monitoring by students and volunteers increases uncertainty as to the frequency and quality of monitoring. Relying on surveyors specifically contracted for the task, as was the case in Santo Antônio, increases uncertainty as to whether funding will be sufficiently reliable to meet labor-law requirements. By concentrating on field infrastructure, we were able to take advantage of different forms of financing for monitoring in different places and time periods.

Costs of monitoring could not be too great if different stakeholders with limited economic resources were to be involved, but the system had to house high-technology systems, such as eddy-flux towers, when available. While it is not possible to foresee all stakeholders, it is possible to provide field infrastructure that most will need, such as access trails and permanent plots.

Because of its complexity, all aspects of biodiversity cannot be measured directly and decisions are made based on surrogates, which are usually maps derived from remote-sensing data, but may be simply the representation of one biological group by another. For instance, vascular plants are often used to identify “habitats”, “ecoregions” or “ecosystems” that purportedly represent boundaries to the distribution of other organisms, such as insects, mammals or fish (e.g. Olson et al. 2001, Higgins et al. 2005). The technology is continually advancing, and it is not possible to predict what remote-sensing products will be available in the future, but the greatest sources of uncertainty at the moment relate to the relationships between surrogates and the target organisms in which we are interested (Magnusson 2004, Franklin 2009, Caro 2010).

Biologists generally stratify and collect only where they “know” that certain types of organisms occur (Henle et al. 2006). When they are forced to sample regularly or randomly because that is where the infrastructure is, they almost always discover that their preconceived ideas were wrong (Oliveira et al. 2008), including ideas about the relationships between species distributions and remote-sensing surrogates. Other researchers involved in country-wide programs have found a priori stratification to be problematical (e.g. Johnson 2012, McDonald 2012). Infrastructure that is not dedicated to particular groups reduces the risk that spatial sampling will be dedicated to a particular taxon. Standardized infrastructure in RAPELD plots has allowed validation of surrogates for which geographic information system (GIS) layers were available (e.g. Schietti et al. 2013).

We were initially uncertain about both the questions that stakeholders would want to answer and the remote-sensing technology that would become available. However, most political decisions are made on scales of tens to hundreds of linear kilometers, and few researchers have the resources to use remote-sensing tools with pixel sizes of a few meters. Therefore, we designed a system with relatively large sampling units (250 m long plots and 5 km long transects) that would allow the use of a wide variety of remote-sensing products available today, and will allow the use of even more in the future as products with smaller pixel sizes come on line.

Many monitoring programs have fixed targets, such as the Alberta Biodiversity Monitoring Institute (ABMI – Haughland et al. 2009), the Center for Tropical Forest Science (CTFS – Condit 1988) and Tropical Ecology, Assessment and Monitoring (TEAM – Martins et al. 2007). Standardization of a limited range of targets (e.g. vascular plants in CTFS plots) facilitates standardization, but leaves much biodiversity without coverage. Standardization for many groups (e.g. ABMI and TEAM) greatly increases the costs, and all standardization of targets reduces the range of stakeholders who will participate and the range of questions that can be answered. Therefore, we developed a system that permits essentially all elements of biodiversity to be monitored, but does not require that all targets are monitored in every site.

Integration can be obtained by associating individual sites with larger initiatives for particular targets. For instance, RAPELD plots are included in the RAINFOR (e.g. Emílio et al. 2013), GVID (e.g. Pezzini et al. 2012) and ATDN (Stropp et al. 2009, ter Steege 2013) vegetation networks. The same plots have been surveyed for taxa as diverse as mites (Franklin et al. 2013), ants (Souza et al. 2012), frogs (Menin et al. 2007) and birds (Bueno et al. 2012). There is a logical trade-off. Very strong standardization reduces coverage, but too little standardization makes wide-scale syntheses impossible. Discovering the most appropriate targets for any particular question is an on-going process, so making a design that is only appropriate for one biological group is not an optimal strategy.

Most biologists now carry GPS devices, and geographic coordinates are the backbone of the Darwin-core system for digitalizing the information in biological collections. However, precise information about collection locations, or a single point representing the headquarters of park personnel in a large reserve are generally not sufficient to evaluate search effort or relate biological data to potential abiotic drivers. When we installed RAPELD modules in areas that had been intensively studied by other monitoring programs, we encountered many difficulties in avoiding disturbance to their plots, because the other programs did not have precise coordinates delimitating their field infrastructure.

RAPELD modules provide researchers with extremely detailed information on the location of trails, plots and large sessile organisms, such as trees. As RAPELD plots are long and thin, it is easy for researchers to locate their organisms quickly using only a compass and a measuring tape. All trails are marked at 50 m intervals (100 m intervals in some older sites), so researchers and local assistants can record relatively precise locations even when they do not have GPS equipment. This has been especially important for the use of RAPELD in environmental-impact studies, because reduction in area occupied is often a more sensitive measure of impact than attempts to estimate absolute numbers of organisms by mark-recapture methods (See “Uncertainty about detection” below).

Different aspects of biodiversity and the environment are usually studied by different researchers, and very often it is difficult to see what these researchers have in common. “Integrated” projects usually involve extensive discussion about how the funds will be divided and the general locations of study sites. However, when the time comes for analysis, it is usually impossible to integrate the different studies that were done on different temporal and spatial scales, even though all were contained within the same geographic envelope. Often, researchers from different disciplines have completely different concepts of what represents replication and independence of observations.

By installing a system of transects, and especially plots, that can be used by a wide range of disciplines, we were able to integrate many studies that previously had been considered too disparate for interdisciplinary studies (see also Henle et al. 2006). Almost all studies that have come out of RAPELD systems have integrated data from a variety of disciplines, but have generally focused on a limited range of biological taxa. However, the RAPELD spatial standardization and emphasis on data storage and availability have allowed integrated studies of concordance in the landscape distribution of different taxa (e.g. Landeiro et al. 2012).

Narrow plots that follow altitudinal contours have less internal variability in environmental predictor variables than conventional square plots, or long thin plots that are not oriented along contour lines (Henle et al. 2006, Castilho et al. 2010). This allows much more precise determination of relationships with predictors than possible with standard plots. As important as reducing uncertainty, is that this reduces costs, and allows detection of changes within a shorter period. For instance, Castilho et al. (2010) were able to show significant relationships between biomass accumulation and environmental predictors with 2-yr intervals between tree surveys in a RAPELD grid, at a fraction of the cost of the implementation of a single large plot, even though the differences would have been within the measurement error for a single large plot.

Most studies in RAPELD plots have investigated the relationships between topographic, soil or hydrological variables and organisms (e.g. Menin et al. 2007, Bueno et al. 2012, Baccaro et al. 2013, Emílio et al. 2013, Schietti et al. 2013), but some studies have investigated relationships among different groups of organisms (e.g. Baccaro et al. 2012, Landeiro et al. 2012) and studies are starting to investigate how relationships change over time (e.g. Espírito-Santo et al. 2009, Castilho et al. 2010). However, evaluation of geographic variation in these relationships based on RAPELD spatial designs replicated in different regions has been undertaken for few groups (e.g. Souza et al. 2012, Zuquim et al. 2012).

I have covered only a few of the sources of uncertainty that a field monitoring scheme has to deal with, but they illustrate the complexity of the problem. We are monitoring because we are uncertain, so uncertainty has to be a central issue in any monitoring scheme. However, biodiversity monitoring is more complex than monitoring physical phenomena, such as weather, because there are so many definitions of biodiversity, and it has different values for different segments of society (see Haila et al. 2014).

When we first implemented RAPELD, we opted for a hierarchical design, with regular sampling at the local level, but sampling sites limited by logistical considerations over larger areas. Although there was little theoretical support for it at the time, it has been useful to respond to a wide range of questions of interest to decision makers (Magnusson et al. 2013), it is similar to designs adopted by other researchers faced with sampling biodiversity over large areas (e.g. McDonald 2012, Olsen et al. 2012), and it is a design that has since been approved by experienced modelers for continent-wide monitoring (e.g. Franklin 2009, Johnson 2012, Reynolds 2012). Because the design is modular, it can be adapted to a wide range of situations. At first, we envisaged that most sites would have 5 km × 5 m grids, but the most favored module today is 5 km × 1 km. We were uncertain about everything and adopted a learning-by-doing approach, while always trying to maximize comparability with data collected previously. We are more surprised at how little we have had to modify the original design than by the changes we have implemented (Magnusson et al. 2013).

We were even uncertain about the questions that researchers will want to answer in the future using the data we are collecting today. Therefore, rather than adopting a taxon- or question-oriented approach, we focused on a spatial design that allows flexibility in questions and taxa studied, while allowing the landscape-geographical approaches required by conservation biology. It is a compromise, but a necessary one. RAPELD sites are being used to answer many specific questions, so the trade-off is generally not between monitoring and answering specific questions. It is between investing in field infrastructure and planning now, rather than in short-term studies that are good for researcher curricula, but that contribute little to long-term, wide-scale conservation planning. At the moment, many citizen monitoring programs, combined with good data availability, are contributing more to our understanding of global phenomena, such as climate change, than are more scientific programs (Schmeller et al. 2009), despite criticisms from academics (e.g. Ferraz et al. 2008). We professional scientists need to learn from the amateurs, and incorporate planning, standardization, and data availability, while maintaining the flexibility that uncertainty demands.

The ideas expressed in this paper were developed by too many people to cite here, but all are participants in the Instituto Nacional de Ciência Tecnologia - Centro de Estudos Integrados da Biodiversidade Amazônica (INCT-CENBAM), and the Brazilian Program for Biodiversity Research (PPBio). I thank Klaus Henle for the opportunity to participate in the workshop and the stimulus for the important interactions through EU Bon (EU FP project grant number 308454).