Susan M. Cormier

Weight-of-evidence evaluation in environmental assessment: Review of qualitative and quantitative approaches

Assessments of human health and ecological risk draw upon multiple types and sources of information, requiring the integration of multiple lines of evidence before conclusions may be reached. Risk assessors often make use of weight-of-evidence (WOE) approaches to perform the integration, whether integrating evidence concerning potential carcinogenicity, toxicity, and exposure from chemicals at a contaminated site, or evaluating processes concerned with habitat loss or modification when managing a natural resource. Historically, assessors have relied upon qualitative WOE approaches, such as professional judgment, or limited quantitative methods, such as direct scoring, to develop conclusions from multiple lines of evidence. Current practice often lacks transparency resulting in risk estimates lacking quantified uncertainty. This paper reviews recent applications of weight of evidence used in human health and ecological risk assessment. Applications are sorted based on whether the approach relies on qualitative and quantitative methods in order to reveal trends in the use of the term weight of evidence, especially as a means to facilitate structured and transparent development of risk conclusions from multiple lines of evidence.

Derivation of a benchmark for freshwater ionic strength.

Because increased ionic strength has caused deleterious ecological changes in freshwater streams, thresholds for effects are needed to inform resource-management decisions. In particular, effluents from surface coal mining raise the ionic strength of receiving streams. The authors developed an aquatic life benchmark for specific conductance as a measure of ionic strength that is expected to prevent the local extirpation of 95% of species from neutral to alkaline waters containing a mixture of dissolved ions in which the mass of SO (4)2- + HCO (3)- ≥ Cl(-). Extirpation concentrations of specific conductance were estimated from the presence and absence of benthic invertebrate genera from 2,210 stream samples in West Virginia. The extirpation concentration is the 95th percentile of the distribution of the probability of occurrence of a genus with respect to specific conductance. In a region with a background of 116 µS/cm, the 5th percentile of the species sensitivity distribution of extirpation concentrations for 163 genera is 300 µS/cm. Because the benchmark is not protective of all genera and protects against extirpation rather than reduction in abundance, this level may not fully protect sensitive species or higher-quality, exceptional waters.

Why and how to combine evidence in environmental assessments: Weighing evidence and building cases

All types of environmental decisions benefit from assessments that assemble and analyze diverse evidence. The diversity of that evidence creates complexities that can be managed using an explicit, well-designed process. We suggest two adaptations from the legal lexicon, weight of evidence and building a case. When weighing evidence, weights are assigned to each piece of evidence, and then the body of evidence is weighed in favor of each hypothesis by amassing the weights. Finally, the total weights of evidence for the alternative hypotheses are compared to determine which alternative has the preponderance of evidence in its favor. When building a case, pieces of evidence are organized to show relationships among multiple hypotheses or complex interactions among agents, events, or processes. We provide processes for weighing evidence and building a case and illustrate both approaches in a case study involving the decline of a kit fox population. The general approach presented here is flexible, transparent, and defensible. During its development, it has been applied to risk assessments for contaminated sites and to causal assessments in aquatic and terrestrial systems. It is intended to balance the need for rigor and discipline with the need for sufficient flexibility to accept all relevant evidence and generate creative solutions to difficult environmental problems.

METHODS DEVELOPMENT AND USE OF MACROINVERTEBRATES AS INDICATORS OF ECOLOGICAL CONDITIONS FOR STREAMS IN THE MID-ATLANTIC HIGHLANDS REGION

The Mid-Atlantic Highlands Assessment (MAHA) included the sampling of macroinvertebrates from 424 wadeable stream sites to determine status and trends, biological conditions, and water quality in first through third order streams in the Mid-Atlantic Highlands Region (MAHR) of the United States in 1993–1995. We identified reference and impaired sites using water chemistry and habitat criteria and evaluated a set of candidate macroinvertebrate metrics using a stepwise process. This process examined several metric characteristics, including ability of metrics to discriminate reference and impaired sites, relative scope of impairment, correlations with chemical and habitat indicators of stream disturbance, redundancy with other metrics, and within-year variability. Metrics that performed well were compared with metrics currently being used by three states in the region: Pennsylvania, Virginia, and West Virginia. Some of the metrics used by these states did not perform well when evaluated using regional data, while other metrics used by all three states in some form, specifically number of taxa, number of EPT taxa, and Hilsenhoff Biotic Index, performed well overall. Reasons for discrepancies between state and regional evaluations of metrics are explored. We also provide a set of metrics that, when used in combination, may provide a useful assessment of stream conditions in the MAHR.

Development and evaluation of the Lake Macroinvertebrate Integrity Index (LMII) for New Jersey lakes and reservoirs.

In response to the recent focus by the U.S. EnvironmentalProtection Agency on bioassessment of lakes, a multimetric index was developed for New Jersey lakes and reservoirs using benthicmacroinvertebrates. Benthic samples were collected fromreference and impaired lakes with muck and intermediate sedimentsin central and northern New Jersey during summer 1997. We used astepwise process to evaluate properties of candidate metrics andselected five for the Lake Macroinvertebrate Integrity Index(LMII): Hilsenhoff Biotic Index (HBI), percent chironomidindividuals, percent collector-gatherer taxa, percentoligochaetes/leeches, and number of Diptera taxa. We scoredmetrics as the fraction of the best expected value (based on allsites) achieved at a site and summed them into the LMII. Evaluation of the LMII showed that it discriminated well betweenreference and impaired lakes and was strongly related to severalpotential stressors. Chemical and physical gradients distinguished between reference and impaired lakes, and the LMIIsummarized these gradients well. The LMII corresponded stronglywith land use, but some lakes with more urban land use stillachieved high scores. Based on a power analysis, the ability ofthe LMII to detect differences in condition was sensitive to thenumber of samples from each lake.

Assessing causation of the extirpation of stream macroinvertebrates by a mixture of ions

Increased ionic concentrations are associated with the impairment of benthic invertebrate assemblages. However, the causal nature of that relationship must be demonstrated so that it can be used to derive a benchmark for conductivity. The available evidence is organized in terms of six characteristics of causation: co-occurrence, preceding causation, interaction, alteration, sufficiency, and time order. The inferential approach is to weight the lines of evidence using a consistent scoring system, weigh the evidence for each causal characteristic, and then assess the body of evidence. Through this assessment, the authors found that a mixture containing the ions Ca(+), Mg(+), HCO 3(-), and SO 4(-), as measured by conductivity, is a common cause of extirpation of aquatic macroinvertebrates in Appalachia where surface coal mining is prevalent. The mixture of ions is implicated as the cause rather than any individual constituent of the mixture. The authors also expect that ionic concentrations sufficient to cause extirpations would occur with a similar salt mixture containing predominately HCO 3(-), SO 4(2-), Ca(2+), and Mg(2+) in other regions with naturally low conductivity. This case demonstrates the utility of the method for determining whether relationships identified in the field are causal.

A Framework for Fully Integrating Environmental Assessment

A new framework for environmental assessment is needed because no existing framework explicitly includes all types of environmental assessments. We propose a framework that focuses on resolving environmental problems by integrating different types of assessments. Four general types of assessments are included: (1) condition assessments to detect chemical, physical, and biological impairments; (2) causal pathway assessments to determine causes and identify their sources; (3) predictive assessments to estimate environmental, economic, and societal risks, and benefits associated with different possible management actions; and (4) outcome assessments to evaluate the results of the decisions of an integrative assessment. The four types of assessments can be neatly arrayed in a two-by-two matrix based on the direction of analysis of causal relationships (rows) and whether the assessment identifies problems or solves them (columns). We suggest that all assessments have a common structure of planning, analysis, and synthesis, thus simplifying terminology and facilitating communication between types of assessments and environmental programs. The linkage between assessments is based on intermediate decisions that initiate another assessment or a final decision signaling the resolution of the problem. The framework is applied to three cases: management of a biologically impaired river, remediation of a contaminated site, and reregistration of a pesticide. We believe that this framework clarifies the relationships among the various types of assessment processes and their links to specific decisions.

Using field data and weight of evidence to develop water quality criteria

ABSTRACT In the United States, ambient aquatic life water quality criteria are derived using guidelines developed in 1985 that include a clear and consistent methodology using data from standard toxicity tests. The methodology from these guidelines has been successful, but a broader methodology is needed because some effects of pollutants do not lend themselves to conventional toxicity testing. Criterion assessment is proposed as that methodology. In criterion assessment, a specific environmental goal is translated into a measurable benchmark of effect that is used together with a modeled exposure–response relationship to estimate a range of exposures that will achieve the specific goal. The model of the exposure–response relationships and the benchmark effect are developed from field data and laboratory data using multiple analytical methods. Then the model is solved for the effect, thereby estimating the criterion, an upper threshold for acceptable exposures. The resulting candidate criteria are synthesized to select criteria and other benchmark values, such as remedial goals. The criterion assessment process is illustrated using the US Environmental Protection Agency Framework for Developing for Suspended and Bedded Sediments Water Quality Criteria, which recommends developing alternative candidate criterion values and then evaluating them to select a final criterion. Candidate criteria may be derived from models of field observations, field manipulations, laboratory tests, or empirical and theoretical models. Final selection of a criterion uses a weight-of-evidence comparison that engenders confidence because causal associations are confirmed on the basis of different assumptions, independent data sets, and varied statistical methods, thereby compensating for the concerns raised by individual studies and methods. Thus, it becomes possible to specify criteria for agents with biological or physical modes of action, as well as those with chemical modes of action, to best achieve environmental goals.

CADDIS: The Causal Analysis/Diagnosis Decision Information System

Biological monitoring and assessment methods have become indispensable tools for evaluating the condition of aquatic and terrestrial ecosystems. When an undesirable biological condition is observed (e.g., a depauperate fish assemblage), its cause (e.g., toxic substances, excess fine sediments, or nutrients) must be determined in order to design appropriate remedial management actions. Causal analysis challenges environmental scientists to bring together, analyze, and synthesize a broad variety of information from monitoring studies, models, and experiments to determine the probable cause of ecological effects. Decision-support systems can play an important role in improving the efficiency, quality and transparency of causal analyses.

The Science and Philosophy of a Method for Assessing Environmental Causes

ABSTRACT When an environmental impairment has been identified, it becomes necessary to identify the cause so that an appropriate action can be planned. However, causation is difficult to establish—both conceptually and in practice. To ensure that the U.S. Environmental Protection Agencys (USEPAs) method for causal assessment is appropriate and defensible, we reviewed concepts of causation from philosophers, statisticians, epidemiologists, and others. This article summarizes the results of that review and explains how it relates to the USEPAs method. We include a five-step process: (1) identify alternative candidate causes; (2) logically eliminate when possible; (3) diagnose when possible; (4) analyze the strength of evidence for remaining candidate causes; and (5) identify the most likely cause. We also encourage three practices: (1) use a consistent process; (2) do not claim proof of causation; and (3) document the evidence and inferences. This approach allows assessors to identify the most likely cause or, failing that, to reduce the set of possible causes and identify information needs for another iteration of causal assessment.