John M. Nocerino

Robust estimation of mean and variance using environmental data sets with below detection limit observations

Scientists, especially environmental scientists, often encounter trace level concentrations that are typically reported as less than a certain limit of detection, L. Type I left-censored data arises when certain low values lying below L are ignored or unknown as they cannot be measured accurately. In many environmental quality assurance and quality control (QA/QC), and groundwater monitoring applications of the United States Environmental Protection Agency (USEPA), values smaller than L are not required to be reported. However, practitioners still need to obtain reliable estimates of the population mean μ, and the standard deviation (S.D.) σ. The problem gets complex when a small number of high concentrations are observed with a substantial number of concentrations below the detection limit. The high-outlying values contaminate the underlying censored sample, leading to distorted estimates of μ and σ. The USEPA, through the National Exposure Research Laboratory-Las Vegas (NERL-LV), under the Office of Research and Development (ORD), has research interests in developing statistically rigorous robust estimation procedures for contaminated left-censored data sets. Robust estimation procedures based upon a proposed (PROP) influence function are shown to result in reliable estimates of population parameters of mean and S.D. using contaminated left-censored samples. It is also observed that the robust estimates thus obtained with or without the outliers are in close agreement with the corresponding classical estimates after the removal of outliers. Several classical and robust methods for the estimation of μ and σ using left-censored (truncated) data sets with potential outliers have been reviewed and evaluated.

Gy sampling theory in environmental studies 2. Subsampling error estimates

Sampling can be a significant source of error in the measurement process. The characterization and cleanup of hazardous waste sites require data that meet site-specific levels of acceptable quality if scientifically supportable decisions are to be made. In support of this effort, the US Environmental Protection Agency (EPA) is investigating methods that relate sample characteristics to analytical performance. Predicted uncertainty levels allow appropriate study design decisions to be made, facilitating more timely and less expensive evaluations. Gy sampling theory can predict a significant fraction of sampling error when certain conditions are met. We report on several controlled studies of subsampling procedures to evaluate the utility of Gy sampling theory applied to laboratory subsampling practices. Several sample types were studied and both analyte and non-analyte containing particles were shown to play important roles affecting the measured uncertainty. Gy sampling theory was useful in predicting minimum uncertainty levels provided the theoretical assumptions were met. Predicted fundamental errors ranged from 46 to 68% of the total measurement variability. The study results also showed sectorial splitting outperformed incremental sampling for simple model systems and suggested that sectorial splitters divide each size fraction independently. Under the limited conditions tested in this study, incremental sampling with a spatula produced biased results when sampling particulate matrices with grain sizes about 1 mm.

Role of Laboratory Sampling Devices and Laboratory Subsampling Methods in Representative Sampling Strategies

Sampling is the act of selecting items from a specified population in order to estimate the parameters of that population (e.g., selecting soil samples to characterize the properties at an environmental site). Sampling occurs at various levels and times throughout an environmental site characterization process. Typically, initial (primary) sampling occurs in the field while subsequent stages of sample size reduction (subsampling) occur until the final laboratory analysis stage. At each step in the measurement process, from planning, site selection, sample collection, sample preparation, through sample analysis, errors can occur that propagate, leading to uncertainty associated with the final result upon which decisions will ultimately be made. The goal of all sampling efforts should be to select samples that are representative of the population (i.e., site) in question. General guidelines, with supporting background and theory, for obtaining representative subsamples for the laboratory analysis of particulate materials using “correct” sampling practices and “correct” sampling devices are presented (“correct” as defined by Gy sampling theory; see Pitard, 1993). Considerations are given to: the constitution and the degree of heterogeneity of the material being sampled, the methods used for sample collection (including what proper tools to use), what it is that the sample is supposed to represent, the mass of the sample needed to be representative, and the bounds of what “representative” actually means.

Robust Procedures for the Identification of Multiple Outliers

Classical and robust/resistant procedures for the estimation of population parameters and the identification of multiple outliers in univariate and multivariate populations are reviewed. The successful identification of anomalous observations depends on the statistical procedures employed. Commercial industries, local communities, and government agencies such as the United States Environmental Protection Agency (U.S. EPA), often need to assess the extent of contamination at polluted sites. Identification of these contaminants having potentially adverse effects on human health is especially important in various ecological and environmental applications. An environmental scientist typically generates and analyzes large amounts of multidimensional data. These practioners often need to identify experimental conditions and results which look suspicious and are significantly different from the rest of the data. The classical Mahalanobis distance (MD) and its variants (e.g., multivariate kurtosis) are routinely used to identify these anomalies. These test statistics depend upon the estimates of population location and scale. The presence of anomalous observations usually results in distorted and unreliable maximum likelihood estimates (MLEs) and ordinary least-squares (OLS) estimates of the population parameters. These in turn result in deflated and distorted classical MDs and lead to masking effects. This means that the results from statistical tests and inference based upon these classical estimates may be misleading. For example, in an environmental monitoring application, it is possible that the classification procedure based upon the distorted estimates may classify a contaminated sample as coming from the clean population and a clean sample as coming from the contaminated part of the site. This in turn can lead to incorrect remediation decisions.

Experimental Design and Optimization

The application of statistical experimental design and optimization (SEDOP) to environmental chemistry research is presented. The use of SEDOP approaches for environmental research has the potential to increase the amount of information and the reliability of results, at a cost comparable to, or lower than, traditional approaches. We demonstrate how researchers can attain these benefits by adhering to a systematic program of design and execution of experiments, including the analysis and interpretation of results. The lack of general knowledge about experimental statistical techniques had hindered their widespread application in the environmental field. To benefit from the SEDOP advantages, the United States Environmental Protection Agency (USEPA) has an ongoing project to investigate applications of statistical design to environmental chemistry problems. There exist standard experimental arrangements (designs) to address all phases of a research program, from identifying important effects, to modeling the behavior of the experimental system of interest, to optimizing the operating conditions (e.g., minimizing waste or maximizing reproducibility). The most useful standard design arrangements (both for system characterization and optimization) are introduced, together with a discussion of their applicability to pollutant analysis as well as their strengths and weaknesses. Practical environmental applications from the literature are presented and discussed from the perspective of the approaches and techniques that they illustrate. Examples include optimization of analyte extraction, instrument calibration, method comparison, ruggedness testing, selection of indicator contaminants, and pollution prevention. The implementation of statistical experimental design today is greatly facilitated by the use of available software for the selection of designs, the planning of experiments, the analysis of data, and the graphical presentation of results.

Robust intervals for some environmental applications

Abstract In an effort to keep the environment clean, local communities, commercial industries, and government agencies, such as the United States Environmental Protection Agency (US EPA), need to assess the extent of contamination at polluted sites. The site characterization results are then used to establish approaches for remediation, to determine background threshold levels of the pollutants of concern, and to make decisions on various environmental monitoring and remediation activities. Some aspects of site characterization depend upon the chemical analyses of soil or water samples collected at the site. In Superfund and other applications these environmental samples are routinely analyzed by the various laboratories participating in quality assurance/quality control (QA/QC) programs, such as the Contract Laboratory Program (CLP) of the US EPA. The performance of those laboratories is typically monitored through statistical quality control (SQC) techniques requiring the use of the estimates of population parameters of location and scale. However, outlying observations, when present, can distort the entire estimation process, which in turn can lead to incorrect decisions. In order to address these issues, several interval estimates have been discussed. The robust procedure based upon the PROP influence function identifies multiple outliers successfully and provides reliable estimates of population parameters. The weights assigned to individual observations are used to obtain estimates of the degrees of freedom (d.f.) associated with the Students t and other relevant statistics. The appropriate use of these intervals together with the robust estimates of the population parameters and of the associated degrees of freedom result in more accurate and precise statistical regions to be used in these applications.