Blaza Toman

Assessment of measurement uncertainty via observation equations

According to the Guide to the Expression of Uncertainty in Measurement (GUM) (1995, Geneva, Switzerland: International Organization for Standardization (ISO)), the uncertainty in an estimate of the value of a measurand is assessed by propagating the uncertainty in estimates of values of input quantities, based on a measurement equation that expresses the former value as a known function of the latter values. However, in measurement situations where some of the input quantities in turn depend on the measurand, this approach is circuitous and ultimately impracticable.An alternative approach starts from the observation equation, which relates the experimental data to the measurand: this allows a uniform treatment of the most diverse metrological problems, and, once it is used in the context of Bayesian inference, also facilitates the exploitation of any information that may pre-exist about the measurand, alongside the information that fresh experimental data provide about it.The widest applicability of the observation equation approach is illustrated with detailed examples concerning the lifetime of mechanical parts, the measurement of mass, the calibration of a non-linear model in biochemistry and the estimation of a consensus value for arsenic concentration in a sample measured by multiple laboratories.

Standard Reference Material 2366 for Measurement of Human Cytomegalovirus DNA

Human cytomegalovirus (CMV), classified as human herpesvirus 5, is ubiquitous in human populations. Infection generally causes little illness in healthy individuals, but can cause life-threatening disease in those who are immunocompromised or in newborns through complications arising from congenital CMV infection. An important aspect in diagnosis and treatment is to track circulating viral load with molecular methods, particularly with quantitative PCR. Standardization is vital, because of interlaboratory variability (due in part to the variety of assays and calibrants). Toward that end, the U.S. National Institute of Standards and Technology produced a Standard Reference Material 2366 appropriate for establishing metrological traceability of assay calibrants. This standard is composed of CMV DNA (Towne(Δ147) bacterial artificial chromosome DNA). Regions of the CMV DNA that are commonly used as targets for PCR assays were sequenced. Digital PCR was used to quantify the DNA, with concentration expressed as copies per microliter. The materials were tested for homogeneity and stability. An interlaboratory study was conducted by Quality Control for Molecular Diagnostics (Glasgow, UK), in which one component of SRM 2366 was included for analysis by participants in a CMV external quality assessment and proficiency testing program.

Bayesian uncertainty analysis under prior ignorance of the measurand versus analysis using the Supplement 1 to the Guide: a comparison

A recent supplement to the GUM (GUM S1) is compared with a Bayesian analysis in terms of a particular task of data analysis, one where no prior knowledge of the measurand is presumed. For the Bayesian analysis, an improper prior density on the measurand is employed. It is shown that both approaches yield the same results when the measurand depends linearly on the input quantities, but generally different results otherwise. This difference is shown to be not a conceptual one, but due to the fact that the two methods correspond to Bayesian analysis under different parametrizations, with ignorance of the measurand expressed by a non-informative prior on a different parameter. The use of the improper prior for the measurand itself may result in an improper posterior probability density function (PDF) when the measurand depends non-linearly on the input quantities. On the other hand, the PDF of the measurand derived by the GUM supplement method is always proper but may sometimes have undesirable properties such as non-existence of moments.It is concluded that for a linear model both analyses can safely be applied. For a non-linear model, the GUM supplement approach may be preferred over a Bayesian analysis using a constant prior on the measurand. But since in this case the GUM S1 PDF may also have undesirable properties, and as often some prior knowledge about the measurand may be established, metrologists are strongly encouraged to express this prior knowledge in terms of a proper PDF which can then be included in a Bayesian analysis. The results of this paper are illustrated by an example of a simple non-linear model.

Analysis of key comparisons: estimating laboratories' biases by a fixed effects model using Bayesian model averaging

We propose a novel procedure for the analysis of key comparison data. The goal of the procedure is to detect biases in the reported measurement results which are not accounted for by quoted uncertainties. A fixed effects bias model is employed which constrains the biases of some of the laboratories to zero. Only the number of these laboratories needs to be specified, not the laboratories themselves. The analysis then runs through all possible different models, each assuming zero biases for a different subset of laboratories. The results from these models are finally merged by employing a Bayesian model averaging technique. Explicit formulae are derived which allow for an easy application of the proposed approach. The procedure is illustrated by its application to data from the CCL-K1 key comparison.

Organic Reference Materials for Hydrogen, Carbon, and Nitrogen Stable Isotope-Ratio Measurements: Caffeines, n-Alkanes, Fatty Acid Methyl Esters, Glycines, l-Valines, Polyethylenes, and Oils

An international project developed, quality-tested, and determined isotope-δ values of 19 new organic reference materials (RMs) for hydrogen, carbon, and nitrogen stable isotope-ratio measurements, in addition to analyzing pre-existing RMs NBS 22 (oil), IAEA-CH-7 (polyethylene foil), and IAEA-600 (caffeine). These new RMs enable users to normalize measurements of samples to isotope-δ scales. The RMs span a range of δ(2)H(VSMOW-SLAP) values from -210.8 to +397.0 mUr or ‰, for δ(13)C(VPDB-LSVEC) from -40.81 to +0.49 mUr and for δ(15)N(Air) from -5.21 to +61.53 mUr. Many of the new RMs are amenable to gas and liquid chromatography. The RMs include triads of isotopically contrasting caffeines, C16 n-alkanes, n-C20-fatty acid methyl esters (FAMEs), glycines, and l-valines, together with polyethylene powder and string, one n-C17-FAME, a vacuum oil (NBS 22a) to replace NBS 22 oil, and a (2)H-enriched vacuum oil. A total of 11 laboratories from 7 countries used multiple analytical approaches and instrumentation for 2-point isotopic normalization against international primary measurement standards. The use of reference waters in silver tubes allowed direct normalization of δ(2)H values of organic materials against isotopic reference waters following the principle of identical treatment. Bayesian statistical analysis yielded the mean values reported here. New RMs are numbered from USGS61 through USGS78, in addition to NBS 22a. Because of exchangeable hydrogen, amino acid RMs currently are recommended only for carbon- and nitrogen-isotope measurements. Some amino acids contain (13)C and carbon-bound organic (2)H-enrichments at different molecular sites to provide RMs for potential site-specific isotopic analysis in future studies.

Comparison of ISO-GUM, draft GUM Supplement 1 and Bayesian statistics using simple linear calibration

We compare three approaches for quantifying uncertainty through a measurement equation: the International Organization for Standardization (ISO) Guide to the Expression of Uncertainty in Measurement (GUM), draft GUM Supplement 1 and Bayesian statistics. For illustration, we use a measurement equation for simple linear calibration that includes both Type A and Type B input variables. We consider three scenarios: (i) the measurement equation is linear with one Type B input variable having a normal distribution, (ii) the measurement equation is non-linear with two Type B input variables each having a normal distribution and (iii) the measurement equation is non-linear with two Type B input variables each having a rectangular distribution. We consider both small and large uncertainties for the Type B input variables. We use each of the three approaches to quantify the uncertainty in measurement for each of the three scenarios. Then we discuss the merits and limitations of each approach.

Bayesian approaches to calculating a reference value in key comparison experiments

International experiments called key comparisons pose an interesting statistical problem: estimation of a quantity called a reference value. This estimator can take many possible forms; so far, none has been accepted as a standard. Recently, this topic has received much international attention. In this article it is argued that a fully Bayesian approach to this problem is compatible with the current practice of metrology and can be used to create models for the varied properties and assumptions of these experiments.

Quantifying the predictive uncertainty of complex numerical models

The overall uncertainty of a model prediction is a combination of the uncertainty of the input parameters and the uncertainty of the model assumptions. The former is referred to as parameter uncertainty; the latter model uncertainty. A method for quantifying model uncertainty is proposed for complicated numerical models that are not amenable to more traditional approaches. The method is based solely on comparisons of model predictions with experimental measurements, with the difference reported in terms of only two metrics, a bias factor and a relative standard deviation. The simplicity of the approach makes it ideal for models used for regulatory compliance because approving authorities often lack detailed training in modelling and uncertainty analysis.

Bayesian uncertainty analysis for a regression model versus application of GUM Supplement 1 to the least-squares estimate

Application of least-squares as, for instance, in curve fitting is an important tool of data analysis in metrology. It is tempting to employ the supplement 1 to the GUM (GUM-S1) to evaluate the uncertainty associated with the resulting parameter estimates, although doing so is beyond the specified scope of GUM-S1. We compare the result of such a procedure with a Bayesian uncertainty analysis of the corresponding regression model. It is shown that under certain assumptions both analyses yield the same results but this is not true in general. Some simple examples are given which illustrate the similarities and differences between the two approaches.

Analysis of key comparison data: critical assessment of elements of current practice with suggested improvements

The degrees of equivalence can be viewed as possibly the main result in the analysis of key comparison data. Their specification as given in the CIPM MRA is discussed and critically assessed in this paper. We argue that there is an ambiguity in the definition and meaning of the (unilateral) degrees of equivalence. As a consequence of this ambiguity uncertainties quoted for (unilateral) degrees of equivalence may be questioned. The ambiguity can be avoided by identifying the quantities that are being estimated by the degrees of equivalence, and we propose a standard statistical model to do this.We then show that the degrees of equivalence are not a unique measure of consistency between the results and the underlying measurand when determined solely from the data. Prior knowledge or additional assumptions are needed for this purpose, and Bayesian methods are particularly suitable to handle that. However, such measures of consistency depend on the chosen additional assumptions and generally are not in accordance with the current CIPM MRA.Fortunately, quantifying consistency between the results and the underlying measurand is not necessary in order to assess equivalence between the laboratories. We show that on the basis of the (unambiguous) pairwise degrees of equivalence the laboratories can be grouped into equivalent subsets, the largest of which may be chosen to select those laboratories whose CMCs are then viewed as validated.