Matthias Greiner

Principles and practical application of the receiver-operating characteristic analysis for diagnostic tests.

We review the principles and practical application of receiver-operating characteristic (ROC) analysis for diagnostic tests. ROC analysis can be used for diagnostic tests with outcomes measured on ordinal, interval or ratio scales. The dependence of the diagnostic sensitivity and specificity on the selected cut-off value must be considered for a full test evaluation and for test comparison. All possible combinations of sensitivity and specificity that can be achieved by changing the tests cut-off value can be summarised using a single parameter; the area under the ROC curve. The ROC technique can also be used to optimise cut-off values with regard to a given prevalence in the target population and cost ratio of false-positive and false-negative results. However, plots of optimisation parameters against the selected cut-off value provide a more-direct method for cut-off selection. Candidates for such optimisation parameters are linear combinations of sensitivity and specificity (with weights selected to reflect the decision-making situation), odds ratio, chance-corrected measures of association (e. g. kappa) and likelihood ratios. We discuss some recent developments in ROC analysis, including meta-analysis of diagnostic tests, correlated ROC curves (paired-sample design) and chance- and prevalence-corrected ROC curves.

Epidemiologic issues in the validation of veterinary diagnostic tests

In this review, we critically discuss the objectives, methods and limitations of different approaches for the validation of diagnostic tests. We show (based on published data and our own experiences) that estimates for the diagnostic sensitivity and specificity may vary among populations and/or subpopulations of animals, conditional on the distribution of influential covariates. Additional variability in those parameter estimates may be attributable to the sampling strategy. The uncertainty about diagnostic parameters is of concern for the decision-maker in the context of clinical diagnosis or quantitative risk assessment as well as for the epidemiologist who uses test data for prevalence estimation or risk-factor studies. Examples for the calculation of diagnostic parameters are presented together with bias-avoidance strategies. We suggest guidelines for an epidemiologic approach to test validation of veterinary diagnostic tests.

Application of diagnostic tests in veterinary epidemiologic studies

Important applications of diagnostic tests in non-clinical areas of veterinary medicine include surveillance, monitoring or screening for disease, prevalence estimation, and risk-factor studies. In this review, we address the processing and analysis of diagnostic data with special emphasis on serological data from enzyme-linked immunosorbent assays (ELISAs). We describe methods that are used to adjust for misclassification (imperfect sensitivity and specificity) in the different applications of diagnostic tests. These methods often have limitations attributable to the uncertainty of sensitivity and specificity estimates. We review descriptive and analytical statistical methods that allow adequate presentation of non-clinical diagnostic data.

Two-graph receiver operating characteristic (TG-ROC): a Microsoft-EXCEL template for the selection of cut-off values in diagnostic tests

TG-ROC, a template for Microsoft-EXCEL, represents a novel, easy-to-handle approach for selecting cut-off values in quantitative diagnostic tests. In addition to graphical representations of test efficiency, Youden index and likelihood ratios as functions of the preselected cut-off value, the software supports the definition of an intermediate range of test results. For this purpose, two cut-off values are established that realise a pre-selected accuracy level (e.g., 90 or 95% sensitivity and specificity) which can be specified by the user.

Two-graph receiver operating characteristic (TG-ROC): update version supports optimisation of cut-off values that minimise overall misclassification costs

TG-ROC was recently introduced as a novel PC-assisted device for cut-off optimisation and evaluation of quantitative serodiagnostic tests (Greiner, 1995, J. Immunol. Methods 185, 145; Greiner et al., 1995, J. Immunol. Methods 185, 123). In this notice a recent extension of the programme is described which facilitates the cut-off selection when estimates of costs associated with false-positive and false-negative results are available.

A field study to estimate the prevalence of Trypanosoma equiperdum in Mongolian horses

From May to July 2000, a cross-sectional study was conducted to estimate the prevalence of Trypanosoma equiperdum in the horse population of the central province (Tuv aimag) of Mongolia. On average, four herds were selected from each of the 29 aimag subdivisions (119 herds). From each herd, 10 horses were sampled in proportion to sex and age categories in the respective herds (1190 horses). Sera from 1122 horses were analysed for T. equiperdum antibodies using two serological assays, the complement fixation test (CFT) and the enzyme-linked immunosorbent assay (ELISA). The crude estimate of the CFT and the ELISA seroprevalence was 7.6 and 6.7%, respectively. Concordance between the CFT and ELISA results was high (96%). The highest number of CFT positive animals was detected in one herd in Möngönmorit (6/10), followed by herds in Bayandelger (5/10) and in Bayantsagaan (5/10). Poor body condition was significantly correlated with positive serological status in both CFT and ELISA. A history of abortion appeared to be a risk factor for both CFT and ELISA seropositivity. Blood samples of all horses belonging to herds with at least three (3/10) seropositive animals (CFT and/or ELISA) were analysed by light microscopy and by PCR using a Trypanosoma (Trypanozoon) brucei specific primer pair. No trypanosomes or any other haemoparasites could be detected in Giemsa stained thin blood smears. Eight out of the 130 samples (6.2%) analysed by PCR gave positive signals. Seven out of the eight PCR positive horses were also serologically positive. One PCR (and ELISA) positive stallion from Möngönmorit showed emaciation, scrotal and preputial oedema and an oedematous skin plaque. From the serological and DNA-based results it is concluded, that trypanosome infections occur in horses in the Tuv aimag of Mongolia. Since at present neither serological nor DNA-based tests allow a subspecies specific identification within the subgenus Trypanozoon, no definitive diagnosis can be given for T. equiperdum. Whether the examined herds are infected with T. equiperdum or with T. evansi, the causative agent of surra, remains an open question. However, based on the clinical findings, the negative parasitological results and the concentration of conspicuous seroprevalences in single herds, circumstantial evidence supports the existence of infections with the causative agent of dourine.

The selection of ELISA cut-off points for testing antibody to Newcastle disease by two-graph receiver operating characteristic (TG-ROC) analysis

Two hundred and sixty field serum samples were tested for Newcastle disease (ND) antibodies using a commercial enzyme-linked immunosorbent assay (ELISA) and the haemagglutination inhibition test (HI). The HI test was regarded as the reference method. Reciprocal titres of 16 and above were considered positive. In this study the two-graph receiver operating characteristic (TG-ROC) analysis was used as a tool for selecting cut-off points. Sensitivity, specificity, efficiency and Youdens index were used as indices of test accuracy. The positive and negative predictive values of the ELISA results were analysed for various prevalence rates.

Estimation of dietary intake of bis(2-ethylhexyl)phthalate (DEHP) by consumption of food in the German population ☆

In the study presented here, we evaluated the exposure of the German population aged 14-80 years to bis(2-ethylhexyl)phthalate (DEHP) from consumption of food by means of deterministic and probabilistic estimations. The study was performed on the basis of an extensive review of literature from around the world reporting measured data on DEHP in food, as well as official German food control data. Only data from individual measurements were considered and used for fitting of distributions. A wide range of concentrations in non-representative samples are reported in the literature. On the basis of the available DEHP concentration data, 37 food categories were characterized which covered all major food classes. Food consumption data were taken from the diet history interviews of the German National Nutrition Survey II (Nationale Verzehrsstudie II) which was performed in 2005/2006 in a representative study population of 15,371 and is the most recent data source of this kind in Germany. Average DEHP intake was estimated deterministically using data on measured concentrations in food (medians and means) and food consumption (means). A total dietary exposure to DEHP of 3.6 (median based) and 9.3μg/kg of BW per day (based on mean values) was estimated deterministically. In addition, distributions of both concentrations and consumption figures were fitted using the @RISK best fit tool for further probabilistic estimations. This approach resulted in estimates within the same range: the estimated median DEHP intake in the whole population (both non-consumers and consumers of the foods considered) was 10.2, the arithmetic mean 14.0 and the 95th percentile 28.6μg/kg of BW per day. The respective estimates for consumers only were 12.4, 18.7 and 36.5μg/kg of BW per day. These results demonstrate that the probabilistic approach is able to estimate broader ranges of exposure even when using data representing an average intake. Moreover, it reflects the uncertainties of the estimation due to insufficient analytical data on concentrations of DEHP in food.

Regulatory toxicology in the twenty-first century: challenges, perspectives and possible solutions

Abstract The advent of new testing systems and “omics”-technologies has left regulatory toxicology facing one of the biggest challenges for decades. That is the question whether and how these methods can be used for regulatory purposes. The new methods undoubtedly enable regulators to address important open questions of toxicology such as species-specific toxicity, mixture toxicity, low-dose effects, endocrine effects or nanotoxicology, while promising faster and more efficient toxicity testing with the use of less animals. Consequently, the respective assays, methods and testing strategies are subject of several research programs worldwide. On the other hand, the practical application of such tests for regulatory purposes is a matter of ongoing debate. This document summarizes key aspects of this debate in the light of the European “regulatory status quo”, while elucidating new perspectives for regulatory toxicity testing.

Prevalence estimation under heterogeneity in the example of bovine trypanosomosis in Uganda

We examine variance estimators of a binomial parameter established under cluster sampling using data from a cross-sectional study of bovine trypanosomosis in Mukono County, Uganda. Fifty farms (referred to as clusters), were sampled with a total sample size of 487 cattle. Trypanosomes were found in 17.9% (87/487) of the total sample. The cluster-level (CL) prevalences were not homogeneously distributed. According to maximum-likelihood parameters established by mixture-distribution analysis, 18% of the cluster had 0% prevalence whereas 48% and 34% of the clusters could be allocated to subpopulations of clusters with mean prevalences 11.6% and 31.9% respectively. We show that this form of heterogeneity invalidates the applicability of the Beta distribution as a model for the distribution of CL prevalences. Furthermore, we provide empirical evidence for a variance inflation due to heterogeneity (inflation factor 2.07) that exceeds the design-based variance inflation due to clustering alone (inflation factor 1.82). The variance inflation due to heterogeneity is given in a closed form so that the approach can be conveniently applied to survey data that involve cluster sampling under heterogeneity.