José Antonio Roldán Nofuentes

Comparing two binary diagnostic tests in the presence of verification bias

The comparison of the accuracy of two binary diagnostic tests has traditionally required knowledge of the real state of the disease in all of the patients in the sample via the application of a gold standard. In practice, the gold standard is not always applied to all patients, which gives rise to the problem of partial verification of the disease. In this study, two methods of comparison of the efficiency of two binary diagnostic tests in the presence of verification bias are proposed. The first method consists of a comparison of the risk of error of two diagnostic tests, and the second a comparison of the kappa coefficients of the risk of error. The maximum likelihood estimators of risks and kappa coefficients are obtained. The tests of hypotheses to compare the risks and the kappa coefficients of two binary diagnostic tests when both are applied to the same random sample in the presence of verification bias are deduced, and simulation experiments are performed in order to investigate the asymptotic behaviour of each test of hypothesis. The results obtained have been applied to a study of coronary stenosis.

Comparison of weighted kappa coefficients of multiple binary diagnostic tests done on the same subjects.

Sensitivity and specificity are classic parameters to assess and compare the performance of binary diagnostic tests versus a gold standard in a population. Another useful parameter to assess and compare the performance of binary tests is the weighted kappa coefficient, which is defined as a measure of the beyond-chance agreement between the diagnostic test and the gold standard. In this study, we deduce the maximum likelihood estimators of the weighted kappa coefficients of multiple binary tests and we propose an asymptotic method to compare the weighted kappa coefficients of multiple binary tests with regard to the same gold standard when all of the diagnostic tests are applied to the same sample of patients. We have carried out simulation experiments to study the type I error and the power of the method that we propose when we compared three binary tests. We have applied the results obtained to the diagnosis of coronary disease.

Global hypothesis test to simultaneously compare the predictive values of two binary diagnostic tests

The positive and negative predictive values of a binary diagnostic test are measures of the clinical accuracy of the diagnostic test, which depend on the sensitivity and specificity of the diagnostic test and the disease prevalence, and therefore they are two interdependent parameters. The comparisons of predictive values in paired designs do not consider the dependence between predictive values. A global hypothesis test has been studied in order to simultaneously compare the predictive values of two or more binary diagnostic tests when the binary tests and the gold standard are applied to all of the individuals in a random sample. This global hypothesis test is an asymptotic hypothesis test based on the chi-square distribution. Simulation experiments have been carried out in order to study the type I error and the power of the global hypothesis test when comparing the predictive values of two and three binary diagnostic tests, respectively. From the results of the simulation experiments, a method has been proposed to simultaneously compare the predictive values of two or more binary diagnostic tests. The results have been applied to the diagnosis of coronary disease.

Average kappa coefficient: a new measure to assess a binary test considering the losses associated with an erroneous classification

The weighted kappa coefficient of a binary diagnostic test (BDT) is a measure of performance of a BDT, and is a function of the sensitivity and the specificity of the diagnostic test, of the disease prevalence and the weighting index. Weighting index represents the relative loss between the false positives and the false negatives. In this study, we propose a new measure of performance of a BDT: the average kappa coefficient. This parameter is the average function of the weighted kappa coefficients and does not depend on the weighting index. We have studied three asymptotic confidence intervals (CIs) for the average kappa coefficient, Wald, logit and bias-corrected bootstrap, and we carried out some simulation experiments to study the asymptotic coverage of each of the three CIs. We have written a program in R, called ‘akcbdt’, to estimate the average kappa coefficient of a BDT. This program is available as supplementary material. The results were applied to two examples.