Likelihood ratio distributions and the (ir)relevance of error rates.

Abstract

Over the last years, several papers have been published that presented likelihood ratio distributions in kinship cases. These data are useful to assess the power of the discussed technology for certain types of kinship investigation, since they tell us what range of likelihood ratios we can expect given the ground truth of the relationship between the investigated individuals. However, in some publications the fraction of (in)correctly classified pairs (when based on a likelihood ratio threshold), are presented as accuracy or error rate, with the interpretational pitfall looming that these can be seen as probabilities that are generally applicable to the investigated type of kinship, on the investigated loci and with the obtained allele frequencies. In this publication we warn against such interpretations. We point out that from the likelihood ratio, probabilistic statements about the ground truth cannot be made, and that therefore this will also not be possible from a weaker statement such as the LR exceeding some threshold value. The statement that the LR exceeds a threshold in itself does has evidential value, and we will explain how to estimate that value from the obtained empirical distributions. We also explain that the concept of error does not apply to the likelihood ratio, but only to decisions. If one takes decisions based on a LR threshold, then it is possible to define error rates, but these are predictive and conditional. They tell us how often we will make wrong classifications in each group (the related pairs and the unrelated pairs) if we apply a LR threshold. They do not tell us how likely it is, once we have made a decision, that this decision is the one that we wanted to make had we known the true relationship. In order to make such a statement we need to have more information. We illustrate the points we address with examples that we have taken from the literature.