Didier Meuwly

Measuring coherence of computer-assisted likelihood ratio methods

Measuring the performance of forensic evaluation methods that compute likelihood ratios (LRs) is relevant for both the development and the validation of such methods. A framework of performance characteristics categorized as primary and secondary is introduced in this study to help achieve such development and validation. Ground-truth labelled fingerprint data is used to assess the performance of an example likelihood ratio method in terms of those performance characteristics. Discrimination, calibration, and especially the coherence of this LR method are assessed as a function of the quantity and quality of the trace fingerprint specimen. Assessment of the coherence revealed a weakness of the comparison algorithm in the computer-assisted likelihood ratio method used.

Introducing a Semi‐Automatic Method to Simulate Large Numbers of Forensic Fingermarks for Research on Fingerprint Identification

Abstract:  Statistical research on fingerprint identification and the testing of automated fingerprint identification system (AFIS) performances require large numbers of forensic fingermarks. These fingermarks are rarely available. This study presents a semi‐automatic method to create simulated fingermarks in large quantities that model minutiae features or images of forensic fingermarks. This method takes into account several aspects contributing to the variability of forensic fingermarks such as the number of minutiae, the finger region, and the elastic deformation of the skin. To investigate the applicability of the simulated fingermarks, fingermarks have been simulated with 5–12 minutiae originating from different finger regions for six fingers. An AFIS matching algorithm was used to obtain similarity scores for comparisons between the minutiae configurations of fingerprints and the minutiae configurations of simulated and forensic fingermarks. The results showed similar scores for both types of fingermarks suggesting that the simulated fingermarks are good substitutes for forensic fingermarks.

Performance Study of a Score-based Likelihood Ratio System for Forensic Fingermark Comparison

In this article, the performance of a score‐based likelihood ratio (LR) system for comparisons of fingerprints with fingermarks is studied. The system is based on an automated fingerprint identification system (AFIS) comparison algorithm and focuses on fingerprint comparisons where the fingermarks contain 6–11 minutiae. The hypotheses under consideration are evaluated at the level of the person, not the finger. The LRs are presented with bootstrap intervals indicating the sampling uncertainty involved. Several aspects of the performance are measured: leave‐one‐out cross‐validation is applied, and rates of misleading evidence are studied in two ways. A simulation study is performed to study the coverage of the bootstrap intervals. The results indicate that the evidential strength for same source comparisons that do not meet the Dutch twelve‐point standard may be substantial. The methods used can be generalized to measure the performance of score‐based LR systems in other fields of forensic science.

Effect of calibration data on forensic likelihood ratio from a face recognition system

A biometric system used for forensic evaluation requires a conversion of the score to a likelihood ratio. A likelihood ratio can be computed as the ratio of the probability of a score given the prosecution hypothesis is true and the probability of a score given the defense hypothesis is true. In this paper we study two different approaches of a forensic likelihood ratio computation in the context of forensic face recognition. These approaches differ in the databases they use to obtain the score distribution under the prosecution and the defense hypothesis and therefore consider slightly different interpretation of these hypotheses. The goal of this study is to quantify the effect of these approaches on the resultant likelihood ratio in the context of evidence evaluation from a face recognition system. A state-of-the art commercial face recognition system is employed for facial images comparison and computation of scores. A simple forensic case is simulated by randomly selecting a small subset from the FRGC database. Images in this subset are used to estimate the score distribution under the prosecution and the defense hypothesis and the effect of different approaches of a likelihood ratio computation is demonstrated and explained. It is observed that there is a significant variation in the resultant likelihood ratios given the databases which are used to model the prosecution and defense hypothesis are varied.

Biometric evidence evaluation: an empirical assessment of the effect of different training data

For an automatic comparison of a pair of biometric specimens, a similarity metric called ‘score’ is computed by the employed biometric recognition system. In forensic evaluation, it is desirable to convert this score into a likelihood ratio. This process is referred to as calibration. A likelihood ratio is the probability of the score given the prosecution hypothesis (which states that the pair of biometric specimens are originated from the suspect) is true divided by the probability of the score given the defence hypothesis (which states that the pair of biometric specimens are not originated from the suspect) is true. In practice, a set of scores (called training scores) obtained from the within-source and between-sources comparison is needed to compute a likelihood ratio value for a score. In likelihood ratio computation, the within-source and between-sources conditions can be anchored to a specific suspect in a forensic case or it can be generic within-source and between-sources comparisons independent of the suspect involved in the case. This results in two likelihood ratio values which differ in the nature of training scores they use and therefore consider slightly different interpretations of the two hypotheses. The goal of this study is to quantify the differences in these two likelihood ratio values in the context of evidence evaluation from a face, a fingerprint and a speaker recognition system. For each biometric modality, a simple forensic case is simulated by randomly selecting a small subset of biometric specimens from a large database. In order to be able to carry out a comparison across the three biometric modalities, the same protocol is followed for training scores set generation. It is observed that there is a significant variation in the two likelihood ratio values.

From Biometric Scores to Forensic Likelihood Ratios

In this chapter, we describe the issue of the interpretation of forensic evidence from scores computed by a biometric system. This is one of the most important topics into the so-called area of forensic biometrics. We will show the importance of the topic, introducing some of the key concepts of forensic science with respect to the interpretation of results prior to their presentation in court, which is increasingly addressed by the computation of likelihood ratios (LR). We will describe the LR methodology, and will illustrate it with an example of the evaluation of fingerprint evidence in forensic conditions, by means of a fingerprint biometric system.

Likelihood ratio data to report the validation of a forensic fingerprint evaluation method

Data to which the authors refer to throughout this article are likelihood ratios (LR) computed from the comparison of 5–12 minutiae fingermarks with fingerprints. These LRs data are used for the validation of a likelihood ratio (LR) method in forensic evidence evaluation. These data present a necessary asset for conducting validation experiments when validating LR methods used in forensic evidence evaluation and set up validation reports. These data can be also used as a baseline for comparing the fingermark evidence in the same minutiae configuration as presented in (D. Meuwly, D. Ramos, R. Haraksim,) [1], although the reader should keep in mind that different feature extraction algorithms and different AFIS systems used may produce different LRs values. Moreover, these data may serve as a reproducibility exercise, in order to train the generation of validation reports of forensic methods, according to [1]. Alongside the data, a justification and motivation for the use of methods is given. These methods calculate LRs from the fingerprint/mark data and are subject to a validation procedure. The choice of using real forensic fingerprint in the validation and simulated data in the development is described and justified. Validation criteria are set for the purpose of validation of the LR methods, which are used to calculate the LR values from the data and the validation report. For privacy and data protection reasons, the original fingerprint/mark images cannot be shared. But these images do not constitute the core data for the validation, contrarily to the LRs that are shared.

Forensic use of fingermarks and fingerprints

The aim of this entry is to describe and explain the main forensic uses of fingermarks and fingerprints. It defines the concepts and provides the nomenclature related to forensic dactyloscopy. It describes the structure of the papillary ridges, the organization of the information in three levels, and its use for the fingerprint classification and individualization processes. It focuses on the variability and the distinctiveness of the marks and the prints and the exploitation of these properties in the forensic context. It emphasizes the difference between the properties of the mark and the prints in relation with the individualization process. It describes the current practice for fingermark evidence evaluation and analyzes the limits of forensic evaluation based on deterministic conclusions. It discusses the admissibility of the fingerprint evidence and provides casework examples involving misidentifications. It introduces the results of statistical research based on empirical data, statistical modeling, and an evaluation framework aiming at the description of the strength of evidence. Finally, it puts in perspective the current practice and the results of research and addresses the question of future developments in the field.

Validation of likelihood ratio methods for forensic evidence evaluation handling multimodal score distributions

This study presents a method for computing likelihood ratios (LRs) from multimodal score distributions, as the ones produced by some commercial off-the-shelf automated fingerprint identification systems (AFISs). The AFIS algorithms used to compare fingermarks and fingerprints were primarily developed for forensic investigation rather than for forensic evaluation purposes. Thus, in some of those algorithms, the computation of discriminating scores is speed-optimised. In the case of the AFIS algorithm used in this work, the speed-optimisation is achieved by performing the comparison in three different stages, each of which outputs scores of different magnitudes. As a consequence, all scores together present a multimodal distribution, even though each fingermark-to-fingerprint comparison generates one single score. This multimodal distribution of scores might be typical for other biometric systems or other algorithms, and the method proposed in this work can be also applied to those cases. As a result, the authors propose a probabilistic model for LR computation that presents more robustness to overfitting and data sparsity than other traditional approaches, like the use of models based on kernel density functions.

ForenFace: a unique annotated forensic facial image dataset and toolset

Few facial image datasets are suitable for forensic research. In this study, the authors present ForenFace, a facial image and video dataset. It contains video sequences and extracted images of 97 subjects recorded with six different surveillance camera of various types. Moreover, it also contains high-resolution images and 3D scans. The novelty of this dataset lies in two aspects: (i) a subset of 435 images (87 subjects, five images per subject) has been manually annotated, yielding a very rich forensically relevant annotation of almost 19.000 facial parts, and (ii) making available a toolset to create, view, and extract the annotation. The authors present protocols and the result of a baseline experiment in which two commercial software packages and an annotated facial feature contained in this dataset are compared. The dataset, the annotation and tools are available under a usage license.