Grzegorz Zadora

A Two‐Level Model for Evidence Evaluation

ABSTRACT: A random effects model using two levels of hierarchical nesting has been applied to the calculation of a likelihood ratio as a solution to the problem of comparison between two sets of replicated multivariate continuous observations where it is unknown whether the sets of measurements shared a common origin. Replicate measurements from a population of such measurements allow the calculation of both within‐group and between‐group variances/covariances. The within‐group distribution has been modelled assuming a Normal distribution, and the between‐group distribution has been modelled using a kernel density estimation procedure. A graphical method of estimating the dependency structure among the variables has been used to reduce this highly multivariate problem to several problems of lower dimension. The approach was tested using a database comprising measurements of eight major elements from each of four fragments from each of 200 glass objects and found to perform well compared with previous approaches, achieving a 15.2% false‐positive rate, and a 5.5% false‐negative rate. The modelling was then applied to two examples of casework in which glass found at the scene of the criminal activity has been compared with that found in association with a suspect.

Information-theoretical assessment of the performance of likelihood ratio computation methods

Performance of likelihood ratio (LR) methods for evidence evaluation has been represented in the past using, for example, Tippett plots. We propose empirical cross‐entropy (ECE) plots as a metric of accuracy based on the statistical theory of proper scoring rules, interpretable as information given by the evidence according to information theory, which quantify calibration of LR values. We present results with a case example using a glass database from real casework, comparing performance with both Tippett and ECE plots. We conclude that ECE plots allow clearer comparisons of LR methods than previous metrics, allowing a theoretical criterion to determine whether a given method should be used for evidence evaluation or not, which is an improvement over Tippett plots. A set of recommendations for the use of the proposed methodology by practitioners is also given.

SEM–EDX—a useful tool for forensic examinations

Abstract There are two main aims of forensic examination of the physical evidences. The first aim is comparison of the evidence with the reference material (called discrimination). The task is to find out whether they could have come from the same object. The second aim, when there is no comparative material available, is a classification of the evidence sample into a group of objects taking into account its specific chemical and physical properties. Scanning electron microscopy with energy dispersive X-ray spectrometry (SEM–EDX) is a powerful tool for forensic scientists to classify and discriminate evidence material because they can simultaneously examine the morphology and the elemental composition of objects. Moreover, the obtained results could be enhanced using some methods of chemometric analysis. A few examples of problems related to the classification and discrimination of selected types of microtraces are presented.

Classification of Glass Fragments Based on Elemental Composition and Refractive Index

Abstract:  The aim of this study was to assess the efficiency of likelihood ratio (LR)‐based measures when they are applied to solving various classification problems for glass objects which are described by elemental composition, and refractive index (RI) values, and compare LR‐based methods to other classification methods such as support vector machines (SVM) and naïve Bayes classifiers (NBC). One hundred and fifty‐three glass objects (23 building windows, 25 bulbs, 32 car windows, 57 containers, and 16 headlamps) were analyzed by scanning electron microscopy coupled with an energy dispersive X‐ray spectrometer. Refractive indices for building and car windows were measured before (RIb), and after (RIa) an annealing process. The proposed scheme for glass fragment(s) classification demonstrates some efficiency, although the classification of car windows (c) and building windows (w) must be treated carefully. This is because of their very similar elemental content. However, a combination of elemental content and information on the change in RI during annealing (ΔRI = RIa−RIb) gave very promising results. A LR model for the classification of glass fragments into use‐type categories for forensic purposes gives slightly higher misclassification rates than SVM and NBC. However, the observed differences between results obtained by all three approaches were very similar, especially when applied to the car window and building window classification problem. Therefore, the LR model can be recommended because of the ease of interpretation of LR‐based measures of certainty.

Likelihood ratio model for classification of forensic evidence

One of the problems of analysis of forensic evidence such as glass fragments, is the determination of their use-type category, e.g. does a glass fragment originate from an unknown window or container? Very small glass fragments arise during various accidents and criminal offences, and could be carried on the clothes, shoes and hair of participants. It is therefore necessary to obtain information on their physicochemical composition in order to solve the classification problem. Scanning Electron Microscopy coupled with an Energy Dispersive X-ray Spectrometer and the Glass Refractive Index Measurement method are routinely used in many forensic institutes for the investigation of glass. A natural form of glass evidence evaluation for forensic purposes is the likelihood ratio--LR=p(E|H(1))/p(E|H(2)). The main aim of this paper was to study the performance of LR models for glass object classification which considered one or two sources of data variability, i.e. between-glass-object variability and(or) within-glass-object variability. Within the proposed model a multivariate kernel density approach was adopted for modelling the between-object distribution and a multivariate normal distribution was adopted for modelling within-object distributions. Moreover, a graphical method of estimating the dependence structure was employed to reduce the highly multivariate problem to several lower-dimensional problems. The performed analysis showed that the best likelihood model was the one which allows to include information about between and within-object variability, and with variables derived from elemental compositions measured by SEM-EDX, and refractive values determined before (RI(b)) and after (RI(a)) the annealing process, in the form of dRI=log(10)|RI(a)-RI(b)|. This model gave better results than the model with only between-object variability considered. In addition, when dRI and variables derived from elemental compositions were used, this model outperformed two other classification methods in classifying test set observations into car or building windows.

Information-theoretical feature selection using data obtained by Scanning Electron Microscopy coupled with and Energy Dispersive X-ray spectrometer for the classification of glass traces

In this work, a selection of the best features for multivariate forensic glass classification using Scanning Electron Microscopy coupled with an Energy Dispersive X-ray spectrometer (SEM-EDX) has been performed. This has been motivated by the fact that the databases available for forensic glass classification are sparse nowadays, and the acquisition of SEM-EDX data is both costly and time-consuming for forensic laboratories. The database used for this work consists of 278 glass objects for which 7 variables, based on their elemental compositions obtained with SEM-EDX, are available. Two categories are considered for the classification task, namely containers and car/building windows, both of them typical in forensic casework. A multivariate model is proposed for the computation of the likelihood ratios. The feature selection process is carried out by means of an exhaustive search, with an Empirical Cross-Entropy (ECE) objective function. The ECE metric takes into account not only the discriminating power of the model in use, but also its calibration, which indicates whether or not the likelihood ratios are interpretable in a probabilistic way. Thus, the proposed model is applied to all the 63 possible univariate, bivariate and trivariate combinations taken from the 7 variables in the database, and its performance is ranked by its ECE. Results show remarkable accuracy of the best variables selected following the proposed procedure for the task of classifying glass fragments into windows (from cars or buildings) or containers, obtaining high (almost perfect) discriminating power and good calibration. This allows the proposed models to be used in casework. We also present an in-depth analysis which reveals the benefits of the proposed ECE metric as an assessment tool for classification models based on likelihood ratios.

A Two-Level Model for Evidence Evaluation in the Presence of Zeros

Abstract:  Likelihood ratios (LRs) provide a natural way of computing the value of evidence under competing propositions. We propose LR models for classification and comparison that extend the ideas of Aitken, Zadora, and Lucy and Aitken and Lucy to include consideration of zeros. Instead of substituting zeros by a small value, we view the presence of zeros as informative and model it using Bernoulli distributions. The proposed models are used for evaluation of forensic glass (comparison and classification problem) and paint data (comparison problem). Two hundred and sixty‐four glass samples were analyzed by scanning electron microscopy, coupled with an energy dispersive X‐ray spectrometer method and 36 acrylic topcoat paint samples by pyrolysis gas chromatography hyphened with mass spectrometer method. The proposed LR model gave very satisfactory results for the glass comparison problem and for most of the classification tasks for glass. Results of comparison of paints were also highly satisfactory, with only 3.0% false positive answers and 2.8% false negative answers.

A likelihood ratio model for the determination of the geographical origin of olive oil

Food fraud or food adulteration may be of forensic interest for instance in the case of suspected deliberate mislabeling. On account of its potential health benefits and nutritional qualities, geographical origin determination of olive oil might be of special interest. The use of a likelihood ratio (LR) model has certain advantages in contrast to typical chemometric methods because the LR model takes into account the information about the sample rarity in a relevant population. Such properties are of particular interest to forensic scientists and therefore it has been the aim of this study to examine the issue of olive oil classification with the use of different LR models and their pertinence under selected data pre-processing methods (logarithm based data transformations) and feature selection technique. This was carried out on data describing 572 Italian olive oil samples characterised by the content of 8 fatty acids in the lipid fraction. Three classification problems related to three regions of Italy (South, North and Sardinia) have been considered with the use of LR models. The correct classification rate and empirical cross entropy were taken into account as a measure of performance of each model. The application of LR models in determining the geographical origin of olive oil has proven to be satisfactorily useful for the considered issues analysed in terms of many variants of data pre-processing since the rates of correct classifications were close to 100% and considerable reduction of information loss was observed. The work also presents a comparative study of the performance of the linear discriminant analysis in considered classification problems. An approach to the choice of the value of the smoothing parameter is highlighted for the kernel density estimation based LR models as well.

Evaluation of evidence value of glass fragments by likelihood ratio and Bayesian Network approaches

Growing interest in applications of Bayesian Networks (BNs) in forensic science raises the question whether BN could be used in forensic practice for the evaluation of glass objects described by the results of physico-chemical analysis, especially the information obtained from analysis performed by Glass Refractive Index Measurement technique. Comparison of glass fragments, i.e. could two glass samples (recovered from, e.g. the suspects clothes and control, collected from the scene of crime) have originated from the same object, is one of the tasks of evaluation of glass fragments for forensic purposes. The second problem is the determination of their use-type category, e.g. does an analysed glass fragment originate from an unknown window or container? This process, known as classification, is especially important when the analysed fragment was recovered from the suspects clothes and there was no control sample. 111 glass objects (car windows, building windows, and containers) were measured in order to determine the refractive index (RI) before (RI(b)) and after the annealing process (RI(a)), from which a new variable dRI=log(10)|RI(a)-RI(b)| was calculated. Results obtained by the application of BN models were compared to results obtained by the application of suitable likelihood ratio models commonly used in the forensic sphere nowadays. The performed research showed that BN models could be satisfactorily applied to obtain the evidence value of glass fragments when RI(b) is used in the comparison problem. Use of BN with dRI in the classification problem also gave good results.

Wine authenticity verification as a forensic problem: An application of likelihood ratio test to label verification

The aim of the study was to investigate the applicability of the likelihood ratio (LR) approach for verifying the authenticity of 178 samples of 3 Italian wine brands: Barolo, Barbera, and Grignolino described by 27 parameters describing their chemical compositions. Since the problem of products authenticity may be of forensic interest, the likelihood ratio approach, expressing the role of the forensic expert, was proposed for determining the true origin of wines. It allows us to analyse the evidence in the context of two hypotheses, that the object belongs to one or another wine brand. Various LR models were the subject of the research and their accuracy was evaluated by the Empirical cross entropy (ECE) approach. The rates of correct classifications for the proposed models were higher than 90% and their performance evaluated by ECE was satisfactory.