Matteo Pardo

Learning from data: a tutorial with emphasis on modern pattern recognition methods

The purposes of this tutorial are twofold. First, it reviews the classical statistical learning scenario by highlighting its fundamental taxonomies and its key aspects. The second aim of the paper is to introduce some modern (ensembles) methods developed inside the machine learning field. The tutorial starts by putting the topic of supervised learning into the broader context of data analysis and by reviewing the classical pattern recognition methods: those based on class-conditional density estimation and the use of the Bayes theorem and those based on discriminant functions. The fundamental topic of complexity control is treated in some detail. Ensembles techniques have drawn considerable attention in recent years: a set of learning machines increases classification accuracy with respect to a single machine. Here, we introduce boosting, in which classifiers adaptively concentrate on the harder examples located near to the classification boundary and output coding, where a set of independent two-class machines solves a multiclass problem. The first successful applications of these methods to data produced by the Pico-2 electronic nose (EN), developed at the University of Brescia, Brescia, Italy, are also briefly shown.

Coffee analysis with an electronic nose

We present the Pico-1 electronic nose based on thin-film semiconductor sensors and an application to the analysis of two groups of seven coffees each. Cups of coffee were also analyzed by two panels of trained judges who assessed quantitative descriptors and a global index (called Hedonic Index, HI) characterizing the sensorial appeal of the coffee. Two tasks are performed by Pico-1. First, for each group, we performed the classification of the seven different coffee types using principal component analysis and multilayer perceptrons for the data analysis. Classification rates were above 90%. Secondly, the panel test descriptors were predicted starting from the measurements performed with Pico-1. The standard deviations for the prediction of the HI are comparable to the uncertainty of the HI itself (0.2 on a 1 to 9 scale for one group of coffees).

Metal Oxide Nanowire and Thin-Film-Based Gas Sensors for Chemical Warfare Simulants Detection

This work concerns with metal oxide (MOX) gas sensors based on nanowires and thin films. We focus on chemical warfare agents (CWAs) detection to compare these materials from the functional point-of-view. We work with different chemicals including simulants for Sarin nerve agents, vescicant gases, cyanide agents, and analytes such as ethanol, acetone, ammonia, and carbon monoxide that can be produced by everyday activities causing false alarms. Explorative data analysis has been used to demonstrate the different sensing performances of nanowires and thin films. Within the chosen application, our analysis reveal that the introduction of nanowires inside the array composed by thin films can improve its sensing capability. Cyanide simulants have been detected at concentrations close to 1 ppm, lower than the Immediately Dangerous for Life and Health (IDLH) value of the respective warfare agent. Higher sensitivity has been obtained to simulants for Sarin and vescicant gases, which have been detected at concentrations close or even lower than 100 ppb. Results demonstrate the suitability of the proposed array to selectively detect CWA simulants with respect to some compounds produced by everyday activities.

Remarks on the use of multilayer perceptrons for the analysis of chemical sensor array data

Multilayer perceptrons (MLPs) are a standard tool for establishing relationships between data in many real world problems, in the absence of a parametric model. In the last decade, they have often been used for analyzing data produced by arrays of chemical sensors [electronic noses (e-noses)]. Still, the central issue of controlling the complexity of an MLP for optimal generalization is frequently overlooked by chemical sensors practitioners causing incorrect or suboptimal results (over or underfitting). In this paper, we will: 1) present different ways of controlling the complexity of an MLP (model order selection, early stopping, and regularization); 2) shortly review the literature on complexity control, inside and outside the e-nose community; and 3) give examples of effective complexity control for two e-noses datasets of different size and learning difficulty. It will be shown that, if early stopping or regularization are adopted, overfitting is avoided whatever the number of hidden units (and, hence, network weights). Another issue tackled in this paper is the influence on the generalization error of the number of principal components over which data are projected (before being fed into the MLP). Simulations show that (test set) performance depends strongly on the number of principal components and that even components with less than 1% of the global variance enhance classification.

A time delay neural network for estimation of gas concentrations in a mixture

Abstract The problem of quantifying the concentrations of CO and NO 2 present in a mixture starting from the electrical response of a sensors array is addressed. A comparison between a traditional approach based on the steady state conductance and one using a time delay neural network is drawn.

A stability based validity method for fuzzy clustering

An important goal in cluster analysis is the internal validation of results using an objective criterion. Of particular relevance in this respect is the estimation of the optimum number of clusters capturing the intrinsic structure of your data. This paper proposes a method to determine this optimum number based on the evaluation of fuzzy partition stability under bootstrap resampling. The method is first characterized on synthetic data with respect to hyper-parameters, like the fuzzifier, and spatial clustering parameters, such as feature space dimensionality, clusters degree of overlap, and number of clusters. The method is then validated on experimental datasets. Furthermore, the performance of the proposed method is compared to that obtained using a number of traditional fuzzy validity rules based on the cluster compactness-to-separation criteria. The proposed method provides accurate and reliable results, and offers better generalization capabilities than the classical approaches.

A hierarchical classification scheme for an Electronic Nose

Abstract A procedure for the classification of data from an Electronic Nose (EN) is proposed, which is beneficial in the case in which the number of classes is big and/or the classes are not nicely clustered (for instance, as seen in a PCA score plot). The procedure consists of separating the original classification problem in successive, less demanding sub-classification tasks. The advantages, which are due to the greater flexibility, include the following: smaller processing times, enhanced performances and better interpretation of the results. Each classification step uses PCA and Multilayer Perceptrons (MLP) in cascade and, for comparison, Simca. The method has been tested on a data set formed by 242 measurements of 14 olive oil types performed with a commercial EN that was equipped with 12 MOS sensors.

Monitoring reliability of sensors in an array by neural networks

Abstract The correlation between the responses of five semiconductor thin films sensors to CO–NO 2 mixtures is exploited to detect a possible malfunctioning of one of the sensors during operation. To this end, at every time instant, the current flowing in each single sensor is estimated as a function of the current flowing in the remaining ones. With multiple linear regression, we obtain, in the case of the worst sensor, a regression coefficient of 0.89. The estimation is then accomplished using the regression ability of five artificial neural networks (ANN), one for each sensor, obtaining at worst a mean estimation error on the test set of 6×10 −3 μA 2 , the signal being of the order of the microampere (μA). In the case of a simulated transient malfunctioning, we show how it is possible to detect on-line which is the sensor that is not working properly. Further, after a fault has been detected, the estimation replaces the damaged sensor response. In this way, the concentration prediction — performed by other ANNs that need the responses of all the sensors — can proceed until the damaged sensor has been replaced.

Decompositive classification models for electronic noses

Abstract The classification of 242 measurements in 14 classes is attempted using two different classification approaches. Measurements have been performed with a commercial electronic nose (EN) comprising 11 chemical sensors on extra-virgin olive oils of 14 different geographical provenances. As we deal with a relatively small data set and a big number of classes, the classification task is quite demanding. We first tackled the global classification task using a single multilayer perceptron (MLP), which gave a misclassification rate of 25%. In order to improve the performance, we studied two different approaches based on ensembles of learning machines, which decompose the classification in subtasks. In the first approach, a classification tree was constructed using a priori knowledge (geographical origin) for the formation of sensible superclasses (union of single classes). At each classification node we both used MLPs and SIMCA (soft independent modeling of class analogy). The second approach applies a learning machine called parallel nonlinear dichotomizers (PND) that is based on the decomposition of a K-class classification problem in a set of two-class tasks. A binary codeword is assigned to each class and each bit is learned by a dichotomizer (implemented by a dedicated MLP). In the reconstruction stage, a pattern is assigned to the class whose codeword is most similar (e.g. in L1 norm) to the output of the set of dichotomizers. We achieved the best results (misclassification error rate of about 10%) using a decomposition based on error correcting output codes (ECOC).

Electronic nose for coffee quality control

Two groups of seven coffees have been analyzed with the Pico-1 Electronic Nose (EN) developed at Brescia University. Inside each group, coffees have been classified with PCA and multilayer perceptrons giving classification rates above 90%. Cups of coffee were analyzed by two panels of trained judges who assessed quantitative descriptors and a global index (Hedonic Index, HI) characterizing the sensorial appeal of the coffee. These parameters were predicted starting from the measurements performed with Pico-1. The standard deviation for the prediction of the HI are comparable to the uncertainty of the HI itself (0.2 on a 1 to 9 scale).