Francesco Tortorella

Reliability Parameters to Improve Combination Strategies in Multi-Expert Systems

Abstract: Recognition systems based on a combination of different experts have been widely investigated in the recent past. General criteria for improving the performance of such systems are based on estimating the reliability associated with the decision of each expert, so as to suitably weight its response in the combination phase. According to the methods proposed to-date, when the expert assigns a sample to a class, the reliability of such a decision is estimated on the basis of the recognition rate obtained by the expert on the chosen class during the training phase. As a consequence, the same reliability value is associated with every decision attributing a sample to a same class, even though it seems reasonable to take into account its dependence on the quality of the specific sample. We propose a method for estimating the reliability of each single recognition act of an expert on the basis of information directly derived from its output. In this way, the reliability value of a decision is more properly estimated, thus allowing a more precise weighting during the combination phase. The definition of the reliability parameters for widely used classification paradigms is discussed, together with the combining rules employing them for weighting the expert opinions. The results obtained by combining four experts in order to recognise handwritten numerals from a standard character database are presented. Comparison with classical combining rules is also reported, and the advantages of the proposed approach outlined.

Maximizing the area under the ROC curve by pairwise feature combination

The majority of the available classification systems focus on the minimization of the classification error rate. This is not always a suitable metric specially when dealing with two-class problems with skewed classes and cost distributions. In this case, an effective criterion to measure the quality of a decision rule is the area under the Receiver Operating Characteristic curve (AUC) that is also useful to measure the ranking quality of a classifier as required in many real applications. In this paper we propose a nonparametric linear classifier based on the maximization of AUC. The approach lies on the analysis of the Wilcoxon-Mann-Whitney statistic of each single feature and on an iterative pairwise coupling of the features for the optimization of the ranking of the combined feature. By the pairwise feature evaluation the proposed procedure is essentially different from other classifiers using AUC as a criterion. Experiments performed on synthetic and real data sets and comparisons with previous approaches confirm the effectiveness of the proposed method.

Automatic classification of clustered microcalcifications by a multiple expert system

Abstract Mammography is a not invasive diagnostic technique widely used for early cancer detection in women breast. A significant visual clue of the disease is the presence of clusters of microcalcifications. The automatic recognition of malignant clusters of microcalcifications, which could be very helpful for diagnostic purposes, is a very difficult task because of the small size of the microcalcifications and of the poor quality of the mammographic images. In this paper we propose a novel approach for classifying clusters of microcalcifications, based on a Multiple Expert System; such system aggregates several experts, some of which are devoted to classify the single microcalcifications while others are aimed to classify the cluster considered as a whole. The final output results from the suitable combination of the two groups of experts. The tests performed on a standard database of 40 mammographic images have confirmed the effectiveness of the approach.

Learning from unbalanced data: A cascade-based approach for detecting clustered microcalcifications

Finding abnormalities in diagnostic images is a difficult task even for expert radiologists because the normal tissue locations largely outnumber those with suspicious signs which may thus be missed or incorrectly interpreted. For the same reason the design of a Computer-Aided Detection (CADe) system is very complex because the large predominance of normal samples in the training data may hamper the ability of the classifier to recognize the abnormalities on the images. In this paper we present a novel approach for computer-aided detection which faces the class imbalance with a cascade of boosting classifiers where each node is trained by a learning algorithm based on ranking instead of classification error. Such approach is used to design a system (CasCADe) for the automated detection of clustered microcalcifications (μCs), which is a severely unbalanced classification problem because of the vast majority of image locations where no μC is present. The proposed approach was evaluated with a dataset of 1599 full-field digital mammograms from 560 cases and compared favorably with the Hologic R2CAD ImageChecker, one of the most widespread commercial CADe systems. In particular, at the same lesion sensitivity of R2CAD (90%) on biopsy proven malignant cases, CasCADe and R2CAD detected 0.13 and 0.21 false positives per image (FPpi), respectively (p-value=0.09), whereas at the same FPpi of R2CAD (0.21), CasCADe and R2CAD detected 93% and 90% of true lesions respectively (p-value=0.11) thus showing that CasCADe can compete with high-end CADe commercial systems.

Multiclassification : reject criteria for the Bayesian combiner

Abstract In the present paper we propose a method for determining the best trade-off between error rate and reject rate for a multi-expert system (MES) using the Bayesian combining rule. The method is based on the estimation of the reliability of each classification act and on the evaluation of the convenience of rejecting the input sample when the reliability is under a threshold, evaluated on the basis of the requirements of the application domain. The adaptability to the given domain represents an original feature since, till now, the problem of defining a reject rule for an MES has not been systematically introduced, and the few existing proposals seldom take into account the requirements of the domain. The method has been widely tested with reference to the recognition of handwritten characters coming from a standard database. The results are also compared with those provided by employing the well-known Chow’s rule.

An Optimal Reject Rule for Binary Classifiers

Binary classifiers are used in many complex classification problems in which the classification result could have serious consequences. Thus, they should ensure a very high reliability to avoid erroneous decisions. Unfortunately, this is rarely the case in real situations where the cost for a wrong classification could be so high that it should be convenient to reject the sample which gives raise to an unreliable result. However, as far as we know, a reject option specifically devised for binary classifiers has not been yet proposed. This paper presents an optimal reject rule for binary classifiers, based on the Receiver Operating Characteristic curve. The rule is optimal since it maximizes a classification utility function, defined on the basis of classification and error costs peculiar for the application at hand. Experiments performed with a data set publicly available confirmed the effectiveness of the proposed reject rule.

A computer-aided detection system for clustered microcalcifications

OBJECTIVE The aim of this paper is to describe a novel system for computer-aided detection of clusters of microcalcifications on digital mammograms. METHODS AND MATERIAL Mammograms are first segmented by means of a tree-structured Markov random field algorithm that extracts the elementary homogeneous regions of interest. An analysis of such regions is then performed by means of a two-stage, coarse-to-fine classification based on both heuristic rules and classifier combination. In this phase, we avoid taking a decision on the single microcalcifications and forward it to the successive phase of clustering realized through a sequential approach. RESULTS The system has been tested on a publicly available database of mammograms and compared with previous approaches. The obtained results show that the system is very effective, especially in terms of sensitivity. CONCLUSIONS The proposed approach exhibits some remarkable advantages both in segmentation and classification phases. The segmentation phase employs an image model that reduces the computational burden, preserving the small details in the image through an adaptive local estimation of all model parameters. The classification stage combines the results of the classifiers focused on the single microcalcification and the cluster as a whole. Such an approach makes a detection system particularly effective and robust with respect to the large variations exhibited by the clusters of microcalcifications.

Graph matching: a fast algorithm and its evaluation

A graph matching algorithm is illustrated and its performance compared with that of a well known algorithm performing the same task. According to the proposed algorithm the matching process is carried out by using a state space representation: a state represents a partial solution of the matching between two graphs, and a transition between states corresponds to the addition of a new pair of matched nodes. A set of feasibility rules is introduced for pruning states corresponding to partial matching solutions not satisfying the required graph morphism. Results outlining the computational cost reduction achieved by the method are given with reference to a set of randomly generated graphs.

Facing Imbalanced Classes through Aggregation of Classifiers

Two class classification problems in real world are often characterized by imbalanced classes. This is a serious issue since a classifier trained on such a data distribution typically exhibits a prediction accuracy highly skewed towards the majority class. To improve the quality of the classifier, many approaches have been proposed till now for building artificially balanced training sets. Such methods are mainly based on undersampling the majority class and/or oversampling the minority class. However, both approaches can produce overfitting or underfitting problems for the trained classifier. In this paper we present a method for building a multiple classifier system in which each constituting classifier is trained on a subset of the majority class and on the whole minority class. The approach has been tested on the detection of microcalcifications on digital mammograms. The results obtained confirm the effectiveness of the method.

Semi-Supervised Learning Techniques in Artificial Olfaction: A Novel Approach to Classification Problems and Drift Counteraction

Semi-supervised learning is a promising research area aiming to develop pattern recognition tools capable to exploit simultaneously the benefits from supervised and unsupervised learning techniques. These can lead to a very efficient usage of the limited number of supervised samples achievable in many artificial olfaction problems like distributed air quality monitoring. We believe it can also be beneficial in addressing another source of limited knowledge we have to face when dealing with real world problems: concept and sensor drifts. In this paper we describe the results of two artificial olfaction investigations that show semi-supervised learning techniques capabilities to boost performance of state-of-the art classifiers and regressors. The use of semi-supervised learning approach resulted in the effective reduction of drift-induced performance degradation in long-term on-field continuous operation of chemical multisensory devices.