Francesco Masulli

Ensembles of Learning Machines

Ensembles of learning machines constitute one of the main current directions in machine learning research, and have been applied to a wide range of real problems. Despite of the absence of an unified theory on ensembles, there are many theoretical reasons for combining multiple learners, and an empirical evidence of the effectiveness of this approach. In this paper we present a brief overview of ensemble methods, explaining the main reasons why they are able to outperform any single classifier within the ensemble, and proposing a taxonomy based on the main ways base classifiers can be generated or combined together.

A fuzzy clustering based segmentation system as support to diagnosis in medical imaging.

In medical imaging uncertainty is widely present in data, because of the noise in acquisition and of the partial volume effects originating from the low resolution of sensors. In particular, borders between tissues are not exactly defined and memberships in the boundary regions are intrinsically fuzzy. Therefore, computer assisted unsupervised fuzzy clustering methods turn out to be particularly suitable for handling a decision making process concerning segmentation of multimodal medical images. By using the possibilistic c-means algorithm as a refinement of a neural network based clustering algorithm named capture effect neural network, we developed the possibilistic neuro fuzzy c-means algorithm (PNFCM). In this paper the PNFCM has been applied to two different multimodal data sets and the results have been compared to those obtained by using the classical fuzzy c-means algorithm. Furthermore, a discussion is presented about the role of fuzzy clustering as a support to diagnosis in medical imaging.

Fuzzy Logic and Applications

Cluster analysis of high-dimensional data has become of special interest in recent years. The term high-dimensional data can refer to a larger number of attributes – 20 or more – as they often occur in database tables. But high-dimensional data can also mean that we have to deal with thousands of attributes as in the context of genomics or proteomics data where thousands of genes or proteins are measured and are considered in some analysis tasks as attributes. A main reason, why cluster analysis of high-dimensional data is different from clustering low-dimensional data, is the concentration of norm phenomenon, which states more or less that the relative differences between distances between randomly distributed points tend to be more and more similar in higher dimensions. On the one hand, fuzzy cluster analysis has been shown to be less sensitive to initialisation than, for instance, the classical k-means algorithm. On the other, standard fuzzy clustering is stronger affected by the concentration of norm phenomenon and tends to fail easily in high dimensions. Here we present a review of why fuzzy clustering has special problems with high-dimensional data and how this can be amended by modifying the fuzzifier concept. We also describe a recently introduced approach based on correlation and an attribute selection fuzzy clustering technique that can be applied when clusters can only be found in lower dimensions.

Artificial Neural Networks and Machine Learning – ICANN 2012

A complex-valued multilayer perceptron (MLP) can approximate a periodic or unbounded function, which cannot be easily realized by a real-valued MLP. Its search space is full of crevasse-like forms having huge condition numbers; thus, it is very hard for existing methods to perform efficient search in such a space. The space also includes the structure of reducibility mapping. The paper proposes a new search method for a complex-valued MLP, which employs both eigen vector descent and reducibility mapping, aiming to stably find excellent solutions in such a space. Our experiments showed the proposed method worked well.

BFGS Optimization for Faster and Automated Supervised Learning

Standard back-propagation learning (BP) is known to have slow convergence properties. Furthermore no general prescription is given for selecting the appropriate learning rate, so success is dependent on a trial and error process. In this work a well known optimization technique (the Broyden-Fletcher-Goldfarb-Shanno memoryless quasi-Newton method) is employed to speed up convergence and to select parameters. The strict locality requirement is relaxed but parallelism of computation is maintained, allowing efficient use of concurrent computation. While requiring only limited changes to BP, this method yields a speed-up factor of 100 – 500 for the medium-size networks considered.

Soft transition from probabilistic to possibilistic fuzzy clustering

In the fuzzy clustering literature, two main types of membership are usually considered: A relative type, termed probabilistic, and an absolute or possibilistic type, indicating the strength of the attribution to any cluster independent from the rest. There are works addressing the unification of the two schemes. Here, we focus on providing a model for the transition from one schema to the other, to exploit the dual information given by the two schemes, and to add flexibility for the interpretation of results. We apply an uncertainty model based on interval values to memberships in the clustering framework, obtaining a framework that we term graded possibility. We outline a basic example of graded possibilistic clustering algorithm and add some practical remarks about its implementation. The experimental demonstrations presented highlight the different properties attainable through appropriate implementation of a suitable graded possibilistic model. An interesting application is found in automated segmentation of diagnostic medical images, where the model provides an interactive visualization tool for this task

Effectiveness of Error Correcting Output Codes in Multiclass Learning Problems

In the framework of decomposition methods for multiclass classification problems, error correcting output codes (ECOC) can be fruitfully used as codewords for coding classes in order to enhance the generalization capability of learning machines. The effectiveness of error correcting output codes depends mainly on the independence of codeword bits and on the accuracy by which each dichotomy is learned. Separated and non-linear dichotomizers can improve the independence among computed codeword bits, thus fully exploiting the error recovering capabilities of ECOC. In the experimentation presented in this paper we compare ECOC decomposition methods implemented through monolithic multi-layer perceptrons and sets of linear and non-linear independent dichotomizers. The most effectiveness of ECOC decomposition scheme is obtained by Parallel Non-linear Dichotomizers (PND), a learning machine based on decomposition of polychotomies into dichotomies, using non linear independent dichotomizers.

Comparing decomposition methods for classification

Decomposition methods for multiclass classification problems constitute a powerful framework to improve generalization capabilities of a large set of learning machines, including support vector machines and multi-layer perceptrons. We present a review of the main decomposition approach to classification and an experimental comparison of One-Per-Class (OPC), Correcting Classifiers (CC) and Error Correcting Output Codes (ECOC) decomposition methods implemented using multi-layer perceptrons as dichotomizers. The results show that CC and ECOC outperform OPC over the considered data sets.

A time delay neural network for estimation of gas concentrations in a mixture

Abstract The problem of quantifying the concentrations of CO and NO 2 present in a mixture starting from the electrical response of a sensors array is addressed. A comparison between a traditional approach based on the steady state conductance and one using a time delay neural network is drawn.

Soft Computing Applications

Learning Fuzzy Classifiers with Evolutionary Algorithms.- Evidence of Chaotic Attractors in Cortical Fast Oscillations Tested by an Artificial Neural Network.- A Possibilistic Framework for Asset Allocation.- Efficient Low-resolution Character Recognition Using Sub-machine-code Genetic Programming.- Accurate Modeling and NN to Reproduce Human Like Trajectories.- A New ANFIS Synthesis Approach for Time Series Forecasting.- Experiments on a Prey Predators System.- Qualitative Models and Fuzzy Systems: An Integrated Approach to System Identification.- Fuzzy Reliability Analysis of Concrete Structures by Using a Genetically Powered Simulation.- System for Remote Diagnosis Based on Fuzzy Inference.- Image Segmentation Using a Genetic Algorithm.- How Fuzzy Logic Can Help Detecting Buried Land Mines.- Neuro-fuzzy Filtering Techniques for the Analysis of Complex Acoustic Scenarios.- P300 Off-line Detection: A Fuzzy-based Support System.- Evolutionary Approaches for Cluster Analysis.- Neural Networks in Missing Data Analysis, Model Identification and Non Linear Control.- Genetic Optimization of Fuzzy Sliding Mode Controllers: An Experimental Study.- Sailboat Dynamics Neural Network Identification and Control.- Fuzzy Logic in Spacecraft Design.