Marco Muselli

A clustering technique for the identification of piecewise affine systems

We propose a new technique for the identification of discrete-time hybrid systems in the piecewise affine (PWA) form. This problem can be formulated as the reconstruction of a possibly discontinuous PWA map with a multi-dimensional domain. In order to achieve our goal, we provide an algorithm that exploits the combined use of clustering, linear identification, and pattern recognition techniques. This allows to identify both the affine submodels and the polyhedral partition of the domain on which each submodel is valid avoiding gridding procedures. Moreover, the clustering step (used for classifying the datapoints) is performed in a suitably defined feature space which allows also to reconstruct different submodels that share the same coefficients but are defined on different regions. Measures of confidence on the samples are introduced and exploited in order to improve the performance of both the clustering and the final linear regression procedure.

A Clustering Technique for the Identification of Piecewise Affine Systems

We propose a new technique for the identification of discrete-time hybrid systems in the Piece-Wise Affine (PWA) form. The identification algorithm proposed in [10] is first considered and then improved under various aspects. Measures of confidence on the samples are introduced and exploited in order to improve the performance of both the clustering algorithm used for classifying the data and the final linear regression procedure. Moreover, clustering is performed in a suitably defined space that allows also to reconstruct different submodels that share the same coefficients but are defined on different regions.

Cancer recognition with bagged ensembles of Support Vector Machines

Abstract Expression-based classification of tumors requires stable, reliable and variance reduction methods, as DNA microarray data are characterized by low size, high dimensionality, noise and large biological variability. In order to address the variance and curse of dimensionality problems arising from this difficult task, we propose to apply bagged ensembles of support vector machines (SVM) and feature selection algorithms to the recognition of malignant tissues. Presented results show that bagged ensembles of SVMs are more reliable and achieve equal or better classification accuracy with respect to single SVMs, whereas feature selection methods can further enhance classification accuracy.

Binary rule generation via Hamming Clustering

The generation of a set of rules underlying a classification problem is performed by applying a new algorithm called Hamming Clustering (HC). It reconstructs the AND-OR expression associated with any Boolean function from a training set of samples. The basic kernel of the method is the generation of clusters of input patterns that belong to the same class and are close to each other according to the Hamming distance. Inputs which do not influence the final output are identified, thus automatically reducing the complexity of the final set of rules. The performance of HC has been evaluated through a variety of artificial and real-world benchmarks. In particular, its application in the diagnosis of breast cancer has led to the derivation of a reduced set of rules solving the associated classification problem.

Identification of piecewise affine and hybrid systems

We focus on the identification of discrete time hybrid systems in the piecewise affine (PWA) form. This problem can be formulated as the reconstruction of a possibly discontinuous PWA map with a multidimensional domain. In order to achieve our goal, we propose an algorithm that exploits the combined use of clustering, linear identification, and classification techniques. This allows one to identify both the affine sub-models and the polyhedral partition of the domain on which each submodel is valid.

Deterministic design for neural network learning: an approach based on discrepancy

The general problem of reconstructing an unknown function from a finite collection of samples is considered, in case the position of each input vector in the training set is not fixed beforehand but is part of the learning process. In particular, the consistency of the empirical risk minimization (ERM) principle is analyzed, when the points in the input space are generated by employing a purely deterministic algorithm (deterministic learning). When the output generation is not subject to noise, classical number-theoretic results, involving discrepancy and variation, enable the establishment of a sufficient condition for the consistency of the ERM principle. In addition, the adoption of low-discrepancy sequences enables the achievement of a learning rate of O(1/L), with L being the size of the training set. An extension to the noisy case is provided, which shows that the good properties of deterministic learning are preserved, if the level of noise at the output is not high. Simulation results confirm the validity of the proposed approach.

Training digital circuits with Hamming clustering

A new algorithm, called Hamming clustering (HC), for the solution of classification problems with binary inputs is proposed. It builds a logical network containing only AND, OR, and NOT ports which, in addition to satisfying all the input-output pairs included in a given finite consistent training set, is able to reconstruct the underlying Boolean function. The basic kernel of the method is the generation of clusters of input patterns that belong to the same class and are close to each other according to the Hamming distance. A pruning phase precedes the construction of the digital circuit so as to reduce its complexity or to improve its robustness. A theoretical evaluation of the execution time required by HC shows that the behavior of the computational cost is polynomial. This result is confirmed by extensive simulations on artificial and real-world benchmarks, which point out also the generalization ability of the logical networks trained by HC.

A New Learning Method for Piecewise Linear Regression

A new connectionist model for the solution of piecewise linear regression problems is introduced; it is able to reconstruct both continuous and non continuous real valued mappings starting from a finite set of possibly noisy samples. The approximating function can assume a different linear behavior in each region of an unknown polyhedral partition of the input domain.The proposed learning technique combines local estimation, clustering in weight space, multicategory classification and linear regression in order to achieve the desired result. Through this approach piecewise affine solutions for general nonlinear regression problems can also be found.

Efficient sampling in approximate dynamic programming algorithms

Abstract Dynamic Programming (DP) is known to be a standard optimization tool for solving Stochastic Optimal Control (SOC) problems, either over a finite or an infinite horizon of stages. Under very general assumptions, commonly employed numerical algorithms are based on approximations of the cost-to-go functions, by means of suitable parametric models built from a set of sampling points in the d-dimensional state space. Here the problem of sample complexity, i.e., how “fast” the number of points must grow with the input dimension in order to have an accurate estimate of the cost-to-go functions in typical DP approaches such as value iteration and policy iteration, is discussed. It is shown that a choice of the sampling based on low-discrepancy sequences, commonly used for efficient numerical integration, permits to achieve, under suitable hypotheses, an almost linear sample complexity, thus contributing to mitigate the curse of dimensionality of the approximate DP procedure.

Switching neural networks: a new connectionist model for classification

A new connectionist model, called Switching Neural Network (SNN), for the solution of classification problems is presented. SNN includes a first layer containing a particular kind of A/D converters, called latticizers, that suitably transform input vectors into binary strings. Then, the subsequent two layers of an SNN realize a positive Boolean function that solves in a lattice domain the original classification problem. Every function realized by an SNN can be written in terms of intelligible rules. Training can be performed by adopting a proper method for positive Boolean function reconstruction, called Shadow Clustering (SC). Simulation results obtained on the StatLog benchmark show the good quality of the SNNs trained with SC.