Isabella Morlini

Neural Network Modeling for Small Datasets

Neural network modeling for small datasets can be justified from a theoretical point of view according to some of Bartletts results showing that the generalization performance of a multilayer perceptron (MLP) depends more on the L1 norm ‖c‖1 of the weights between the hidden layer and the output layer rather than on the total number of weights. In this article we investigate some geometrical properties of MLPs and drawing on linear projection theory, we propose an equivalent number of degrees of freedom to be used in neural model selection criteria like the Akaike information criterion and the Bayes information criterion and in the unbiased estimation of the error variance. This measure proves to be much smaller than the total number of parameters of the network usually adopted, and it does not depend on the number of input variables. Moreover, this concept is compatible with Bartletts results and with similar ideas long associated with projection-based models and kernel models. Some numerical studies involving both real and simulated datasets are presented and discussed.

On Multicollinearity and Concurvity in Some Nonlinear Multivariate Models

Recent developments of multivariate smoothing methods provide a rich collection of feasible models for nonparametric multivariate data analysis. Among the most interpretable are those with smoothed additive terms. Construction of various methods and algorithms for computing the models have been the main concern in literature in this area. Less results are available on the validation of computed fit, instead, and many applications of nonparametric methods end up in computing and comparing the generalized validation error or related indexes. This article reviews the behaviour of some of the best known multivariate nonparametric methods, based on subset selection and on projection, when (exact) collinearity or multicollinearity (near collinearity) is present in the input matrix. It shows the possible aliasing effects in computed fits of some selection methods and explores the properties of the projection spaces reached by projection methods in order to help data analysts to select the best model in case of ill conditioned input matrices. Two simulation studies and a real data set application are presented to illustrate further the effects of collinearity or multicollinearity in the fit.

Artificial neural network estimation of rainfall intensity from radar observations

Volumetric scans of radar reflectivity Z and gage measurements of rainfall intensity R are used to explore the capabilities of three artificial neural networks to identify and reproduce the functional relationship between Z and R. The three networks are a multilayer perceptron, a Bayesian network, and a radial basis function network. For each of them, numerical experiments are conducted incorporating in the network inputs different descriptions of the space-time variability of Z. Space variability refers to the observations of Z along the vertical atmospheric profile, at 11 constant altitude plan position indicator levels, namely ZT = (Z1,…,Z11). Time variability refers to the observations of Z at the time intervals prior to that for which the estimate of R is provided. Space variability is evaluated by performing a principal component analysis over standardized values of Z, namely Z˜, and the first two principal components of Z˜ (which describe 91% of the original variance) are used to synthesize the elements of Z into fewer orthogonal inputs for the networks. Network predictions significantly improve when the models are trained with the two principal components of Z˜ with respect to the case in which only Z1 is used. Increasing the time horizon further improves the performances of the Bayesian network but is found to worsen the performances of the other two networks.

A latent variables approach for clustering mixed binary and continuous variables within a Gaussian mixture model

For clustering objects, we often collect not only continuous variables, but binary attributes as well. This paper proposes a model-based clustering approach with mixed binary and continuous variables where each binary attribute is generated by a latent continuous variable that is dichotomized with a suitable threshold value, and where the scores of the latent variables are estimated from the binary data. In economics, such variables are called utility functions and the assumption is that the binary attributes (the presence or the absence of a public service or utility) are determined by low and high values of these functions. In genetics, the latent response is interpreted as the ‘liability’ to develop a qualitative trait or phenotype. The estimated scores of the latent variables, together with the observed continuous ones, allow to use a multivariate Gaussian mixture model for clustering, instead of using a mixture of discrete and continuous distributions. After describing the method, this paper presents the results of both simulated and real-case data and compares the performances of the multivariate Gaussian mixture model and of a mixture of joint multivariate and multinomial distributions. Results show that the former model outperforms the mixture model for variables with different scales, both in terms of classification error rate and reproduction of the clusters means.

Fuzzy Composite Indicators: An Application for Measuring Customer Satisfaction

Composite indicators should ideally measure multidimensional concepts which cannot be captured by a single variable. In this chapter, we suggest a method based on fuzzy set theory for the construction of a fuzzy synthetic index of a latent phenomenon (e.g., well-being, quality of life, etc.), using a set of manifest variables measured on different scales (quantitative, ordinal and binary). A few criteria for assigning values to the membership function are discussed, as well as criteria for defining the weights of the variables. For ordinal variables, we propose a fuzzy quantification method based on the sampling cumulative function and a weighting system which takes into account the relative frequency of each category. An application regarding the results of a survey on the users of a contact center is presented.

A New Class of Weighted Similarity Indices Using Polytomous Variables

We introduce new similarity measures between two subjects, with reference to variables with multiple categories. In contrast to traditionally used similarity indices, they also take into account the frequency of the categories of each attribute in the sample. This feature is useful when dealing with rare categories, since it makes sense to differently evaluate the pairwise presence of a rare category from the pairwise presence of a widespread one. A weighting criterion for each category derived from Shannon’s information theory is suggested. There are two versions of the weighted index: one for independent categorical variables and one for dependent variables. The suitability of the proposed indices is shown in this paper using both simulated and real world data sets.

Dissimilarity and similarity measures for comparing dendrograms and their applications

In this paper we propose a new index Z for measuring the dissimilarity between two hierarchical clusterings (or dendrograms). This index is a metric since it satisfies the axioms of non-negativity, symmetry and triangle inequality. A desirable property of this index is that it can be decomposed into the contributions pertaining to each stage of the hierarchies. We show the relations of such components with the currently used criteria for comparing two partitions. We obtain a global similarity index as the complement to one of the suggested dissimilarity and we derive its adjustment for agreement due to chance. We obtain similarity indexes pertaining to each stage of the hierarchies as the complement to one of the additive parts of the global distance Z. We consider the use of the proposed distance for more than two dendrograms and its use for the consensus of classifications and variable selection in cluster analysis. A series of simulation experiments and an application to a real data set are presented.

Equivalent Number of Degrees of Freedom for Neural Networks

The notion of equivalent number of degrees of freedom (e.d.f.) to be used in neural network modeling from small datasets has been introduced in Ingrassia and [Morlini (2005)]. It is much smaller than the total number of parameters and it does not depend on the number of input variables. We generalize our previous results and discuss the use of the e.d.f. in the general framework of multivariate nonparametric model selection. Through numerical simulations, we also investigate the behavior of model selection criteria like AIC, GCV and BIC/SBC, when the e.d.f. is used instead of the total number of the adaptive parameters in the model.

On the Dynamic Time Warping for Computing the Dissimilarity Between Curves

Dynamic time warping (DTW) is a technique for aligning curves that considers two aspects of variations: horizontal and vertical, or domain and range. This alignment is an essential preliminary in many applications before classification or functional data analysis. A problem with DTW is that the algorithm may fail to find the natural alignment of two series since it is mostly influenced by salient features rather than by the overall shape of the sequences. In this paper, we first deepen the DTW algorithm, showing relationships and differences with the curve registration technique, and then we propose a modification of the algorithm that considers a smoothed version of the data.

Radial basis function networks with partially classified data

The problem of estimating a classification rule with partially classified observations, which often occurs in biological and ecological modelling, and which is of major interest in pattern recognition, is discussed. Radial basis function networks for classification problems are presented and compared with the discriminant analysis with partially classified data, in situations where some observations in the training set are unclassified. An application on a set of morphometric data obtained from the skulls of 288 specimens of Microtus subterraneus and Microtus multiplex is performed. This example illustrates how the use of both classified and unclassified observations in the estimate of the hidden layer parameters has the potential to greatly improve the network performances.