Patrick J. F. Groenen

Nonlinear Principal Components Analysis: Introduction and Application.

The authors provide a didactic treatment of nonlinear (categorical) principal components analysis (PCA). This method is the nonlinear equivalent of standard PCA and reduces the observed variables to a number of uncorrelated principal components. The most important advantages of nonlinear over linear PCA are that it incorporates nominal and ordinal variables and that it can handle and discover nonlinear relationships between variables. Also, nonlinear PCA can deal with variables at their appropriate measurement level; for example, it can treat Likert-type scales ordinally instead of numerically. Every observed value of a variable can be referred to as a category. While performing PCA, nonlinear PCA converts every category to a numeric value, in accordance with the variables analysis level, using optimal quantification. The authors discuss how optimal quantification is carried out, what analysis levels are, which decisions have to be made when applying nonlinear PCA, and how the results can be interpreted. The strengths and limitations of the method are discussed. An example applying nonlinear PCA to empirical data using the program CATPCA (J. J. Meulman, W. J. Heiser, & SPSS, 2004) is provided.

Data analysis, classification, and related methods

The volume presents new developments in data analysis and classification, and gives a state of the art impression of these scientific fields at the turn of the Millenium. Areas that receive considerable attention in this book are Cluster Analysis, Data Mining, Multidimensional and Symbolic Data Analysis, Decision and Regression Trees. The volume contains a refereed selection of original papers, overview papers, and innovative applications presented at the 7th Conference of the International Federation of Classification Societies (IFCS-2000), with contributions from eminent scientists all over the world. The reader finds introductory material into various areas and kaleidoscopic views of recent technical and methodological developments in widely different areas within data analysis and classification. The presence of a large number of application papers demonstrates the usefulness of the recently developed techniques.

Cluster differences scaling with a within-clusters loss component and a fuzzy successive approximation strategy to avoid local minima

Cluster differences scaling is a method for partitioning a set of objects into classes and simultaneously finding a low-dimensional spatial representation ofK cluster points, to model a given square table of dissimilarities amongn stimuli or objects. The least squares loss function of cluster differences scaling, originally defined only on the residuals of pairs of objects that are allocated to different clusters, is extended with a loss component for pairs that are allocated to the same cluster. It is shown that this extension makes the method equivalent to multidimensional scaling with cluster constraints on the coordinates. A decomposition of the sum of squared dissimilarities into contributions from several sources of variation is described, including the appropriate degrees of freedom for each source. After developing a convergent algorithm for fitting the cluster differences model, it is argued that the individual objects and the cluster locations can be jointly displayed in a configuration obtained as a by-product of the optimization. Finally, the paper introduces a fuzzy version of the loss function, which can be used in a successive approximation strategy for avoiding local minima. A simulation study demonstrates that this strategy significantly outperforms two other well-known initialization strategies, and that it has a success rate of 92 out of 100 in attaining the global minimum.

Fuzzy clustering with squared Minkowski distances

Abstract This paper presents a new fuzzy clustering model based on a root of the squared Minkowski distance which includes squared and unsquared Euclidean distances and the L 1 -distance. An algorithm is presented that is based on iterative majorization and yields a convergent series of monotone nonincreasing loss function values. This algorithm coincides under some condition with the ISODATA algorithm of Dunn (J. Cybernet. 3 (1974) 32–57) and the fuzzy c -means algorithm of Bezdek (Ph.D. Thesis, Cornell University, Ithaca, 1973) for squared Euclidean distance and with an algorithm of Jajuga (Fuzzy Sets and Systems 39 (1991) 43–50) for L 1 -distances. To find a global minimum we compare a special strategy called fuzzy steps with fuzzy Kohonen clustering networks (FKCN) (Pattern Recognition 27 (1994) 757–764) and multistart. Fuzzy steps and FKCN are based on finding updates for a decreasing weighting exponent, which seems to work particularly well for hard clustering. To assess the performance of the methods, two numerical experiments and a simulation study are performed.

The Tunneling Method for Global Optimization in Multidimensional Scaling.

This paper focuses on the problem of local minima of the STRESS function. It turns out that unidimensional scaling is particularly prone to local minima, whereas full dimensional scaling with Euclidean distances has a local minimum that is global. For intermediate dimensionality with Euclidean distances it depends on the dissimilarities how severe the local minimum problem is. For city-block distances in any dimensionality many different local minima are found. A simulation experiment is presented that indicates under what conditions local minima can be expected. We introduce the tunneling method for global minimization, and adjust it for multidimensional scaling with general Minkowski distances. The tunneling method alternates a local search step, in which a local minimum is sought, with a tunneling step in which a different configuration is sought with the same STRESS as the previous local minimum. In this manner successively better local minima are obtained, and experimentation so far shows that the last one is often a global minimum.

The majorization approach to multidimensional scaling for Minkowski distances

The majorization method for multidimensional scaling with Kruskals STRESS has been limited to Euclidean distances only. Here we extend the majorization algorithm to deal with Minkowski distances with 1≤p≤2 and suggest an algorithm that is partially based on majorization forp outside this range. We give some convergence proofs and extend the zero distance theorem of De Leeuw (1984) to Minkowski distances withp>1.

Identifying response styles: A latent-class bilinear multinomial logit model

Respondents can vary strongly in the way they use rating scales. Specifically, respondents can exhibit a variety of response styles, which threatens the validity of the responses. The purpose of this article is to investigate how response style and content of the items affect rating scale responses. The authors develop a novel model that accounts for different types of response styles, content of items, and background characteristics of respondents. By imposing a bilinear parameter structure on a multinomial logit model, the authors graphically distinguish the effects on the response behavior of the characteristics of a respondent and the content of an item. The authors combine this approach with finite mixture modeling, yielding two segmentations of the respondents: one for response style and one for item content. They apply this latent-class bilinear multinomial logit model to the well-known List of Values in a cross-national context. The results show large differences in the opinions and the response styles of respondents and reveal previously unknown response styles. Some response styles appear to be valid communication styles, whereas other response styles often concur with inconsistent opinions of the items and seem to be response bias.

Visualizing time-varying correlations across stock markets

We propose a graphical method to visualize possible time-varying correlations between stock market returns. The method can be useful for observing stable or emerging clusters of stock markets with similar behavior. The graphs, which originate from applying multidimensional scaling techniques (MDS), may also guide the construction of multivariate econometric models. We illustrate our method for the returns and absolute returns of 13 important stock markets.

Stability of Nonlinear Principal Components Analysis: An Empirical Study Using the Balanced Bootstrap

Principal components analysis (PCA) is used to explore the structure of data sets containing linearly related numeric variables. Alternatively, nonlinear PCA can handle possibly nonlinearly related numeric as well as nonnumeric variables. For linear PCA, the stability of its solution can be established under the assumption of multivariate normality. For nonlinear PCA, however, standard options for establishing stability are not provided. The authors use the nonparametric bootstrap procedure to assess the stability of nonlinear PCA results, applied to empirical data. They use confidence intervals for the variable transformations and confidence ellipses for the eigenvalues, the component loadings, and the person scores. They discuss the balanced version of the bootstrap, bias estimation, and Procrustes rotation. To provide a benchmark, the same bootstrap procedure is applied to linear PCA on the same data. On the basis of the results, the authors advise using at least 1,000 bootstrap samples, using Procrustes rotation on the bootstrap results, examining the bootstrap distributions along with the confidence regions, and merging categories with small marginal frequencies to reduce the variance of the bootstrap results.

I-Scal: Multidimensional scaling of interval dissimilarities

Multidimensional scaling aims at reconstructing dissimilarities between pairs of objects by distances in a low-dimensional space. However, in some cases the dissimilarity itself is unknown, but the range of the dissimilarity is given. Such fuzzy data give rise to a data matrix in which each dissimilarity is an interval of values. These interval dissimilarities are modelled by the ranges of the distances defined as the minimum and maximum distance between two rectangles representing the objects. Previously, two approaches for such data have been proposed and one of them is investigated. A new algorithm called I-Scal is developed. Because I-Scal is based on iterative majorization it has the advantage that each iteration is guaranteed to improve the solution until no improvement is possible. In addition, a rational start configuration is proposed that is helpful in locating a good quality local minima. In a simulation study, the quality of this algorithm is investigated and I-Scal is compared with one previously proposed algorithm. Finally, I-Scal is applied on an empirical example of dissimilarity intervals of sounds.