Henk A. L. Kiers

Towards a standardized notation and terminology in multiway analysis

This paper presents a standardized notation and terminology to be used for three‐ and multiway analyses, especially when these involve (variants of) the CANDECOMP/PARAFAC model and the Tucker model. The notation also deals with basic aspects such as symbols for different kinds of products, and terminology for three‐ and higher‐way data. The choices for terminology and symbols to be used have to some extent been based on earlier (informal) conventions. Simplicity and reduction of the possibility of confusion have also played a role in the choices made. Copyright

PARAFAC2 - Part I. A direct fitting algorithm for the PARAFAC2 model

PARAFAC is a generalization of principal component analysis (PCA) to the situation where a set of data matrices is to be analysed. If each data matrix has the same row and column units, the resulting data are three‐way data and can be modelled by the PARAFAC1 model. If each data matrix has the same column units but different (numbers of) row units, the PARAFAC2 model can be used. Like the PARAFAC1 model, the PARAFAC2 model gives unique solutions under certain mild assumptions, whereas it is less severely constrained than PARAFAC1. It may therefore also be used for regular three‐way data in situations where the PARAFAC1 model is too restricted. Usually the PARAFAC2 model is fitted to a set of matrices with cross‐products between the column units. However, this model‐fitting procedure is computationally complex and inefficient. In the present paper a procedure for fitting the PARAFAC2 model directly to the set of data matrices is proposed. It is shown that this algorithm is more efficient than the indirect fitting algorithm. Moreover, it is more easily adjusted so as to allow for constraints on the parameter matrices, to handle missing data, as well as to handle generalizations to sets of three‐ and higher‐way data. Furthermore, with the direct fitting approach we also gain information on the row units, in the form of ‘factor scores’. As will be shown, this elaboration of the model in no way limits the feasibility of the method. Even though full information on the row units becomes available, the algorithm is based on the usually much smaller cross‐product matrices only. Copyright

PARAFAC2—PART II. MODELING CHROMATOGRAPHIC DATA WITH RETENTION TIME SHIFTS

This paper offers an approach for handling retention time shifts in resolving chromatographic data using the PARAFAC2 model. In Part I of this series an algorithm for PARAFAC2 was developed and extended to N‐way arrays. It was discussed that the PARAFAC2 model has a number of attractive features. It is unique under mild conditions though it puts fewer restrictions on the data than the well‐known PARAFAC1 model. This has important implications for the modeling of chromatographic data in which retention time shifts can be regarded as a violation of the assumption of parallel proportional profiles underlying the PARAFAC1 model. The PARAFAC2 model does not assume parallel proportional elution profiles, but only that the matrix of elution profiles preserve its ‘inner‐product structure’ from sample to sample. This means that the cross‐products of the matrix holding the elution profiles in its columns remain constant. Here an application using chromatographic separation based on the molecular size of thick juice samples from the beet sugar industry illustrates the benefit of using the PARAFAC2 model. Copyright

Cross-validation of component models: A critical look at current methods

In regression, cross-validation is an effective and popular approach that is used to decide, for example, the number of underlying features, and to estimate the average prediction error. The basic principle of cross-validation is to leave out part of the data, build a model, and then predict the left-out samples. While such an approach can also be envisioned for component models such as principal component analysis (PCA), most current implementations do not comply with the essential requirement that the predictions should be independent of the entity being predicted. Further, these methods have not been properly reviewed in the literature. In this paper, we review the most commonly used generic PCA cross-validation schemes and assess how well they work in various scenarios.

Three-mode principal components analysis: Choosing the numbers of components and sensitivity to local optima

A method that indicates the numbers of components to use in fitting the three-mode principal components analysis (3MPCA) model is proposed. This method, called DIFFIT, aims to find an optimal balance between the fit of solutions for the 3MPCA model and the numbers of components. The achievement of DIFFIT is compared with that of two other methods, both based on two-way PCAs, by means of a simulation study. It was found that DIFFIT performed considerably better than the other methods in indicating the numbers of components. The 3MPCA model can be estimated by the TUCKALS3 algorithm, which is an alternating least squares algorithm. In a study of how sensitive TUCKALS3 is at hitting local optima, it was found that, if the numbers of components are specified correctly, TUCKALS3 never hits a local optimum. The occurrence of local optima increased as the difference between the numbers of underlying components and the numbers of components as estimated by TUCKALS3 increased. Rationally initiated TUCKALS3 runs hit local optima less often than randomly initiated runs.

Selecting among three-mode principal component models of different types and complexities: a numerical convex hull based method.

Several three-mode principal component models can be considered for the modelling of three-way, three-mode data, including the Candecomp/Parafac, Tucker3, Tucker2, and Tucker1 models. The following question then may be raised: given a specific data set, which of these models should be selected, and at what complexity (i.e. with how many components)? We address this question by proposing a numerical model selection heuristic based on a convex hull. Simulation results show that this heuristic performs almost perfectly, except for Tucker3 data arrays with at least one small mode and a relatively large amount of error.

The Hull Method for Selecting the Number of Common Factors

A common problem in exploratory factor analysis is how many factors need to be extracted from a particular data set. We propose a new method for selecting the number of major common factors: the Hull method, which aims to find a model with an optimal balance between model fit and number of parameters. We examine the performance of the method in an extensive simulation study in which the simulated data are based on major and minor factors. The study compares the method with four other methods such as parallel analysis and the minimum average partial test, which were selected because they have been proven to perform well and/or they are frequently used in applied research. The Hull method outperformed all four methods at recovering the correct number of major factors. Its usefulness was further illustrated by its assessment of the dimensionality of the Five-Factor Personality Inventory (Hendriks, Hofstee, & De Raad, 1999). This inventory has 100 items, and the typical methods for assessing dimensionality prove to be useless: the large number of factors they suggest has no theoretical justification. The Hull method, however, suggested retaining the number of factors that the theoretical background to the inventory actually proposes.

Hierarchical relations among three-way methods

A number of methods for the analysis of three-way data are described and shown to be variants of principal components analysis (PCA) of the two-way supermatrix in which each two-way slice is “strung out” into a column vector. The methods are shown to form a hierarchy such that each method is a constrained variant of its predecessor. A strategy is suggested to determine which of the methods yields the most useful description of a given three-way data set.

Weighted Least Squares Fitting Using Ordinary Least Squares Algorithms.

A general approach for fitting a model to a data matrix by weighted least squares (WLS) is studied. This approach consists of iteratively performing (steps of) existing algorithms for ordinary least squares (OLS) fitting of the same model. The approach is based on minimizing a function that majorizes the WLS loss function. The generality of the approach implies that, for every model for which an OLS fitting algorithm is available, the present approach yields a WLS fitting algorithm. In the special case where the WLS weight matrix is binary, the approach reduces to missing data imputation.

Factorial k-means analysis for two-way data

A discrete clustering model together with a continuous factorial one are fitted simultaneously to two-way data, with the aim of identifying the best partition of the objects, described by the best orthogonal linear combinations of the variables (factors) according to the least-squares criterion. This methodology named for its features factorialk-means analysis has a very wide range of applications since it fulfills a double objective: data reduction and synthesis, simultaneously in the direction of objects and variables; variable selection in cluster analysis, identifying variables that most contribute to determine the classification of the objects. The least-squares fitting problem proposed here is mathematically formalized as a quadratic constrained minimization problem with mixed variables. An iterative alternating least-squares algorithm based on two main steps is proposed to solve the quadratic constrained problem. Starting from the cluster centroids, the subspace projection is found that leads to the smallest distances between object points and centroids. Updating the centroids, the partition is detected assigning objects to the closest centroids. At each step the algorithm decreases the least-squares criterion, thus converging to an optimal solution. Two data sets are analyzed to show the features of the factorial k-means model. The proposed technique has a fast algorithm that allows researchers to use it also with large data sets.