Rolf Ergon

Reduced PCR/PLSR models by subspace projections

Abstract Latent variables models used in principal component regression (PCR) or partial least squares regression (PLSR) often use a high number of components, and this makes interpretation of score and loading plots difficult. These plots are essential parts of multivariate modeling, and there is therefore a need for a reduction of the number of components without loss of prediction power. In this work, it is shown that such reductions of PCR models with a common number of components for all responses, as well as of PLSR (PLS1 and PLS2) models, may be obtained by projection of the X modeling objects onto a subspace containing the estimators bˆ i for the different responses y i . The theoretical results are substantiated in three real world data set examples, also showing that the presented model reduction method may work quite well also for PCR models with different numbers of components for different responses, as well as for a set of individual PLSR (PLS1) models. Examples of interpretational advantages of reduced models in process monitoring applications are included.

Constrained numerical optimization of PCR/PLSR predictors

Abstract Assuming a fully known latent variables (LV) model, the optimal multivariate calibration predictor is found from Kalman filtering theory. From this follows the best possible column space for a loading weight matrix W opt. in a predictor based on the latent variables, and thus the optimal factorization of the regressor matrix X . Although the optimal predictor cannot be directly determined in a practical case, we may still make an attempt to find it. The paper presents a simple algorithm for a constrained numerical search for a W opt. matrix spanning the optimal column space, using a principal component analysis (PCR) or a partial least squares (PLS) factorization as a starting point. The constraint is necessary in order to avoid overfitting, and it is based on an assumption of a smooth predictor. A simulation example and data from a metal ion mixture experiment are used to demonstrate the feasibility of the proposed method.

On multivariate calibration with unlabeled data

In principal component regression (PCR) and partial least‐squares regression (PLSR), the use of unlabeled data, in addition to labeled data, helps stabilize the latent subspaces in the calibration step, typically leading to a lower prediction error. For using unlabeled data in PLSR, a non‐sequential approach based on optimal filtering (OF) has been proposed in the literature. In this work, a sequential version of the OF‐based PLSR and a PCA‐based PLSR (PLSR applied to PCA‐preprocessed data) are proposed. It is shown analytically that the sequential version of the OF‐based PLSR is equivalent to that of PCA‐based PLSR, which leads to a new interpretation of OF. Simulated and experimental data sets are used to point out the usefulness and pitfalls of using unlabeled data. Unlabeled data can replace labeled data to some extent, thereby leading to an economic benefit. However, in the presence of drift, the use of unlabeled data can result in an increase in prediction error compared to that obtained with a model based on labeled data alone. Copyright

Dynamic system multivariate calibration with low-sampling-rate y data

When the data in principal component regression (PCR) or partial least squares regression (PLSR) form time series, it may be possible to improve the prediction/estimation results by utilizing the correlation between neighboring observations. The estimators may then be identified from experimental data using system identification methods. This is possible also in cases where the response variables in the experimental data are sampled at a low and possibly irregular rate, while the regressor variables are sampled at a higher rate. After a discussion of the options available, the paper shows how the autocorrelation of the regressor variables in such multirate sampling cases may be utilized by identification of parsimonious output error (OE) estimators. An example using acoustic power spectrum regressor data is finally presented. Copyright

Model Choice and Squared Prediction Errors in PLS Regression

Squared prediction errors (SPE) in

When three traits make a line: Evolution of phenotypic plasticity and genetic assimilation through linear reaction norms in stochastic environments

{\bf X}

Comparison of simple projection methods (OPLS, PLSO, and TP) for separation of predicting and non‐predicting information in PLSR and PCR, with focus on DA

are discussed in relation to the conventional PLSR versus bidiagonalization model and algorithm issue concerning residual and prediction consistency, with focus on process monitoring and fault detection. Our analysis leads to the conclusion that conventional PLSR based on the NIPALS algorithm is ambiguous in SPE values caused by process faults. The basic reason for this is that the sample residuals are not found as projections onto the orthogonal complement of the space where the scores and regression solution are located, and where also the statistical

The environmental zero‐point problem in evolutionary reaction norm modeling

{\it T}^{\rm 2}

PLS post-processing by similarity transformation (PLS+ST): a simple alternative to OPLS

limit is defined. The alternative non‐orthogonalized PLSR and bidiagonalization (Bidiag2) algorithms, as well as a simple re‐formulation of the NIPALS algorithm (RE‐PLSR), give unambiguous SPE values, and the last two of these also retain orthogonal score vectors. While prediction results from all of these methods in theory are identical, our conclusion is that methods where the

PLS score-loading correspondence and a bi-orthogonal factorization

{\it T}^{\rm 2}