Christine M. Wehlburg

Concentration Residual Augmented Classical Least Squares (CRACLS): A Multivariate Calibration Method with Advantages over Partial Least Squares

A significant extension to the classical least-squares (CLS) algorithm called concentration residual augmented CLS (CRACLS) has been developed. Previously, unmodeled sources of spectral variation have rendered CLS models ineffective for most types of problems, but with the new CRACLS algorithm, CLS-type models can be applied to a significantly wider range of applications. This new quantitative multivariate spectral analysis algorithm iteratively augments the calibration matrix of reference concentrations with concentration residuals estimated during CLS prediction. Because these residuals represent linear combinations of the unmodeled spectrally active component concentrations, the effects of these components are removed from the calibration of the analytes of interest. This iterative process allows the development of a CLS-type calibration model comparable in prediction ability to implicit multivariate calibration methods such as partial least squares (PLS) even when unmodeled spectrally active components are present in the calibration sample spectra. In addition, CRACLS retains the improved qualitative spectral information of the CLS algorithm relative to PLS. More importantly, CRACLS provides a model compatible with the recently presented prediction-augmented CLS (PACLS) method. The CRACLS/PACLS combination generates an adaptable model that can achieve excellent prediction ability for samples of unknown composition that contain unmodeled sources of spectral variation. The CRACLS algorithm is demonstrated with both simulated and real data derived from a system of dilute aqueous solutions containing glucose, ethanol, and urea. The simulated data demonstrate the effectiveness of the new algorithm and help elucidate the principles behind the method. Using experimental data, we compare the prediction abilities of CRACLS and PLS during cross-validated calibration. In combination with PACLS, the CRACLS predictions are comparable to PLS for the prediction of the glucose, ethanol, and urea components for validation samples collected when significant instrument drift was present. However, the PLS predictions required recalibration using nonstandard cross-validated rotations while CRACLS/PACLS was rapidly updated during prediction without the need for time-consuming cross-validated recalibration. The CRACLS/PACLS algorithm provides a more general approach to removing the detrimental effects of unmodeled components.

New Hybrid Algorithm for Maintaining Multivariate Quantitative Calibrations of a Near-Infrared Spectrometer

Our newly developed prediction-augmented classical least-squares/partial least-squares (PACLS/PLS) hybrid algorithm can correct for the presence of unmodeled sources of spectral variation such as instrument drift by explicitly incorporating known or empirically derived information about the unmodeled spectral variation. We have tested the ability of the new hybrid algorithm to maintain a multivariate calibration in the presence of instrument drift using a near-infrared (NIR) spectrometer (7500–11 000 cm−1) to quantitate dilute aqueous solutions containing glucose, ethanol, and urea. The spectral variations required to update the multivariate models for both short- and long-term drift were obtained using a single representative midpoint sample whose spectrum was repeatedly measured during collection of calibration data and during collection of separate validation sample spectra on three subsequent days. The performance of the PACLS/PLS model for maintaining a calibration was compared to PLS with subset recalibration, a method that has previously been applied to maintenance and transfer of calibration. Without drift corrections, both PACLS/PLS and PLS had poor predictive ability on sample spectra collected on subsequent days. Unlike previous maintenance of calibration studies that corrected for long-term drift only, the PACLS/PLS and PLS models demonstrated the best predictive abilities when short-term drift was also corrected. The PACLS/PLS hybrid model outperformed PLS with subset recalibration for near real-time predictions when instrument drift was determined from the repeat samples closest in time to the measurement of the unknown. Near real-time standard errors of prediction (SEPs) for the hybrid model were comparable to the cross-validated SEPs obtained with the original calibration model.

New Hybrid Algorithm for Transferring Multivariate Quantitative Calibrations of Intra-vendor Near-Infrared Spectrometers

A new prediction-augmented classical least-squares/partial least-squares (PACLS/PLS) hybrid algorithm is ideally suited for use in transferring multivariate calibrations between spectrometers. Spectral variations such as instrument response differences can be explicitly incorporated into the algorithm through the use of subset sample spectra collected on both spectrometers. Two current calibration transfer methods, subset recalibration and piecewise direct standardization (PDS), also utilize subset sample spectra to facilitate transfer of calibration. The three methods were applied to the transfer of quantitative multivariate calibration models for near-infrared (NIR) data of organic samples containing chlorobenzene, heptane, and toluene between a primary and three secondary spectrometers that were all the same model, called intra-vendor transfer of calibration. The hybrid PACLS/PLS method outperformed subset recalibration and provided predictions equivalent to PDS with additive background correction on the two secondary spectrometers whose instrument drift appeared to be dominated by simple linear baseline variations. One of the secondary spectrometers had complex instrument drift that was captured by repeatedly measuring the spectrum of a single repeat sample. In calculating a transfer function to correct prediction spectra, PDS assumes no instrumental drift on the secondary spectrometer. Therefore, PDS was unable to directly accommodate both the subset samples and the use of a single repeat sample to transfer and maintain a calibration on that secondary instrument. In order to implement the transfer of calibration with PDS in the presence of complex instrument drift, recalibrated PLS models that included the repeat spectra from the secondary spectrometer were used to predict the spectra transformed by PDS. The importance of correcting for drift on the secondary spectrometer during calibration transfer was illustrated by the improvements in prediction for all three methods vs. using only the instrument response differences derived from the subset sample spectra. When the effects of instrument drift were complex on the secondary spectrometer, the PACLS/PLS hybrid algorithm outperformed both PDS and subset recalibration. Through the explicit incorporation of spectral variations, due to instrument response differences and drift on the secondary spectrometer, the PACLS/PLS algorithm was successful at intra-vendor transfer of calibrations between NIR spectrometers.

Optimization and characterization of an imaging Hadamard spectrometer

Hadamard Transform Spectrometer (HTS) approaches share the multiplexing advantages found in Fourier transform spectrometers. Interest in Hadamard systems has been limited due to data storage/computational limitations and the inability to perform accurate high order masking in a reasonable amount of time. Advances in digital micro-mirror array (DMA) technology have opened the door to implementing an HTS for a variety of applications including fluorescent microscope imaging and Raman imaging. A Hadamard transform spectral imager (HTSI) for remote sensing offers a variety of unique capabilities in one package such as variable spectral and temporal resolution, no moving parts (other than the micro-mirrors) and vibrational insensitivity. An HTSI for remote sensing using a Texas Instrument digital micro-mirror device (DMD) is being designed for use in the spectral region 1.25 - 2.5 micrometers . In an effort to optimize and characterize the system, an HTSI sensor system simulation has been concurrently developed. The design specifications and hardware components for the HTSI are presented together with results calculated by the HTSI simulation that include the effects of digital (vs. analog) scene data input, detector noise, DMD rejection ratios, multiple diffraction orders and multiple Hadamard mask orders.

Importance of Prediction Outlier Diagnostics in Determining a Successful Inter-Vendor Multivariate Calibration Model Transfer:

This paper reports on the transfer of calibration models between Fourier transform near-infrared (FT-NIR) instruments from four different manufacturers. The piecewise direct standardization (PDS) method is compared with the new hybrid calibration method known as prediction augmented classical least squares/partial least squares (PACLS/PLS). The success of a calibration transfer experiment is judged by prediction error and by the number of samples that are flagged as outliers that would not have been flagged as such if a complete recalibration were performed. Prediction results must be acceptable and the outlier diagnostics capabilities must be preserved for the transfer to be deemed successful. Previous studies have measured the success of a calibration transfer method by comparing only the prediction performance (e.g., the root mean square error of prediction, RMSEP). However, our study emphasizes the need to consider outlier detection performance as well. As our study illustrates, the RMSEP values for a calibration transfer can be within acceptable range; however, statistical analysis of the spectral residuals can show that differences in outlier performance can vary significantly between competing transfer methods. There was no statistically significant difference in the prediction error between the PDS and PACLS/PLS methods when the same subset sample selection method was used for both methods. However, the PACLS/PLS method was better at preserving the outlier detection capabilities and therefore was judged to have performed better than the PDS algorithm when transferring calibrations with the use of a subset of samples to define the transfer function. The method of sample subset selection was found to make a significant difference in the calibration transfer results using the PDS algorithm, while the transfer results were less sensitive to subset selection when the PACLS/PLS method was used.

High Speed 2D Hadamard Transform Spectral Imager

Hadamard Transform Spectrometer (HTS) approaches share the multiplexing advantages found in Fourier transform spectrometers. Interest in Hadamard systems has been limited due to data storage/computational limitations and the inability to perform accurate high order masking in a reasonable amount of time. Advances in digital micro-mirror array (DMA) technology have opened the door to implementing an HTS for a variety of applications including fluorescent microscope imaging and Raman imaging. A Hadamard transform spectral imager (HTSI) for remote sensing offers a variety of unique capabilities in one package such as variable spectral and temporal resolution, no moving parts (other than the micro-mirrors) and vibration tolerance. Two approaches to for 2D HTS systems have been investigated in this LDRD. The first approach involves dispersing the incident light, encoding the dispersed light then recombining the light. This method is referred to as spectral encoding. The other method encodes the incident light then disperses the encoded light. The second technique is called spatial encoding. After creating optical designs for both methods the spatial encoding method was selected as the method that would be implemented because the optical design was less costly to implement.

Theoretical description and numerical simulations of a simplified Hadamard transform imaging spectrometer

A familiar concept in imaging spectrometry is that of the three dimensional data cube, with one spectral and two spatial dimensions. However, available detectors have at most two dimensions, which generally leads to the introduction of either scanning or multiplexing techniques for imaging spectrometers. For situations in which noise increases less rapidly than as the square root of the signal, multiplexing techniques have the potential to provide superior signal-to-noise ratios. This paper presents a theoretical description and numerical simulations for a new and simple type of Hadamard transform multiplexed imaging spectrometer. Compared to previous types of spatially encoded imaging spectrometers, it increases etendue by eliminating the need for anamorphically compressed re-imaging onto the entrance aperture of a monochromator or spectrophotometer. Compared to previous types of spectrally encoded imaging spectrometers, it increases end-to-end transmittance by eliminating the need for spectral re-combining optics. These simplifications are attained by treating the pixels of a digital mirror array as virtual entrance slits and the pixels of a 2-D array detector as virtual exit slits of an imaging spectrometer, and by applying a novel signal processing technique.

Reducing System Artifacts in Hyperspectral Image Data Analysis with the Use of Estimates of the Error Covariance in the Data

Hyperspectral Fourier transform infrared images have been obtained from a neoprene sample aged in air at elevated temperatures. The massive amount of spectra available from this heterogeneous sample provides the opportunity to perform quantitative analysis of the spectral data without the need for calibration standards. Multivariate curve resolution (MCR) methods with non-negativity constraints applied to the iterative alternating least squares analysis of the spectral data has been shown to achieve the goal of quantitative image analysis without the use of standards. However, the pure-component spectra and the relative concentration maps were heavily contaminated by the presence of system artifacts in the spectral data. We have demonstrated that the detrimental effects of these artifacts can be minimized by adding an estimate of the error covariance structure of the spectral image data to the MCR algorithm. The estimate is added by augmenting the concentration and pure-component spectra matrices with scores and eigenvectors obtained from the mean-centered repeat image differences of the sample. The implementation of augmentation is accomplished by employing efficient equality constraints on the MCR analysis. Augmentation with the scores from the repeat images is found to primarily improve the pure-component spectral estimates while augmentation with the corresponding eigenvectors primarily improves the concentration maps. Augmentation with both scores and eigenvectors yielded the best result by generating less noisy pure-component spectral estimates and relative concentration maps that were largely free from a striping artifact that is present due to system errors in the FT-IR images. The MCR methods presented are general and can also be applied productively to non-image spectral data.