W.J. Melssen

Using support vector machines for time series prediction

Abstract Time series prediction can be a very useful tool in the field of process chemometrics to forecast and to study the behaviour of key process parameters in time. This creates the possibility to give early warnings of possible process malfunctioning. In this paper, time series prediction is performed by support vector machines (SVMs), Elman recurrent neural networks, and autoregressive moving average (ARMA) models. A comparison of these three methods is made based on their predicting ability. In the field of chemometrics, SVMs are hardly used even though they have many theoretical advantages for both classification and regression tasks. These advantages stem from the specific formulation of a (convex) objective function with constraints which is solved using Lagrange Multipliers and has the characteristics that: (1) a global optimal solution exists which will be found, (2) the result is a general solution avoiding overtraining, (3) the solution is sparse and only a limited set of training points contribute to this solution, and (4) nonlinear solutions can be calculated efficiently due to the usage of inner products. The method comparison is performed on two simulated data sets and one real-world industrial data set. The simulated data sets are a data set generated according to the ARMA principles and the Mackey–Glass data set, often used for benchmarking. The first data set is relatively easy whereas the second data set is a more difficult nonlinear chaotic data set. The real-world data set stems from a filtration unit in a yarn spinning process and it contains differential pressure values. These values are a measure of the contamination level of a filter. For practical purposes, it is very important to predict these values accurately because they play a crucial role in maintaining the quality of the process considered. As it is expected, it appears that the ARMA model performs best for the ARMA data set while the SVM and the Elman networks perform similarly. For the more difficult benchmark data set the SVM outperforms the ARMA model and in most of the cases outperforms the best of several Elman neural networks. For the real-world set, the SVM was trained using a training set containing only one tenth of the points of the original training set which was used for the other methods. This was done to test its performance if only few data would be available. Using the same test set for all methods, it appeared that prediction results were equally well for both the SVM and the ARMA model whereas the Elman network could not be used to predict these series.

Using artificial neural networks for solving chemical problems Part I. Multi-layer feed-forward networks

Abstract Smits, J.R.M., Melssen, W.J., Buydens, L.M.C. and Kateman, G., 1994. Using artificial neural networks for solving chemical problems. Part I. Multi-layer feed-forward networks. Chemometrics and Intelligent Laboratory Systems , 22: 165–189. This tutorial focuses on the practical issues concerning applications of different types of neural networks. The tutorial is divided into two parts. In the first part, an overview of the general appearance of neural networks is given and the multi-layer feed-forward neural network is described. In the second part, the Kohonen self-organising feature map and the Hopfield network are discussed. Since the multi-layer feed-forward neural network is one of the most popular networks, the theory concerning this network can easily be found in other references (B.J. Wythoff, Chemom. Intell. Lab. Syst. , 18 (1993) 115–155) and is therefore only described superficially in this paper. Much attention is paid to the practical issues concerning applications of the networks. For each network, a description is given of the types of problems which can be tackled by the specific neural network, followed by a protocol for the development of the system. It is seen that different neural networks are suited for different kinds of problems. Application of the networks is not always straightforward; a lot of constraints and conditions have to be fulfilled when using neural networks properly. They appear to be powerful techniques, but often a lot of experience is needed. In this paper some guidelines are given to avoid the most common difficulties in applying neural networks to chemical problems.

USING ARTIFICIAL NEURAL NETWORKS FOR SOLVING CHEMICAL PROBLEMS .2. KOHONEN SELF-ORGANIZING FEATURE MAPS AND HOPFIELD NETWORKS

This second part of a Tutorial on neural networks focuses on the Kohonen self-organising feature map and the Hopfield network. First a theoretical description of each type is given. The practical issues concerning applications of the networks are then discussed. For each network, a description is given of the types of problems which can be tackled by the specific neural network, followed by a protocol for the development of the neural network system. It is seen that different neural networks are suited for different kinds of problems. Guidelines to avoid common difficulties in using neural networks are also given.

Selecting a representative training set for the classification of demolition waste using remote NIR sensing

Abstract In the AUTOSORT project, the goal is the separation of demolition waste in three fractions: wood, plastics and stone. A remote near-infrared sensor measures reduced reflectance spectra (mini-spectra) of objects. Linear discriminant analysis (LDA) is used for the classification of these spectra. To obtain the LDA model, a representative training set is needed. New LDA-models will be regularly needed for recalibrations. Small training sets will save a lot of labour and additional costs. Two object selection methods are investigated: the Kennard–Stone algorithm and a statistical test procedure. Training sets are acquired from which the mini-spectra are used to obtain LDA models. In the training sets, the object amounts and their ratios are varied. Two object ratios are applied: the ratios as they occur in the complete data set and the equalised ratios. The Kennard–Stone selection algorithm is the preferred method. It gives a unique list of objects, mainly sampled at the cluster borders: partial cluster overlap is better defined. This is in contradiction with the sets of objects, accepted by the statistical test procedure: those objects tend to occur around the fraction means. This is a drawback for the classification performance: some accepted training sets are unacceptable. The ratios between the fraction amounts are not important, but equal fraction amounts are preferred. Selecting 25 objects for each fraction should be suitable.

Two-dimensional mapping of IR spectra using a parallel implemented self-organising feature map

Abstract Melssen, W.J., Smits, J.R.M., Rolf, G.H. and Kateman, G., 1993. Two-dimensional mapping of IR spectra using a parallel implemented self-organising feature map. Chemometrics and Intelligent Laboratory Systems , 18: 195–204. A large data base containing 3284 infrared (IR) spectra (1327 wavelengths) of various molecules was investigated with a self-organising feature map (Kohonen network). In order to reduce the time required to train the network, a parallel implementation of the algorithm was developed. Application of the Kohonen network appears to be a powerful technique in mapping a high dimensional data space onto a two-dimensional one. Fragment coding was used to indicate the presence or absence of chemical functional groups in a molecule. Two-dimensional maps have been constructed for several fragments. Some preliminary results are presented in this paper. It appeared that some of the fragments were mapped onto relatively small regions (clusters) in the map. Mostly, these fragments were characterised by a high separability index, indicating that these functional groups were easily recognised by the network. Next, it was shown that, for some of the fragments which formed clusters in the map, a further differentiation into sub-fragments appeared to be possible. We conclude that the analysis of Kohonen maps yields valuable information which may be used for the practical design of a modular tree-like system of dedicated multi-layer feed-forward neural networks for the automated interpretation of infrared spectra.

Automatic correction of peak shifts in Raman spectra before PLS regression

A simple algorithm is presented for the fast and fully automatic removal of peak shifts in large spectral data sets. It is able to remove peak shifts exceeding the discrete spectral resolution. The algorithm has been applied to Raman spectra of three solution copolymerizations of styrene (Sty) and butyl acrylate (BA) performed on three separate days. Small Raman peak shifts, smaller than the spectral resolution, and induced by the on/off switching of the laser and the repositioning of the grating, could successfully be removed prior to partial least squares (PLS) regression. It is demonstrated that after shift correction, the ability of PLS to predict Sty and BA concentrations is improved. Due to its speed, the algorithm is suitable to eliminate real time (on-line) spectral shifts from the spectra. This makes re-calibration or model adaptation forced by spectral shifts superfluous. The proposed algorithm may likewise be used to remove peak/wavelength shifts in mid-IR, near-IR, and NMR spectra, as well as to remove retention time shifts in chromatograms.

Application of principal component analysis to detect outliers and spectral deviations in near-field surface-enhanced Raman spectra

Abstract A recently developed technique measures near-field surface-enhanced Raman spectra with 100-nm resolution, enabling a fast survey on the sample surface. This technique has two bottlenecks. One is a general problem: signal changes are attributed to either the sample composition or the substrate morphology. Therefore, it is mandatory to detect even small signal changes in order to distinguish between these two effects. Secondly, huge data amounts make the spectrum interpretation tedious. How to find the interesting and important information? To investigate these problems, a sample, containing dye-labeled DNA-fragments that are drop-coated onto a silver island substrate, is measured. The enhanced Raman spectra yield indirect information on the DNA-fragments. The goal of this investigation is to provide a tool that allows a fast and reliable spectral analysis. Is it possible to distinguish local differences in the sample composition and to correlate them with the sample morphology? A general explorative data analyses tool, principal component analysis (PCA), is used for a first investigation. PCA has a useful side-effect: spikes, well-known artifacts, are also detected. After removing these artifacts, PCA facilitated the detection of three neighboring spectra, clearly deviating from the others. Probably, the DNA double-strand unfolded and generated a direct Raman-signal. The automated PCA-procedure gives identical results. It is concluded that a general explorative tool can solve two major difficulties. Application of dedicated chemometrical tools could improve the results. The combination of chemometrics and this new technique is powerful and promising.

Plastic material identification with spectroscopic near infrared imaging and artificial neural networks

Abstract A remote sensing spectroscopic near infrared (NIR) system has been installed in an experimental laboratory setup for real-time plastic identification in mixed household waste. The identification of waste objects is performed in two steps. First, the experimental measurement setup is used for the acquisition of the spectroscopic image data and second, a non-linear transformation is performed by a neural network for supervised classification of these measured images. This new identification system needs an evaluation of its on-line classification performance. However, not only the percentage of correct material classification is of interest, but also the corresponding precision and the circumstances of operation (robustness) such as differences in temperature and humidity. Furthermore, this qualitative identification system incorporates additional complications with respect to the variability of the sensor response, such as variable waste sample positions and shapes accompanied with contaminated waste samples. In order to validate such a system, the robustness, repeatability and reproducibility of the classification system are considered. The final identification system is able to identify plastics with the required success rate of 80%, but improvement is to be expected when some experimental parameters can be stabilized.

Parallel processing of chemical information in a local area network—II. A parallel cross-validation procedure for artificial neural networks

This paper describes a parallel cross-validation (PCV) procedure, for testing the predictive ability of multi-layer feed-forward (MLF) neural networks models, trained by the generalized delta learning rule. The PCV program has been parallelized to operate in a local area computer network. Development and execution of the parallel application was aided by the HYDRA programming environment, which is extensively described in Part I of this paper. A brief theoretical introduction on MLF networks is given and the problems, associated with the validation of predictive abilities, will be discussed. Furthermore, this paper comprises a general outline of the PCV program. Finally, the parallel PCV application is used to validate the predictive ability of an MLF network modeling a chemical non-linear function approximation problem which is described extensively in the literature.

Application of a Spectroscopic Infrared Focal Plane Array Sensor for On-Line Identification of Plastic Waste

A spectroscopic near-infrared imaging system, using a focal plane array (FPA) detector, is presented for remote and on-line measurements on a macroscopic scale. On-line spectroscopic imaging requires high-speed sensors and short image processing steps. Therefore, the use of a focal plane array detector in combination with fast chemometric software is investigated. As these new spectroscopic imaging systems generate so much data, multivariate statistical techniques are needed to extract the important information from the multidimensional spectroscopic images. These techniques include principal component analysis (PCA) and linear discriminant analysis (LDA) for supervised classification of spectroscopic image data. Supervised classification is a tedious task in spectroscopic imaging, but a procedure is presented to facilitate this task and to provide more insight into and control over the composition of the datasets. The identification system is constructed, implemented, and tested for a real-world application of plastic identification in municipal solid waste.