Y. Mallet

Classification using adaptive wavelets for feature extraction

A major concern arising from the classification of spectral data is that the number of variables or dimensionality often exceeds the number of available spectra. This leads to a substantial deterioration in performance of traditionally favoured classifiers. It becomes necessary to decrease the number of variables to a manageable size, whilst, at the same time, retaining as much discriminatory information as possible. A new and innovative technique based on adaptive wavelets, which aims to reduce the dimensionality and optimize the discriminatory information is presented. The discrete wavelet transform is utilized to produce wavelet coefficients which are used for classification. Rather than using one of the standard wavelet bases, we generate the wavelet which optimizes specified discriminant criteria.

RECENT DEVELOPMENTS IN DISCRIMINANT ANALYSIS ON HIGH DIMENSIONAL SPECTRAL DATA

There are basically two strategies which can be used to discriminate high dimensional spectral data. It is common practice to first reduce the dimensionality by some feature extraction preprocessing method, and then use an appropriate (low-dimensional) classifier. An alternative procedure is to use a (high-dimensional) classifier which is capable of handling a large number of variables. We introduce some novel dimension reducing techniques as well as low and high dimensional classifiers which have evolved only recently. The discrete wavelet transform is introduced as a method for extracting features. The Fourier transform, principal component analysis, stepwise strategies, and other variable selection methods for reducing the dimensionality are also discussed. The low dimensional classifier, flexible discriminant analysis is a new method which combines nonparametric regression with Fishers linear discriminant analysis to achieve nonlinear decision boundaries. We also discuss some of the time honoured techniques such as Fishers linear discriminant analysis, and the Bayesian linear and quadratic methods. The modern high dimensional classifiers which we report on are penalized discriminant analysis and regularized discriminant analysis. Each of the classifiers and a selection of dimensionality reducing techniques are applied to the discrimination of seagrass spectral data. Results indicate a promising future for wavelets in discriminant analysis, and the recently introduced flexible and penalized discriminant analysis. Regularized discriminant analysis also performs well.

Integrated Feature Extraction Using Adaptive Wavelets

On-line quality control in the manufacturing and processing industry is increasingly being undertaken by analyzing collinear data such as spectra sampled by a spectrometer. Spectral data are very highly dimensional and are characterized by having highly correlated, localized structures. Wavelets are therefore most effective in extracting the important local features in spectra by reducing the number of variables whilst, at the same time, retaining as much information as possible and facilitating the automated analysis and interpretation of spectra. There are many kinds of wavelets which exist in the literature, but the fundamental problem to overcome is deciding which wavelet will produce the best results for a particular application. Rather than using an ‘off-the-shelf’wavelet, an automated search is performed for the wavelet which optimizes specified multivariate modeling criteria. The spectral data analyzed in this chapter are of importance to the agricultural, pharmaceutical and mining industries as well as the environmental sciences.

Chapter 3 – Fundamentals of Wavelet Transforms

This chapter discusses various aspects of the wavelet transform when applied to continuous functions or signals. Wavelets form a set of basis functions, which linearly combine to represent a function f(t), from the space of square integrable functions L 2 (R). Functions in this space have finite energy. Because wavelet basis functions linearly combine to represent functions from L 2 (R) they are from this space as well. The reason for choosing this space largely relates to the properties of the L 2 norm. The wavelet basis functions are derived by translating and dilating one basic wavelet, called a “mother wavelet.” The dilated and translated wavelet basis functions are called “children wavelets.” The wavelet coefficients are the coefficients in the expansion of the wavelet basis functions. The wavelet transform is the procedure for computing the wavelet coefficients. The wavelet coefficients convey information about the weight that a wavelet basis function contributes to the function. The chapter introduces the continuous wavelet transform and discusses the conditions required for invertibility and the inverse or reconstruction formula.

Chapter 6 - Wavelet Packet Transforms and Best Basis Algorithms

This chapter provides an overview of wavelet packet transforms and best basis algorithms. The wavelet packet transform (WPT) is an extension of the discrete wavelet transform (DWT). The basic difference between the wavelet packet transform and the wavelet transform relates to which coefficients are passed through the low-pass and high-pass filters. With the wavelet transform, the scaling coefficients are filtered through each of these filters. With the WPT, not only do the scaling coefficients pass through the low-pass and high-pass filters, but so do the wavelet coefficients. Because both the scaling and wavelet coefficients are filtered there is a surplus of information stored in the WPT, which has a binary tree structure. An advantage of this redundant information is that it provides greater freedom in choosing an orthogonal basis. The best basis algorithm seeks a basis in the WPT, which optimizes some criterion function. Thus, the best basis algorithm is a task specific algorithm in that the particular basis is dependent on the role for which it is used.

Robust and Non-parametric Methods in Multiple Regressions of Environmental Data

Statistical regression methods in environmental chemistry are of vital importance. Regression techniques provide environmental chemical analysts with the ability to calibrate instruments and model large environmental systems.

Chapter 18 - Application of Adaptive Wavelets in Classification and Regression

This chapter demonstrates the ways in which the adaptive wavelet algorithm can be implemented in conjunction with classification analysis and regression methods. The data used in each of these applications are spectral data sets where the reflectance/absorbance of substances is measured at regular increments in the wavelength domain. Discriminant analysis techniques (also called classification techniques) are concerned with classifying objects into one of two or more classes. Discriminant techniques are considered to be learning procedures. Given, a set of objects whose class identity is known, a model “learns” from the variables that have been measured for each of the objects—a procedure, which can be used to assign a new object, whose class identity is unknown, into one of the predefined classes. Such a procedure is performed using a well-defined discriminatory rule. The chapter explains classification assessment criteria. The correct classification rate (CCR) or misclassification rate (MCR) is the most favored assessment criteria in discriminant analysis. The chapter presents results in terms of correct classification rates, for their ease in interpretation and uses a probability based criterion function in the construction of the filter coefficients.

Chapter 8 - The Adaptive Wavelet Algorithm for Designing Task Specific Wavelets

The adaptive wavelet algorithm for designing task specific wavelets is described in the chapter. There exist many different kinds or families of wavelets. These wavelet families are defined by their respective filter coefficients, which are readily available for the situation when m = 2 and include for example the Daubechies wavelets, Coiflets, Symlets, and the Meyer and Haar wavelets. One basic issue to overcome is deciding, which set (or family) of filter coefficients produce the best results for a particular application. It is possible to trial different sets of filter coefficients and proceeds with the family of filter coefficients, which produces the most desirable results. The chapter describes one method for generating own set of filter coefficients. The chapter demonstrates the ways in which wavelets can be designed to suit almost any general application, and designing wavelets for the classification of spectral data is also considered.

Chapter 4 – The Discrete Wavelet Transform in Practice

In this chapter we present the finite discrete wavelet transform (DWT) using matrices. The main difference between the description of the DWT in this chapter, compared to the description given in the previous chapter, is now we consider the DWT for discrete data, as opposed to continuous functions. We first provide a brief introduction to matrix theory and discuss some special forms of matrices such as, orthogonal and banded matrices. We then give the matrix representation of the DWT for finite-length signals and discuss some of the practical issues, including signal length and boundary conditions. The same techniques presented in this chapter can be extended to provide the basis for the development of task specific wavelets in Chapter 7. The reader who is familiar with matrix terms and definitions may skip Section 2.