Andrew G. Bruce

Block Coordinate Relaxation Methods for Nonparametric Wavelet Denoising

Abstract An important class of nonparametric signal processing methods entails forming a set of predictors from an overcomplete set of basis functions associated with a fast transform (e.g., wavelet packets). In these methods, the number of basis functions can far exceed the number of sample values in the signal, leading to an ill-posed prediction problem. The “basis pursuit” denoising method of Chen, Donoho, and Saunders regularizes the prediction problem by adding an l 1 penalty term on the coefficients for the basis functions. Use of an l 1 penalty instead of l 2 has significant benefits, including higher resolution of signals close in time/frequency and a more parsimonious representation. The l 1 penalty, however, poses a challenging optimization problem that was solved by Chen, Donoho and Saunders using a novel application of interior-point algorithms (IP). This article investigates an alternative optimization approach based on block coordinate relaxation (BCR) for sets of basis functions that are the finite union of sets of orthonormal basis functions (e.g., wavelet packets). We show that the BCR algorithm is globally convergent, and empirically, the BCR algorithm is faster than the IP algorithm for a variety of signal denoising problems.

Robust wavelet denoising

For extracting a signal from noisy data, waveshrink and basis pursuit are powerful tools both from an empirical and asymptotic point of view. They are especially efficient at estimating spatially inhomogeneous signals when the noise is Gaussian. Their performance is altered when the noise has a long tail distribution, for instance, when outliers are present. We propose a robust wavelet-based estimator using a robust loss function. This entails solving a nontrivial optimization problem and appropriately choosing the smoothing and robustness parameters. We illustrate the advantage of the robust wavelet denoising procedure on simulated and real data.

Wavelet analysis [for signal processing]

As every engineering student knows, any signal can be portrayed as an overlay of sinusoidal waveforms of assorted frequencies. But while classical analysis copes superbly with naturally occurring sinusoidal behavior-the kind seen in speech signals-it is ill-suited to representing signals with discontinuities, such as the edges of features in images. Latterly, another powerful concept has swept applied mathematics and engineering research: wavelet analysis. In contrast to a Fourier sinusoid, which oscillates forever, a wavelet is localized in time-it lasts for only a few cycles. Like Fourier analysis, however, wavelet analysis uses an algorithm to decompose a signal into simpler elements. Here, the authors describe how localized waveforms are powerful building blocks for signal analysis and rapid prototyping-and how they are now available in software toolkits.

Wavelet shrinkage for unequally spaced data

Wavelet shrinkage (WaveShrink) is a relatively new technique for nonparametric function estimation that has been shown to have asymptotic near-optimality properties over a wide class of functions. As originally formulated by Donoho and Johnstone, WaveShrink assumes equally spaced data. Because so many statistical applications (e.g., scatterplot smoothing) naturally involve unequally spaced data, we investigate in this paper how WaveShrink can be adapted to handle such data. Focusing on the Haar wavelet, we propose four approaches that extend the Haar wavelet transform to the unequally spaced case. Each approach is formulated in terms of continuous wavelet basis functions applied to a piecewise constant interpolation of the observed data, and each approach leads to wavelet coefficients that can be computed via a matrix transform of the original data. For each approach, we propose a practical way of adapting WaveShrink. We compare the four approaches in a Monte Carlo study and find them to be quite comparable in performance. The computationally simplest approach (isometric wavelets) has an appealing justification in terms of a weighted mean square error criterion and readily generalizes to wavelets of higher order than the Haar.

Duplicate document detection in DocBrowse

Duplicate documents are frequently found in large databases of digital documents, such as those found in digital libraries or in the government declassification effort. Efficient duplicate document detection is important not only to allow querying for similar documents, but also to filter out redundant information in large document databases. We have designed three different algorithm to identify duplicate documents. The first algorithm is based on features extracted from the textual content of a document, the second algorithm is based on wavelet features extracted from the document image itself, and the third algorithm is a combination of the first two. These algorithms are integrated within the DocBrowse system for information retrieval from document images which is currently under development at MathSoft. DocBrowse supports duplicate document detection by allowing (1) automatic filtering to hide duplicate documents, and (2) ad hoc querying for similar or duplicate documents. We have tested the duplicate document detection algorithms on 171 documents and found that text-based method has an average 11-point precision of 97.7 percent while the image-based method has an average 11- point precision of 98.9 percent. However, in general, the text-based method performs better when the document contains enough high-quality machine printed text while the image- based method performs better when the document contains little or no quality machine readable text.

Non-Gaussian seasonal adjustment: X-12-ARIMA versus robust structural models

This study compares X-12-ARIMA and MING, two new seasonal adjustment methods designed to handle outliers and structural changes in a time series. X-12-ARIMA is a successor to the X-11-ARIMA seasonal adjustment method, and is being developed at the US Bureau of the Census. MING is a ‘Mixture based Non-Gaussian’ method for seasonal adjustment using time series structural models and is implemented as a function in the S-Plus language. The procedures are compared using 29 macroeconomic time series from the US Bureau of the Census. These series have both outliers and structural changes, providing a good testbed for comparing non-Gaussian methods. For the 29 series, the X-12-ARIMA decomposition consistently leads to smoother seasonal factors which are as or more ‘flexible’ than the MING seasonal component. On the other hand, MING is more stable, particularly in the way it handles outliers and level shifts. This study relies heavily on graphical tools for comparing seasonal adjustment methods.

WaveShrink: shrinkage functions and thresholds

Donoho and Johnstones WaveShrink procedure has proven valuable for signal de-noising and non-parametric regression. WaveShrink is based on the principle of shrinking wavelet coefficients towards zero to remove noise. WaveShrink has very broad asymptotic near- optimality properties. In this paper, we introduce a new shrinkage scheme, semisoft, which generalizes hard and soft shrinkage. We study the properties of the shrinkage functions, and demonstrate that semisoft shrinkage offers advantages over both hard shrinkage (uniformly smaller risk and less sensitivity to small perturbations in the data) and soft shrinkage (smaller bias and overall L2 risk). We also construct approximate pointwise confidence intervals for WaveShrink and address the problem of threshold selection.

Smoothing and Robust Wavelet Analysis

In a series of papers, Donoho and Johnstone develop a powerful theory based on wavelets for extracting non-smooth signals from noisy data. Several nonlinear smoothing algorithms are presented which provide high performance for removing Gaussian noise from. a wide range of spatially inhomogeneous signals. However, like other methods based on the linear wavelet transform, these algorithms are very sensitive to certain types of non-Gaussian noise, such as outliers. In this paper, we develop outlier resistant wavelet transforms. In these transforms, outliers and outlier patches are localized to just a few scales. By using the outlier resistant wavelet transforms, we improve upon the Donoho and Johnstone nonlinear signal extraction methods. The outlier resistant wavelet algorithms are included with the S+Wavelets object-oriented toolkit for wavelet analysis.

DocBrowse: a system for information retrieval from document image data

This paper presents the software architecture for DocBrowse: a system for mixed text/graphics document image analysis and retrieval. DocBrowse is an open and extensible environment that permits the user to visually manage and perform queries on highly degraded document image databases. DocBrowse also serves as a research environment for developing document image analysis and query by image example (QBIE) algorithms. The system consists of a user interface, an object-relational document database and a variety of document image analysis engines. Using DocBrowse, it is possible to perform queries that retrieve documents based on both graphical and textual content. We describe the graphical user interface and visual image browser that is used to perform such queries. We also describe our approach to QBIE, the database structure, and the analysis engines incorporated in DocBrowse.

Lossy compression of acoustic backscatter data

We develop lossy compression algorithms for underwater acoustic data and evaluate the effects of the compression on two applications: target-detection and study of ocean floor temperature. We use data from an experiment of sediment transport conducted by the Applied Physics Laboratory at the University of Washington. We apply a variety of wavelet- based vector quantization data compression algorithms to acoustic sonar scans. We sue pruned tree-structured vector quantization (PTSVQ) with the generalized Breiman, Friedman, Olshen, and Stone algorithm to simultaneously prune trees that correspond to different wavelet subbands. We determine that while targets can be detected at compression ratios of over 100:1, compression ratios greater than 4:1 lead to unacceptable loss in accuracy for use of the data in ocean floor temperature studies. We find that PTSVQ applied to wavelet coefficients uniformly gives better results at low bit rates than PTSVQ applied to the untransformed data. For target detection, better compression is obtained by compressing in polar coordinates while for ocean temperature measurement, better compression is obtained by compressing in Cartesian coordinates. Finally although SNR versus entropy measurements are a popular and easy way of measuring the success of compression experiments, they are not good predictors of compression performance for scientific data.