Kenneth C. Chou

Multiscale systems, Kalman filters, and Riccati equations

An algorithm analogous to the Rauch-Tung-Striebel algorithm/spl minus/consisting of a fine-to-coarse Kalman filter-like sweep followed by a coarse-to-fine smoothing step/spl minus/was developed previously by the authors (ibid. vol.39, no.3, p.464-78 (1994)). In this paper they present a detailed system-theoretic analysis of this filter and of the new scale-recursive Riccati equation associated with it. While this analysis is similar in spirit to that for standard Kalman filters, the structure of the dyadic tree leads to several significant differences. In particular, the structure of the Kalman filter error dynamics leads to the formulation of an ML version of the filtering equation and to a corresponding smoothing algorithm based on triangularizing the Hamiltonian for the smoothing problem. In addition, the notion of stability for dynamics requires some care as do the concepts of reachability and observability. Using these system-theoretic constructs, the stability and steady-state behavior of the fine-to-coarse Kalman filter and its Riccati equation are analysed. >

Recursive and iterative estimation algorithms for multiresolution stochastic processes

A particular class of processes defined on dyadic trees is treated. Three algorithms are given for optimal estimation/reconstruction for such processes: one reminiscent of the Laplacian pyramid and making efficient use of Haar transforms, a second that is iterative in nature and can be viewed as a multigrid relaxation algorithm, and a third that represents an extension of the Rauch-Tung-Striebel algorithm to processes on dyadic trees. The last involves a discrete Riccati equation, which in this case has three steps: prediction, merging and measurement update. Related work and extensions are briefly discussed.<<ETX>>

Modeling and estimation of multiscale stochastic processes

The authors introduce a class of multiscale stochastic processes which are Markov in scale and which are characterized by dynamic state models evolving in scale. The models for these processes are motivated by the theory of multiscale representations and the wavelet transform. The authors formulate an optimal estimation problem based on these models, which has potential applications to sensor fusion problems where there exist data from sensors of differing resolution, and provide an efficient algorithm based on the wavelet transform. They give examples applying these models to first-order Gauss-Markov processes.<<ETX>>

Multiresolution stochastic models, data fusion, and wavelet transforms

Abstract In this paper we describe and analyze a class of multiscale stochastic processes which are modeled using dynamic representations evolving in scale based on the wavelet transform. The statistical structure of these models is Markovian in scale, and in addition the eigenstructure of these models is given by the wavelet transform. The implication of this is that by using the wavelet transform we can convert the apparently complicated problem of fusing noisy measurements of our process at several different resolutions into a set of decoupled, standard recursive estimation problems in which scale plays the role of the time-like variable. In addition we show how the wavelet transform, which is defined for signals that extend from −∞ to +∞, can be adapted to yield a modified transform matched to the eigenstructure of our multiscale stochastic models over finite intervals. Finally, we illustrate the promise of this methodology by applying it to estimation problems, involving single and multi-scale data, for a first-order Gauss-Markov process. As we show, while this process is not precisely in the class we define, it can be well-approximated by our models, leading to new, highly parallel and scale-recursive estimation algorithms for multi-scale data fusion. In addition our framework extends immediately to 2D signals where the computational benefits are even more significant.

Maximum likelihood identification of multiscale stochastic models using the wavelet transform and the EM algorithm

The authors address the problem of estimating the parameters of a class of multiscale stochastic processes that can be modeled by state-space dynamic systems driven by white noise in scale rather than in time. They present a maximum likelihood identification method for estimating the parameters of the multiscale stochastic models given data which are based on the wavelet transform and the expectation-maximization algorithm. Numerical examples are provided for identifying the parameters of the state-space models based on synthesized data to demonstrate the accuracy and the efficiency of the algorithm. In the examples the effects of performing system identification are illustrated based on data at both multiple and single scales. The single-scale case can be viewed as the standard problem of fitting model parameters to data, where here the model is not standard.<<ETX>>

Gaussian mixture model classifiers for machine monitoring

We describe a statistical pattern-recognition approach to machine monitoring. The approach comprises a classification scheme using Gaussian mixture models (GMMs) that classifies features based on a time-frequency representation using the wavelet transform. The GMM trained with the EM algorithm has comparable flexibility with the multilayered perceptron in modeling nonstationary, multimodal machine signal characteristics, but has significantly fewer parameters to train. Also, using an example set of machine signals we show that the wavelet transform is particularly appropriate for capturing the time-frequency properties of transients of varying time constants and harmonic content. The benefits of both the GMM classifier and wavelet representation are manifested in superior classification performance and much lower computational complexity, as well as better robustness to finite-sample effects.<<ETX>>

Representation of Green's Function Integral Operators Using Wavelet Transforms

In this paper, we analyze the representation of integral operators whose kernels are Greens functions for a class of linear differential equations using wavelets with a finite number of vanishing moments. In particular, we show how wavelets can be used to generate a sparse representation of these operators. We show that the matrix associated with the discretized integral operator represented in the wavelet basis is sparse and, in particular, contains multiple bands of various widths. The particular banded structure of the wavelet representation of the operator follows from the fact that the associated Greens function is smooth away from the source point and is singular at some order; that is, for some T, its Tth derivative is discontinuous at the source point. We derive bounds on the magnitude of the coefficients of the integral operator in the wavelet basis as a function of scale and position and, in particular, in terms of whether or not the coefficient lies within a band. Based on these bounds, we can approximate the operator by ignoring coefficients not lying within these bands, thus producing a sparse representation. This sparse representation is extremely beneficial for numerical applications in which one would like to apply the Greens function operator efficiently: normally if such an operator mapped N points into N points, it would require O(N 2) operations; however, with the wavelet transform, the mapping would require only O((4M + 2γLM + 2γ(1 - γ)M - 3)N) operations, where 2M is the length associated with the support of the wavelet function, L = log2 N - 1, and γ = 1/ln2. An application example in which this is important is the control of smart structures in which a large number of embedded sensors and actuators must be coordinated to achieve disturbance rejection on the surface of the structure.

A MULTI-RESOLUTION , PROBABILISTIC APPROACH TO TWO- DIMENSIONAL INVERSE CONDUCTIVITY PROBLEMS*

Abstract In this paper a method of estimating the conductivity in a bounded 2-D domain at multiple spatial resolutions given boundary excitations and measurements is presented. The problem is formulated as a maximum-likelihood estimation problem and an algorithm that consists of a sequence of estimates at successively finer scales is presented. By exploiting the structure of the physics, the problem at each scale is divided into two linear subproblems, each of which is solved using parallelizable relaxation schemes. The success of our algorithm on synthetic data is demonstrated and numerical results based on using the algorithm as a tool for exploring estimation performance is presented, as well as results based on using the algorithm to study the well-posedness of the problem, and the effects of fine-scale variations on coarse scale estimates. Examples based on analytical results that further the understanding of these issues are also presented. The results suggest the use of inhomogeneous spatial scales as a possible way of overcoming ill-posedness at points far away from the boundary.

Kalman filtering and Riccati equations for multiscale processes

Multiscale representations of signals and multiscale algorithms are addressed. A two-sweep smoothing algorithm is analyzed for fusing multiscale measurements of multiscale processes defined on trees. The algorithm is a generalization of the Rauch-Tung-Striebel algorithm for the smoothing of time series, and the filtering step differs from that of time series in that it consists of the successive fusing of data from level to level, thus introducing a new type of Riccati equation. The fusion step makes it necessary to view the optimal estimation as producing a maximum likelihood (ML) estimate which is then combined with prior statistics, and it is the dynamics of the ML estimate recursion which must be analyzed. Elements of a system theory required to derive bounds on the error covariance of the filter are developed. These results are then used along with a careful definition of stability on trees both to prove the stability of the filter and to give results on the steady-state filter.<<ETX>>

Modeling And Estimation Of Multiresolution Stochastic Processes And Random Fields

In this poster, we provide an overview of several components of a research effort aimed at the development of a theory of multiresolution stochastic modeling and associated techniques for optimal multiscale statistical signal and image processing. As we describe, a natural framework for developing such atheory is the study of stochastic processes indexed by nodes on lattices or trees in which different depths in the tree or lattice correspond to different spatial scales in representing a signal or image. In particular we show how the wavelet transform directly suggests such a modeling paradigm. This perspective then leads directly to the investigation of several classes of dynamic models and related notions of “multiscale stationarity” in which scale plays the role of a time-like variable. In particular we describe the elements of a dynamic system theory on trees based on a specific notion of stationarity on trees. This notion of stationarity leads directly to a class of state space models on homogeneous trees. We describe several elements of the system theory for such models and also describe the natural, extremely efficient algorithmic structures for optimal estimation that these models suggest: one class of algorithms has a multigrid relaxation structure; a second uses the scale-to-scale whitening property of wavelet transforms for our models; and a third leads to a new class of Riccati equations involving the usual predict and update steps and a new ‘(fusion” step as information is propagated from fine to coarse scales. This framework allows us to consider in a very natural way the fusion of data from sensors with differing resolutions. In particular we present results on the fusion of noisy 1st-order Gauss-Markov processes which demonstrate the richness of our class of models in approximating processes. These results also show the ease with which this framework allows us to interpolate sparse, non-uniformly sampled fine data using coarser measurements with fuller coverage. lLaboratory for Information and Decision Systems and Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge,MA 02139, USA. The work of these authors was also supported in part by the Air Force Office of Scientific Research under Grant AFOSR-88-0032, by the National Science Foundation under Grants 9015281-MIP and INT-9002393 and by the US Army Research Office under Contract DAAL03-86-K-0171. In addition some of this research was performed while KCC and ASW were visitors at IRISA and while ASW received support from INRIA.