Guangzhi Cao

The Sparse Matrix Transform for Covariance Estimation and Analysis of High Dimensional Signals

Covariance estimation for high dimensional signals is a classically difficult problem in statistical signal analysis and machine learning. In this paper, we propose a maximum likelihood (ML) approach to covariance estimation, which employs a novel non-linear sparsity constraint. More specifically, the covariance is constrained to have an eigen decomposition which can be represented as a sparse matrix transform (SMT). The SMT is formed by a product of pairwise coordinate rotations known as Givens rotations. Using this framework, the covariance can be efficiently estimated using greedy optimization of the log-likelihood function, and the number of Givens rotations can be efficiently computed using a cross-validation procedure. The resulting estimator is generally positive definite and well-conditioned, even when the sample size is limited. Experiments on a combination of simulated data, standard hyperspectral data, and face image sets show that the SMT-based covariance estimates are consistently more accurate than both traditional shrinkage estimates and recently proposed graphical lasso estimates for a variety of different classes and sample sizes. An important property of the new covariance estimate is that it naturally yields a fast implementation of the estimated eigen-transformation using the SMT representation. In fact, the SMT can be viewed as a generalization of the classical fast Fourier transform (FFT) in that it uses “butterflies” to represent an orthonormal transform. However, unlike the FFT, the SMT can be used for fast eigen-signal analysis of general non-stationary signals.

Sparse Matrix Transform for Hyperspectral Image Processing

A variety of problems in remote sensing require that a covariance matrix be accurately estimated, often from a limited number of data samples. We investigate the utility of several variants of a recently introduced covariance estimator-the sparse matrix transform (SMT), a shrinkage-enhanced SMT, and a graph-constrained SMT-in the context of several of these problems. In addition to two more generic measures of quality based on likelihood and the Frobenius norm, we specifically consider weak signal detection, dimension reduction, anomaly detection, and anomalous change detection. The estimators are applied to several hyperspectral data sets, including some randomly rotated data, to elucidate the kinds of problems and the kinds of data for which SMT is well or poorly suited. The SMT is based on the product of K pairwise coordinate (Givens) rotations, and we also introduce and compare two novel approaches for estimating the most effective choice for K .

Fast signal analysis and decomposition on graphs using the Sparse Matrix Transform

Recently, the Sparse Matrix Transform (SMT) has been proposed as a tool for estimating the eigen-decomposition of high dimensional data vectors [1]. The SMT approach has two major advantages: First it can improve the accuracy of the eigendecomposition, particularlywhen the number of observations, n, is less the the vector dimension, p. Second, the resulting SMT eigen-decomposition is very fast to apply, i.e. O(p). In this paper, we present an SMT eigen-decomposition method suited for application to signals that live on graphs. This new SMT eigen-decomposition method has two major advantages over the more generic method presented in [1]. First, the resulting SMT can be more accurately estimated due to the graphical constraint. Second, the computation required to design the SMT from training data is dramatically reduced from an average observed complexity of p3 to p log p.

Noniterative MAP Reconstruction Using Sparse Matrix Representations

We present a method for noniterative maximum a posteriori (MAP) tomographic reconstruction which is based on the use of sparse matrix representations. Our approach is to precompute and store the inverse matrix required for MAP reconstruction. This approach has generally not been used in the past because the inverse matrix is typically large and fully populated (i.e., not sparse). In order to overcome this problem, we introduce two new ideas. The first idea is a novel theory for the lossy source coding of matrix transformations which we refer to as matrix source coding. This theory is based on a distortion metric that reflects the distortions produced in the final matrix-vector product, rather than the distortions in the coded matrix itself. The resulting algorithms are shown to require orthonormal transformations of both the measurement data and the matrix rows and columns before quantization and coding. The second idea is a method for efficiently storing and computing the required orthonormal transformations, which we call a sparse-matrix transform (SMT). The SMT is a generalization of the classical FFT in that it uses butterflies to compute an orthonormal transform; but unlike an FFT, the SMT uses the butterflies in an irregular pattern, and is numerically designed to best approximate the desired transforms. We demonstrate the potential of the noniterative MAP reconstruction with examples from optical tomography. The method requires offline computation to encode the inverse transform. However, once these offline computations are completed, the noniterative MAP algorithm is shown to reduce both storage and computation by well over two orders of magnitude, as compared to a linear iterative reconstruction methods.

Weak signal detection in hyperspectral imagery using sparse matrix transform (smt) covariance estimation

Many detection algorithms in hyperspectral image analysis, from well-characterized gaseous and solid targets to deliberately uncharacterized anomalies and anomalous changes, depend on accurately estimating the covariance matrix of the background. In practice, the background covariance is estimated from samples in the image, and imprecision in this estimate can lead to a loss of detection power. In this paper, we describe the sparse matrix transform (SMT) and investigate its utility for estimating the covariance matrix from a limited number of samples. The SMT is formed by a product of pairwise coordinate (Givens) rotations. Experiments on hyperspectral data show that the estimate accurately reproduces even small eigenvalues and eigenvectors. In particular, we find that using the SMT to estimate the covariance matrix used in the adaptive matched filter leads to consistently higher signal-to-clutter ratios.

Fast space-varying convolution and its application in stray light reduction

Space-varying convolution often arises in the modeling or restoration of images captured by optical imaging systems. For example, in applications such as microscopy or photography the distortions introduced by lenses typically vary across the field of view, so accurate restoration also requires the use of space-varying convolution. While space-invariant convolution can be efficiently implemented with the Fast Fourier Transform (FFT), space-varying convolution requires direct implementation of the convolution operation, which can be very computationally expensive when the convolution kernel is large. In this paper, we develop a general approach to the efficient implementation of space-varying convolution through the use of matrix source coding techniques. This method can dramatically reduce computation by approximately factoring the dense space-varying convolution operator into a product of sparse transforms. This approach leads to a tradeoff between the accuracy and speed of the operation that is closely related to the distortion-rate tradeoff that is commonly made in lossy source coding. We apply our method to the problem of stray light reduction for digital photographs, where convolution with a spatially varying stray light point spread function is required. The experimental results show that our algorithm can achieve a dramatic reduction in computation while achieving high accuracy.

Fast and Efficient Stored Matrix Techniques for Optical Tomography

A barrier to the use of optical tomography in practical applications is the high computational cost of iterative image reconstruction. This paper introduces a novel method for direct reconstruction of the image from a pre-computed and stored inverse matrix. Since the inverse matrix for optical tomography is generally quite large and not sparse, it is necessary to store the inverse matrix using lossy source coding techniques. A key innovation is the method used for matrix representation and the technique used for computing the required matrix-vector product. This representation is based on transforms of the image and sensor spaces which are designed to minimize reconstructed image distortion. Simulations indicate that the technique can dramatically reduce the storage and computation requirements by exploiting redundancy in the transformed matrix.

Results in non-iterative MAP reconstruction for optical tomography

Maximum a posteriori (MAP) estimation has been shown to be an effective method for reconstructing images from optical diffusion tomography data. However, one disadvantage of MAP reconstruction is that it typically requires the use of iterative methods which are computationally intensive. However, the direct reconstruction of MAP images is possible when the forward model is linear (or linearized) and the noise and image prior are assumed Gaussian. These non-iterative MAP reconstruction techniques only require the multiplication of an inverse matrix by a data vector to compute the reconstruction, but they depend on a combination of lossy source coding techniques and sparse matrix transforms to make the required matrix-vector product computation both computationally and memory efficient. In this paper, we show examples of how non-iterative MAP reconstruction methods can be used to dramatically reduce computation and storage for MAP reconstruction. Simulations of fluorescence optical diffusion tomography (FODT) measurements and corresponding reconstructions are used to demonstrate the potential value of these techniques. Numerical examples show the non-iterative MAP reconstruction can substantially reduce both storage and computation, as compared to traditional iterative reconstruction methods.

FAST RECONSTRUCTION ALGORITHMS FOR OPTICAL TOMOGRAPHY USING SPARSE MATRIX REPRESENTATIONS

This paper introduces a novel method for reconstructing optical tomography images using pre-computed transforms. Our approach is to pre-compute and store the inverse matrix required for MAP reconstruction using lossy source coding techniques. We show how lossy source coding techniques can be used to store the large and non-sparse matrix by applying a wavelet transform in the image space and appropriate orthonormal transforms in the sensor space. Lossy coding dramatically reduces the number of non-zero coefficients, thereby proportionately reducing both the required storage and computation time. However, if the number of sensor measurements is large, the storage and computation of the orthonormal transforms can become prohibitive. For this purpose, we introduce a general method for approximating any orthonormal transform by a series of sparse binary transforms. This sparse matrix transform technique is then used together with lossy coding to result in a fast reconstruction algorithm for optical tomography. Simulations indicate that the technique can dramatically reduce the storage and computation requirements in reconstruction by exploiting redundancy in the transformed matrices.

Localization of an absorbing inhomogeneity in a scattering medium in a statistical framework

An approach for the fast localization and detection of an absorbing inhomogeneity in a tissuelike scattering medium is presented. The probability of detection as a function of the size, location, and absorptive properties of the inhomogeneity is investigated. The detection sensitivity in relation to the source and detector location serves as a basis for instrument design.