Leonardo R. Bachega

The Sparse Matrix Transform for Covariance Estimation and Analysis of High Dimensional Signals

Covariance estimation for high dimensional signals is a classically difficult problem in statistical signal analysis and machine learning. In this paper, we propose a maximum likelihood (ML) approach to covariance estimation, which employs a novel non-linear sparsity constraint. More specifically, the covariance is constrained to have an eigen decomposition which can be represented as a sparse matrix transform (SMT). The SMT is formed by a product of pairwise coordinate rotations known as Givens rotations. Using this framework, the covariance can be efficiently estimated using greedy optimization of the log-likelihood function, and the number of Givens rotations can be efficiently computed using a cross-validation procedure. The resulting estimator is generally positive definite and well-conditioned, even when the sample size is limited. Experiments on a combination of simulated data, standard hyperspectral data, and face image sets show that the SMT-based covariance estimates are consistently more accurate than both traditional shrinkage estimates and recently proposed graphical lasso estimates for a variety of different classes and sample sizes. An important property of the new covariance estimate is that it naturally yields a fast implementation of the estimated eigen-transformation using the SMT representation. In fact, the SMT can be viewed as a generalization of the classical fast Fourier transform (FFT) in that it uses “butterflies” to represent an orthonormal transform. However, unlike the FFT, the SMT can be used for fast eigen-signal analysis of general non-stationary signals.

Sparse Matrix Transform for Hyperspectral Image Processing

A variety of problems in remote sensing require that a covariance matrix be accurately estimated, often from a limited number of data samples. We investigate the utility of several variants of a recently introduced covariance estimator-the sparse matrix transform (SMT), a shrinkage-enhanced SMT, and a graph-constrained SMT-in the context of several of these problems. In addition to two more generic measures of quality based on likelihood and the Frobenius norm, we specifically consider weak signal detection, dimension reduction, anomaly detection, and anomalous change detection. The estimators are applied to several hyperspectral data sets, including some randomly rotated data, to elucidate the kinds of problems and the kinds of data for which SMT is well or poorly suited. The SMT is based on the product of K pairwise coordinate (Givens) rotations, and we also introduce and compare two novel approaches for estimating the most effective choice for K .

Evaluating and improving local hyperspectral anomaly detectors

This paper addresses two issues related to the detection of hyperspectral anomalies. The first issue is the evaluation of anomaly detector performance even when labeled data is not available. The second issue is the estimation of the covariance structure of the data in local detection methods, such as the RX detector, when the number of available training pixels n is not much larger than (and may even be smaller than) the data dimensionality p.

Fast signal analysis and decomposition on graphs using the Sparse Matrix Transform

Recently, the Sparse Matrix Transform (SMT) has been proposed as a tool for estimating the eigen-decomposition of high dimensional data vectors [1]. The SMT approach has two major advantages: First it can improve the accuracy of the eigendecomposition, particularlywhen the number of observations, n, is less the the vector dimension, p. Second, the resulting SMT eigen-decomposition is very fast to apply, i.e. O(p). In this paper, we present an SMT eigen-decomposition method suited for application to signals that live on graphs. This new SMT eigen-decomposition method has two major advantages over the more generic method presented in [1]. First, the resulting SMT can be more accurately estimated due to the graphical constraint. Second, the computation required to design the SMT from training data is dramatically reduced from an average observed complexity of p3 to p log p.

GTKDynamo: a PyMOL plug-in for QC/MM hybrid potential simulations

Hybrid quantum chemical/molecular mechanical (QCMM) potentials are very powerful tools for molecular simulation. They are especially useful for studying processes in condensed phase systems, such as chemical reactions that involve a relatively localized change in electronic structure and where the surrounding environment contributes to these changes but can be represented with more computationally efficient functional forms. Despite their utility, however, these potentials are not always straightforward to apply since the extent of significant electronic structure changes occurring in the condensed phase process may not be intuitively obvious. To facilitate their use, we have developed an open‐source graphical plug‐in, GTKDynamo that links the PyMOL visualization program and the pDynamo QC/MM simulation library. This article describes the implementation of GTKDynamo and its capabilities and illustrates its application to QC/MM simulations.

Hypothesis testing in high-dimensional space with the Sparse Matrix Transform

This paper discusses the use of the Sparse Matrix Transform (SMT) to model the covariance structure of high-dimensional data in the likelihood ratio test used for hypothesis testing. The SMT has been shown to produce more accurate estimates of covariance matrices when the number of training samples n is much less than the number of dimensions p of the data. Several experiments with face recognition and hyperspectral images show that SMT-based hypothesis testing can be superior to other methods in at least two general aspects: First, the SMT-based method is more robust to the size of the training set, remaining accurate even when only a few training samples are available; Second, the total computation required to apply the method is very low, making it attractive for use in low-power devices, or in applications requiring fast computation.

Classification of high-dimensional data using the Sparse Matrix Transform

In this paper, we develop a classification method for high-dimensional data based on the Sparse Matrix Transform (SMT). The recently proposed SMT has been shown to produce more accurate estimates of covariance matrices when the number of training samples n is much less than the number of dimensions p of the data. Here we introduce a classifier that uses the SMT to model the covariance structure of the data. Experiments in face recognition using the FERET face database show that our method is superior to a conceptually very similar and low-dimensional method in at least two key aspects: First, the SMT classifier is more robust to the size of the training set, remaining accurate even when only a few training samples are available; Second, the total computation required to apply the SMT classifier to high-dimensional data is very low, making this method attractive for use in low-power and mobile devices, or in application settings requiring fast computation

Communication efficient signal detection in correlated clutter for wireless sensor networks

We study a problem of detecting deterministic signals buried in correlated clutter using wireless sensor networks. We are specifically interested in developing a distributed algorithm over the network to detect the presence of a deterministic signal while keeping low communication delay and energy associated with the distributed computation. In this paper, we deploy a distributed version of the Sparse Matrix Transform (SMT) that decorrelates a signal measured by a number of sensors in order to compute a matched filter. The matched filter represents the sum of the Log-Likelihood Ratios over all the sensors of the two hypotheses corresponding to whether a deterministic signal is present or not. We show through numerical simulations that our algorithm is very efficient in terms of communication energy and delay while sustaining a high Signal-to-Clutter Ratio.

Ant: A Debugging Framework for MPI Parallel Programs

This paper describes Ant, a debugging framework targeting MPI parallel programs. The Ant framework statically analyzes programs, marking code regions as being executed by all processes or executed by only some of the processes. The analyzed program is then instrumented with calls to an invariant violation monitoring and detection library. The analysis allows regions to be instrumented based on whether all, or less than all, processes execute the region. Ant’s instrumentation strategy allows sampled monitoring across processes in regions executed by all processes. We present a case study using Ant with C-DIDUCE (a variant of DIDUCE for C) to find violations of value invariants in parallel C/MPI programs. Ant’s instrumentation strategy reduces the overhead of monitoring by over 14 times with less impact on accuracy than a scheme that simply distributes monitoring over all processes executing the program.

Distributed signal decorrelation in wireless sensor networks using the sparse matrix transform

In this paper, we propose the vector SMT, a new decorrelating transform suitable for performing distributed anomaly detection in wireless sensor networks (WSN). Here, we assume that each sensor in the network performs vector measurements, instead of a scalar ones. The proposed transform decorrelates a sequence of pairs of vector sensor measurements, until the vectors from all sensors are completely decorrelated. We perform simulations with a network of cameras, where each camera records an image of the monitored environment from its particular viewpoint. Results show that the proposed transform effectively decorrelates image measurements from the multiple cameras in the network. Because it enables joint processing of the multiple images, our method provides significant improvements to anomaly detection accuracy when compared to the baseline case when we process the images independently.