Vivek K Goyal

Theoretical foundations of transform coding

Discusses various aspects of transform coding, including: source coding, constrained source coding, the standard theoretical model for transform coding, entropy codes, Huffman codes, quantizers, uniform quantization, bit allocation, optimal transforms, transforms visualization, partition cell shapes, autoregressive sources, transform optimization, synthesis transform optimization, orthogonality and independence, and departures form the standard model.

Compressive Sampling and Lossy Compression

Recent results in compressive sampling have shown that sparse signals can be recovered from a small number of random measurements. This property raises the question of whether random measurements can provide an efficient representation of sparse signals in an information-theoretic sense. Through both theoretical and experimental results, we show that encoding a sparse signal through simple scalar quantization of random measurements incurs a significant penalty relative to direct or adaptive encoding of the sparse signal. Information theory provides alternative quantization strategies, but they come at the cost of much greater estimation complexity.

Necessary and Sufficient Conditions for Sparsity Pattern Recovery

The paper considers the problem of detecting the sparsity pattern of a k -sparse vector in \BBR n from m random noisy measurements. A new necessary condition on the number of measurements for asymptotically reliable detection with maximum-likelihood (ML) estimation and Gaussian measurement matrices is derived. This necessary condition for ML detection is compared against a sufficient condition for simple maximum correlation (MC) or thresholding algorithms. The analysis shows that the gap between thresholding and ML can be described by a simple expression in terms of the total signal-to-noise ratio (SNR), with the gap growing with increasing SNR. Thresholding is also compared against the more sophisticated Lasso and orthogonal matching pursuit (OMP) methods. At high SNRs, it is shown that the gap between Lasso and OMP over thresholding is described by the range of powers of the nonzero component values of the unknown signals. Specifically, the key benefit of Lasso and OMP over thresholding is the ability of Lasso and OMP to detect signals with relatively small components.

Filter bank frame expansions with erasures

We study frames for robust transmission over the Internet. In our previous work, we used quantized finite-dimensional frames to achieve resilience to packet losses; here, we allow the input to be a sequence in l/sub 2/(Z) and focus on a filter-bank implementation of the system. We present results in parallel, R/sup N/ or C/sup N/ versus l/sub 2/(Z), and show that uniform tight frames, as well as newly introduced strongly uniform tight frames, provide the best performance.

Asymptotic Analysis of MAP Estimation via the Replica Method and Applications to Compressed Sensing

The replica method is a nonrigorous but well-known technique from statistical physics used in the asymptotic analysis of large, random, nonlinear problems. This paper applies the replica method, under the assumption of replica symmetry, to study estimators that are maximum a posteriori (MAP) under a postulated prior distribution. It is shown that with random linear measurements and Gaussian noise, the replica-symmetric prediction of the asymptotic behavior of the postulated MAP estimate of an -dimensional vector “decouples” as scalar postulated MAP estimators. The result is based on applying a hardening argument to the replica analysis of postulated posterior mean estimators of Tanaka and of Guo and Verdú. The replica-symmetric postulated MAP analysis can be readily applied to many estimators used in compressed sensing, including basis pursuit, least absolute shrinkage and selection operator (LASSO), linear estimation with thresholding, and zero norm-regularized estimation. In the case of LASSO estimation, the scalar estimator reduces to a soft-thresholding operator, and for zero norm-regularized estimation, it reduces to a hard threshold. Among other benefits, the replica method provides a computationally tractable method for precisely predicting various performance metrics including mean-squared error and sparsity pattern recovery probability.

First-Photon Imaging

Computing an Image Firing off a burst of laser pulses and detecting the back-reflected photons is a widely used method for constructing three-dimensional (3D) images of a scene. Kirmani et al. (p. 58, published online 29 November) describe an active imaging method in which pulsed laser light raster scans a scene and a single-photon detector is used to detect the first photon of the back-reflected laser light. Exploiting spatial correlations of photons scattered from different parts of the scene allows computation of a 3D image. Importantly, for biological applications, the technique allows the laser power to be reduced without sacrificing image quality. A computational imaging method based on photon timing enables three-dimensional imaging under low light flux conditions. Imagers that use their own illumination can capture three-dimensional (3D) structure and reflectivity information. With photon-counting detectors, images can be acquired at extremely low photon fluxes. To suppress the Poisson noise inherent in low-flux operation, such imagers typically require hundreds of detected photons per pixel for accurate range and reflectivity determination. We introduce a low-flux imaging technique, called first-photon imaging, which is a computational imager that exploits spatial correlations found in real-world scenes and the physics of low-flux measurements. Our technique recovers 3D structure and reflectivity from the first detected photon at each pixel. We demonstrate simultaneous acquisition of sub–pulse duration range and 4-bit reflectivity information in the presence of high background noise. First-photon imaging may be of considerable value to both microscopy and remote sensing.

Message-Passing De-Quantization With Applications to Compressed Sensing

Estimation of a vector from quantized linear measurements is a common problem for which simple linear techniques are suboptimal-sometimes greatly so. This paper develops message-passing de-quantization (MPDQ) algorithms for minimum mean-squared error estimation of a random vector from quantized linear measurements, notably allowing the linear expansion to be overcomplete or undercomplete and the scalar quantization to be regular or non-regular. The algorithm is based on generalized approximate message passing (GAMP), a recently-developed Gaussian approximation of loopy belief propagation for estimation with linear transforms and nonlinear componentwise-separable output channels. For MPDQ, scalar quantization of measurements is incorporated into the output channel formalism, leading to the first tractable and effective method for high-dimensional estimation problems involving non-regular scalar quantization. The algorithm is computationally simple and can incorporate arbitrary separable priors on the input vector including sparsity-inducing priors that arise in the context of compressed sensing. Moreover, under the assumption of a Gaussian measurement matrix with i.i.d. entries, the asymptotic error performance of MPDQ can be accurately predicted and tracked through a simple set of scalar state evolution equations. We additionally use state evolution to design MSE-optimal scalar quantizers for MPDQ signal reconstruction and empirically demonstrate the superior error performance of the resulting quantizers. In particular, our results show that non-regular quantization can greatly improve rate-distortion performance in some problems with oversampling or with undersampling combined with a sparsity-inducing prior.Estimation of a vector from quantized linear measurements is a common problem for which simple linear techniques are suboptimal—sometimes greatly so. This paper develops generalized approximate message passing (GAMP) alg orithms for minimum mean-squared error estimation of a random vector from quantized linear measurements, notably allowi ng the linear expansion to be overcomplete or undercomplete and th e scalar quantization to be regular or non-regular. GAMP is a recently-developed class of algorithms that uses Gaussianpproximations in belief propagation and allows arbitrary separable input and output channels. Scalar quantization of measurements is incorporated into the output channel formalism, leading to the first tractable and effective method for high-dimensional estimation problems involving non-regular scalar quantization. Non-regular quantization is empirically demonstrated to greatly improve rate–distortion performance in some problems with oversampling or with undersampling combined with a sparsit yinducing prior. Under the assumption of a Gaussian measurement matrix with i.i.d. entries, the asymptotic error performan ce of GAMP can be accurately predicted and tracked through the state evolution formalism. We additionally use state evolution t o design MSE-optimal scalar quantizers for GAMP signal reconstruction and empirically demonstrate the superior error performance of the resulting quantizers.

Optimal quantization of random measurements in compressed sensing

Quantization is an important but often ignored consideration in discussions about compressed sensing. This paper studies the design of quantizers for random measurements of sparse signals that are optimal with respect to mean-squared error of the lasso reconstruction. We utilize recent results in high-resolution functional scalar quantization and homotopy continuation to approximate the optimal quantizer. Experimental results compare this quantizer to other practical designs and show a noticeable improvement in the operational distortion-rate performance.

Fast slice‐selective radio‐frequency excitation pulses for mitigating B +1 inhomogeneity in the human brain at 7 Tesla

A novel radio‐frequency (RF) pulse design algorithm is presented that generates fast slice‐selective excitation pulses that mitigate B  +1 inhomogeneity present in the human brain at high field. The method is provided an estimate of the B  +1 field in an axial slice of the brain and then optimizes the placement of sinc‐like “spokes” in kz via an L1‐norm penalty on candidate (kx, ky) locations; an RF pulse and gradients are then designed based on these weighted points. Mitigation pulses are designed and demonstrated at 7T in a head‐shaped water phantom and the brain; in each case, the pulses mitigate a significantly nonuniform transmit profile and produce nearly uniform flip angles across the field of excitation (FOX). The main contribution of this work, the sparsity‐enforced spoke placement and pulse design algorithm, is derived for conventional single‐channel excitation systems and applied in the brain at 7T, but readily extends to lower field systems, nonbrain applications, and multichannel parallel excitation arrays. Magn Reson Med 59:1355–1364, 2008.

Multiple description vector quantization with a coarse lattice

A multiple description (MD) lattice vector quantization technique for two descriptions was previously introduced in which fine and coarse codebooks are both lattices. The encoding begins with quantization to the nearest point in the fine lattice. This encoding is an inherent optimization for the decoder that receives both descriptions; performance can be improved with little increase in complexity by considering all decoders in the initial encoding step. The altered encoding relies only on the symmetries of the coarse lattice. This allows us to further improve performance without a significant increase in complexity by replacing the fine lattice codebook with a nonlattice codebook that respects many of the symmetries of the coarse lattice. Examples constructed with the two-dimensional (2-D) hexagonal lattice demonstrate large improvement over time sharing between previously known quantizers.