Christopher A. Metzler

From Denoising to Compressed Sensing

A denoising algorithm seeks to remove noise, errors, or perturbations from a signal. Extensive research has been devoted to this arena over the last several decades, and as a result, todays denoisers can effectively remove large amounts of additive white Gaussian noise. A compressed sensing (CS) reconstruction algorithm seeks to recover a structured signal acquired using a small number of randomized measurements. Typical CS reconstruction algorithms can be cast as iteratively estimating a signal from a perturbed observation. This paper answers a natural question: How can one effectively employ a generic denoiser in a CS reconstruction algorithm? In response, we develop an extension of the approximate message passing (AMP) framework, called denoising-based AMP (D-AMP), that can integrate a wide class of denoisers within its iterations. We demonstrate that, when used with a high-performance denoiser for natural images, D-AMP offers the state-of-the-art CS recovery performance while operating tens of times faster than competing methods. We explain the exceptional performance of D-AMP by analyzing some of its theoretical features. A key element in D-AMP is the use of an appropriate Onsager correction term in its iterations, which coerces the signal perturbation at each iteration to be very close to the white Gaussian noise that denoisers are typically designed to remove.

BM3D-AMP: A new image recovery algorithm based on BM3D denoising

A denoising algorithm seeks to remove perturbations or errors from a signal. The last three decades have seen extensive research devoted to this arena, and as a result, todays denoisers are highly optimized algorithms that effectively remove large amounts of additive white Gaussian noise. A compressive sensing (CS) reconstruction algorithm seeks to recover a structured signal acquired from a small number of randomized measurements. Typical CS reconstruction algorithms can be cast as iteratively estimating a signal from a perturbed observation. This paper answers a natural question: How can one effectively employ a generic denoiser in a CS reconstruction algorithm? In response, we develop a denoising-based approximate message passing (D-AMP) algorithm that is capable of high-performance reconstruction. We demonstrate using the high performance BM3D denoiser that D-AMP offers state-of-the-art CS recovery performance for natural images (on average 9dB better than sparsity-based algorithms), while operating tens of times faster than the only competitive method. A critical insight in our approach is the use of an appropriate Onsager correction term in the D-AMP iterations, which coerces the signal perturbation at each iteration to be very close to the white Gaussian noise that denoisers are typically designed to remove. On the analytical side, we develop a new state evolution framework for deterministic signals that accurately predicts the performance of D-AMP and enables us to derive several useful theoretical features.

Coherent inverse scattering via transmission matrices: Efficient phase retrieval algorithms and a public dataset

A transmission matrix describes the input-output relationship of a complex wavefront as it passes through/reflects off a multiple-scattering medium, such as frosted glass or a painted wall. Knowing a mediums transmission matrix enables one to image through the medium, send signals through the medium, or even use the medium as a lens. The double phase retrieval method is a recently proposed technique to learn a mediums transmission matrix that avoids difficult-to-capture interferometric measurements. Unfortunately, to perform high resolution imaging, existing double phase retrieval methods require (1) a large number of measurements and (2) an unreasonable amount of computation. In this work we focus on the latter of these two problems and reduce computation times with two distinct methods: First, we develop a new phase retrieval algorithm that is significantly faster than existing methods, especially when used with an amplitude-only spatial light modulator (SLM). Second, we calibrate the system using a phase-only SLM, rather than an amplitude-only SLM which was used in previous double phase retrieval experiments. This seemingly trivial change enables us to use a far faster class of phase retrieval algorithms. As a result of these advances, we achieve a 100x reduction in computation times, thereby allowing us to image through scattering media at state-of-the-art resolutions. In addition to these advances, we also release the first publicly available transmission matrix dataset. This contribution will enable phase retrieval researchers to apply their algorithms to real data. Of particular interest to this community, our measurement vectors are naturally i.i.d. subgaussian, i.e., no coded diffraction pattern is required.

Optimal recovery from compressive measurements via denoising-based approximate message passing

Recently progress has been made in compressive sensing by replacing simplistic sparsity models with more powerful denoisers. In this paper, we develop a framework to predict the performance of denoising-based signal recovery algorithms based on a new deterministic state evolution formalism for approximate message passing. We compare our deterministic state evolution against its more classical Bayesian counterpart. We demonstrate that, while the two state evolutions are very similar, the deterministic framework is far more flexible. We apply the deterministic state evolution to explore the optimality of denoising-based approximate message passing (D-AMP). We prove that, while D-AMP is suboptimal for certain classes of signals, no algorithm can uniformly outperform it.

Dynamic model generation for application of compressed sensing to cryo-electron tomography reconstruction

Cryo-electron tomography (cryo-ET), which produces three dimensional images at molecular resolution, is one of many applications that requires image reconstruction from projection measurements acquired with irregular measurement geometry. Although Fourier transform based reconstruction methods have been widely and successfully used in medical imaging for over 25 years, assumptions of regular measurement geometry and a band limited source cause direction sensitive artifacts when applied to cryo-ET. Iterative space domain methods such as compressed sensing could be applied to this severely underdetermined system with a limited range of projection angles and projection length, but progress has been hindered by the computational and storage requirements of the very large projection matrix of observation partials. In this paper we derive a method of dynamically computing the elements of the projection matrix accurately for continuous basis functions of limited extent with arbitrary beam width. Storage requirements are reduced by a factor of order 107 and there is no access overhead. This approach for limited angle and limited view measurement geometries is posed to enable dramatically improved reconstruction performance and is easily adapted to parallel computing architectures.

An Expectation-Maximization Approach to Tuning Generalized Vector Approximate Message Passing

Generalized Vector Approximate Message Passing (GVAMP) is an efficient iterative algorithm for approximately minimum-mean-squared-error estimation of a random vector \(\mathbf {x}\sim p_{\mathbf {x}}(\mathbf {x})\) from generalized linear measurements, i.e., measurements of the form \(\mathbf {y}=Q(\mathbf {z})\) where \(\mathbf {z}=\mathbf {Ax}\) with known \(\mathbf {A}\), and \(Q(\cdot )\) is a noisy, potentially nonlinear, componentwise function. Problems of this form show up in numerous applications, including robust regression, binary classification, quantized compressive sensing, and phase retrieval. In some cases, the prior \(p_{\mathbf {x}}\) and/or channel \(Q(\cdot )\) depend on unknown deterministic parameters \(\varvec{\theta }\), which prevents a direct application of GVAMP. In this paper we propose a way to combine expectation maximization (EM) with GVAMP to jointly estimate \(\mathbf {x}\) and \(\varvec{\theta }\). We then demonstrate how EM-GVAMP can solve the phase retrieval problem with unknown measurement-noise variance.

BM3D-PRGAMP: Compressive phase retrieval based on BM3D denoising

The explosion of computational imaging has seen the frontier of image processing move past linear problems, like denoising and deblurring, and towards non-linear problems such as phase retrieval. There has a been a corresponding research thrust into non-linear image recovery algorithms, but in many ways this research is stuck where linear problem research was twenty years ago: Models, if used at all, are simple designs like sparsity or smoothness. In this paper we use denoisers to impose elaborate and accurate models in order to perform inference on generalized linear systems. More specifically, we use the state-of-the-art BM3D denoiser within the Generalized Approximate Message Passing (GAMP) framework to solve compressive phase retrieval. Our method demonstrates recovery performance equivalent to existing techniques using fewer than half as many measurements. This dramatic improvement in compressive phase retrieval performance opens the door for a whole new class of imaging systems.

Iterative reconstruction from limited angle, limited view projections for cryo-electron tomography

Improved accurate measurement models and improved iterative reconstruction algorithms would benefit cryo-electron tomography (cryo-ET) performance. Filtered back- projection and related algorithms, successful in CT and MRI, assume a measurement model which is not well matched to the limited range of projection angles, large angular increments, and incomplete projections in cryo-ET. Iterative methods, such as compressed sensing (CS) can include irregular measurement models and spatial extent constraints, and have great potential for solution of severely under-determined systems. This paper uses source models with square and pyramidal basis functions and variable finite width aperture measurement to compare space domain and frequency domain CS reconstruction approaches in the cryo-ET context.