Singanallur Venkatakrishnan

Plug-and-Play priors for model based reconstruction

Model-based reconstruction is a powerful framework for solving a variety of inverse problems in imaging. In recent years, enormous progress has been made in the problem of denoising, a special case of an inverse problem where the forward model is an identity operator. Similarly, great progress has been made in improving model-based inversion when the forward model corresponds to complex physical measurements in applications such as X-ray CT, electron-microscopy, MRI, and ultrasound, to name just a few. However, combining state-of-the-art denoising algorithms (i.e., prior models) with state-of-the-art inversion methods (i.e., forward models) has been a challenge for many reasons. In this paper, we propose a flexible framework that allows state-of-the-art forward models of imaging systems to be matched with state-of-the-art priors or denoising models. This framework, which we term as Plug-and-Play priors, has the advantage that it dramatically simplifies software integration, and moreover, it allows state-of-the-art denoising methods that have no known formulation as an optimization problem to be used. We demonstrate with some simple examples how Plug-and-Play priors can be used to mix and match a wide variety of existing denoising models with a tomographic forward model, thus greatly expanding the range of possible problem solutions.

A Model Based Iterative Reconstruction Algorithm For High Angle Annular Dark Field-Scanning Transmission Electron Microscope (HAADF-STEM) Tomography

High angle annular dark field (HAADF)-scanning transmission electron microscope (STEM) data is increasingly being used in the physical sciences to research materials in 3D because it reduces the effects of Bragg diffraction seen in bright field TEM data. Typically, tomographic reconstructions are performed by directly applying either filtered back projection (FBP) or the simultaneous iterative reconstruction technique (SIRT) to the data. Since HAADF-STEM tomography is a limited angle tomography modality with low signal to noise ratio, these methods can result in significant artifacts in the reconstructed volume. In this paper, we develop a model based iterative reconstruction algorithm for HAADF-STEM tomography. We combine a model for image formation in HAADF-STEM tomography along with a prior model to formulate the tomographic reconstruction as a maximum a posteriori probability (MAP) estimation problem. Our formulation also accounts for certain missing measurements by treating them as nuisance parameters in the MAP estimation framework. We adapt the iterative coordinate descent algorithm to develop an efficient method to minimize the corresponding MAP cost function. Reconstructions of simulated as well as experimental data sets show results that are superior to FBP and SIRT reconstructions, significantly suppressing artifacts and enhancing contrast.

TIMBIR: A Method for Time-Space Reconstruction From Interlaced Views

Synchrotron X-ray computed tomography (SXCT) is increasingly being used for 3-D imaging of material samples at micron and finer scales. The success of these techniques has increased interest in 4-D reconstruction methods that can image a sample in both space and time. However, the temporal resolution of widely used 4-D reconstruction methods is severely limited by the need to acquire a very large number of views for each reconstructed 3-D volume. Consequently, the temporal resolution of current methods is insufficient to observe important physical phenomena. Furthermore, measurement nonidealities also tend to introduce ring and streak artifacts into the 4-D reconstructions. In this paper, we present a time-interlaced model-based iterative reconstruction (TIMBIR) method, which is a synergistic combination of two innovations. The first innovation, interlaced view sampling, is a novel method of data acquisition, which distributes the view angles more evenly in time. The second innovation is a 4-D model-based iterative reconstruction algorithm (MBIR), which can produce time-resolved volumetric reconstruction of the sample from the interlaced views. In addition to modeling both the sensor noise statistics and the 4-D object, the MBIR algorithm also reduces ring and streak artifacts by more accurately modeling the measurement nonidealities. We present reconstructions of both simulated and real X-ray synchrotron data, which indicate that TIMBIR can improve temporal resolution by an order of magnitude relative to existing approaches.

Plug-and-Play Priors for Bright Field Electron Tomography and Sparse Interpolation

Many material and biological samples in scientific imaging are characterized by nonlocal repeating structures. These are studied using scanning electron microscopy and electron tomography. Sparse sampling of individual pixels in a two-dimensional image acquisition geometry, or sparse sampling of projection images with large tilt increments in a tomography experiment, can enable high speed data acquisition and minimize sample damage caused by the electron beam. In this paper, we present an algorithm for electron tomographic reconstruction and sparse image interpolation that exploits the nonlocal redundancy in images. We adapt a framework, termed plug-and-play priors, to solve these imaging problems in a regularized inversion setting. The power of the plug-and-play approach is that it allows a wide array of modern denoising algorithms to be used as a “prior model” for tomography and image interpolation. We also present sufficient mathematical conditions that ensure convergence of the plug-and-play approach, and we use these insights to design a new nonlocal means denoising algorithm. Finally, we demonstrate that the algorithm produces higher quality reconstructions on both simulated and real electron microscope data, along with improved convergence properties compared to other methods.

Model-Based Iterative Reconstruction for Bright-Field Electron Tomography

Bright-Field (BF) electron tomography (ET) has been widely used in the life sciences for 3-D imaging of biological specimens. However, while BF-ET is popular in the life sciences, 3-D BF-ET imaging has been avoided in the physical sciences due to measurement anomalies from crystalline samples caused by dynamical diffraction effects such as Bragg scatter. In practice, these measurement anomalies cause undesirable artifacts in 3-D reconstructions computed using filtered back-projection (FBP). Alternatively, model-based iterative reconstruction (MBIR) is a powerful framework for tomographic reconstruction that combines a forward model for the measurement system and a prior model for the object to obtain reconstructions by minimizing a single cost function. In this paper, we present an MBIR algorithm for BF-ET reconstruction from crystalline materials that can account for the presence of anomalous measurements. We propose a new forward model for the acquisition system which accounts for the presence of anomalous measurements and combine it with a prior model for the object to obtain the MBIR cost function. We then propose a fast algorithm based on majorization-minimization to find a minimum of the corresponding cost function. Results on simulated as well as real data show that our method can dramatically improve reconstruction quality as compared to FBP and conventional MBIR without anomaly modeling.

Model based iterative reconstruction for Bright Field electron tomography

Bright Field (BF) electron tomography (ET) has been widely used in the life sciences to characterize biological specimens in 3D. While BF-ET is the dominant modality in the life sciences it has been generally avoided in the physical sciences due to anomalous measurements in the data due to a phenomenon called “Bragg scatter” - visible when crystalline samples are imaged. These measurements cause undesirable artifacts in the reconstruction when the typical algorithms such as Filtered Back Projection (FBP) and Simultaneous Iterative Reconstruction Technique (SIRT) are applied to the data. Model based iterative reconstruction (MBIR) provides a powerful framework for tomographic reconstruction that incorporates a model for data acquisition, noise in the measurement and a model for the object to obtain reconstructions that are qualitatively superior and quantitatively accurate. In this paper we present a novel MBIR algorithm for BF-ET which accounts for the presence of anomalous measurements from Bragg scatter in the data during the iterative reconstruction. Our method accounts for the anomalies by formulating the reconstruction as minimizing a cost function which rejects measurements that deviate significantly from the typical Beer’s law model widely assumed for BF-ET. Results on simulated as well as real data show that our method can dramatically improve the reconstructions compared to FBP and MBIR without anomaly rejection, suppressing the artifacts due to the Bragg anomalies.

Model-based iterative reconstruction for synchrotron X-ray tomography

Synchrotron based X-ray tomography is widely used for three dimensional imaging of materials at the micron scale. Tomographic data collected from a synchrotron is often affected by non-idealities in the measurement system and sudden “blinding” of detector pixels during the acquisition. Typically, reconstructions are done using analytical reconstruction techniques combined with pre/post-processing steps to correct for the non-idealities, resulting in loss of detail while still producing noisy reconstructions with some artifacts. In this paper, we present a model-based iterative reconstruction (MBIR) algorithm for synchrotron X-ray tomography that can automatically handle the non-idealities as a part of the reconstruction. First, we develop a forward model that accounts for the non-idealities in the measurement system and for the occurrence of outliers in the measurement. Next, we combine the forward model with a prior model of the object to formulate the MBIR cost function and propose an algorithm to minimize the cost. Results on a real data set show that the MBIR reconstructions are superior to the analytical reconstructions effectively suppressing noise as well as other artifacts.

Bayesian tomographic reconstruction for high angle annular dark field (HAADF) scanning transmission electron microscopy (STEM)

HAADF-STEM data is increasingly being used in the physical sciences to study materials in 3D because it is free from the diffraction effects seen in Bright Field STEM data and satisfies the projection requirement for tomography. Typically, reconstruction is performed using Filtered Back Projection (FBP) or the SIRT algorithm. In this paper, we develop a Bayesian reconstruction algorithm for HAADF-STEM tomography which models the image formation, the noise characteristics of the measurement, and the inherent smoothness in the object. Reconstructions of polystyrene functionalized Titanium dioxide nano particle assemblies show results that are qualitatively superior to FBP and SIRT reconstructions, significantly suppressing artifacts and enhancing contrast.

Advanced prior modeling for 3D bright field electron tomography

Many important imaging problems in material science involve reconstruction of images containing repetitive non-local structures. Model-based iterative reconstruction (MBIR) could in principle exploit such redundancies through the selection of a log prior probability term. However, in practice, determining such a log prior term that accounts for the similarity between distant structures in the image is quite challenging. Much progress has been made in the development of denoising algorithms like non-local means and BM3D, and these are known to successfully capture non-local redundancies in images. But the fact that these denoising operations are not explicitly formulated as cost functions makes it unclear as to how to incorporate them in the MBIR framework. In this paper, we formulate a solution to bright field electron tomography by augmenting the existing bright field MBIR method to incorporate any non-local denoising operator as a prior model. We accomplish this using a framework we call plug-and-play priors that decouples the log likelihood and the log prior probability terms in the MBIR cost function. We specifically use 3D non-local means (NLM) as the prior model in the plug-and-play framework, and showcase high quality tomographic reconstructions of a simulated aluminum spheres dataset, and two real datasets of aluminum spheres and ferritin structures. We observe that streak and smear artifacts are visibly suppressed, and that edges are preserved. Also, we report lower RMSE values compared to the conventional MBIR reconstruction using qGGMRF as the prior model.

A multi‐slice simulation algorithm for grazing‐incidence small‐angle X‐ray scattering

Grazing-incidence small-angle X-ray scattering (GISAXS) is an important technique in the characterization of samples at the nanometre scale. A key aspect of GISAXS data analysis is the accurate simulation of samples to match the measurement. The distorted-wave Born approximation (DWBA) is a widely used model for the simulation of GISAXS patterns. For certain classes of sample such as nanostructures embedded in thin films, where the electric field intensity variation is significant relative to the size of the structures, a multi-slice DWBA theory is more accurate than the conventional DWBA method. However, simulating complex structures in the multi-slice setting is challenging and the algorithms typically used are designed on a case-by-case basis depending on the structure to be simulated. In this paper, an accurate algorithm for GISAXS simulations based on the multi-slice DWBA theory is presented. In particular, fundamental properties of the Fourier transform have been utilized to develop an algorithm that accurately computes the average refractive index profile as a function of depth and the Fourier transform of the portion of the sample within a given slice, which are key quantities required for the multi-slice DWBA simulation. The results from this method are compared with the traditionally used approximations, demonstrating that the proposed algorithm can produce more accurate results. Furthermore, this algorithm is general with respect to the sample structure, and does not require any sample-specific approximations to perform the simulations.