Hsiou-Yuan Liu

3D imaging in volumetric scattering media using phase-space measurements

We demonstrate the use of phase-space imaging for 3D localization of multiple point sources inside scattering material. The effect of scattering is to spread angular (spatial frequency) information, which can be measured by phase space imaging. We derive a multi-slice forward model for homogenous volumetric scattering, then develop a reconstruction algorithm that exploits sparsity in order to further constrain the problem. By using 4D measurements for 3D reconstruction, the dimensionality mismatch provides significant robustness to multiple scattering, with either static or dynamic diffusers. Experimentally, our high-resolution 4D phase-space data is collected by a spectrogram setup, with results successfully recovering the 3D positions of multiple LEDs embedded in turbid scattering media.

SEAGLE: Sparsity-Driven Image Reconstruction Under Multiple Scattering

Multiple scattering of an electromagnetic wave as it passes through an object is a fundamental problem that limits the performance of current imaging systems. In this paper, we describe a new technique—called Series Expansion with Accelerated Gradient Descent on the Lippmann–Schwinger Equation—for robust imaging under multiple scattering based on a combination of an iterative forward model and a total variation regularizer. The proposed method can account for multiple scattering, which makes it advantageous in applications where single scattering approximations are inaccurate. Specifically, the method relies on a series expansion of the scattered wave with an accelerated-gradient method. This expansion guarantees the convergence of the forward model even for strongly scattering objects. One of our key insights is that it is possible to obtain an explicit formula for computing the gradient of an iterative forward model with respect to the unknown object, thus enabling fast image reconstruction with the state-of-the-art fast iterative shrinkage/thresholding algorithm. The proposed method is validated on diffraction tomography, where complex electric field is captured at different illumination angles.

Compressive imaging with iterative forward models

We propose a new compressive imaging method for reconstructing 2D or 3D objects from their scattered wave-field measurements. Our method relies on a novel, nonlinear measurement model that can account for the multiple scattering phenomenon, which makes the method preferable in applications where linear measurement models are inaccurate. We construct the measurement model by expanding the scattered wave-field with an accelerated-gradient method, which is guaranteed to converge and is suitable for large-scale problems. We provide explicit formulas for computing the gradient of our measurement model with respect to the unknown image, which enables image formation with a sparsity-driven numerical optimization algorithm. We validate the method both analytically and with numerical simulations.

4D phase-space multiplexing for fluorescent microscopy

Phase-space measurements enable characterization of second-order spatial coherence properties and can be used for digital aberration removal or 3D position reconstruction. Previous methods use a scanning aperture to measure the phase space spectrogram, which is slow and light inefficient, while also attenuating information about higher-order correlations. We demonstrate a significant improvement of speed and light throughput by incorporating multiplexing techniques into our phase-space imaging system. The scheme implements 2D coded aperture patterning in the Fourier (pupil) plane of a microscope using a Spatial Light Modulator (SLM), while capturing multiple intensity images in real space. We compare various multiplexing schemes to scanning apertures and show that our phase-space reconstructions are accurate for experimental data with biological samples containing many 3D fluorophores.

Multiplexed phase-space imaging for 3D fluorescence microscopy

Optical phase-space functions describe spatial and angular information simultaneously; examples of optical phase-space functions include light fields in ray optics and Wigner functions in wave optics. Measurement of phase-space enables digital refocusing, aberration removal and 3D reconstruction. High-resolution capture of 4D phase-space datasets is, however, challenging. Previous scanning approaches are slow, light inefficient and do not achieve diffraction-limited resolution. Here, we propose a multiplexed method that solves these problems. We use a spatial light modulator (SLM) in the pupil plane of a microscope in order to sequentially pattern multiplexed coded apertures while capturing images in real space. Then, we reconstruct the 3D fluorescence distribution of our sample by solving an inverse problem via regularized least squares with a proximal accelerated gradient descent solver. We experimentally reconstruct a 101 Megavoxel 3D volume (1010×510×500µm with NA 0.4), demonstrating improved acquisition time, light throughput and resolution compared to scanning aperture methods. Our flexible patterning scheme further allows sparsity in the sample to be exploited for reduced data capture.

3D fluorescence microscopy with multi-shot coded aperture measurements (Conference Presentation)

Coded aperture imaging for 3D reconstructions have been applied in both photography and microscopy to successfully recover depth information from images captured with a patterned pupil plane. Generally, single-shot coded aperture microscopy methods require the 3D sample to be extremely sparse and with no overlapping features. Here, we extend coded aperture microscopy methods to multi-shot 3D fluorescent imaging with wave-optical forward models, enabling thicker and denser 3D samples to be faithfully reconstructed with efficient data capture via compressed sensing and advanced inverse algorithms.. We demonstrate successful reconstruction of a 500um thick brine shrimp sample. We further study the minimum required number of codes to capture full fluorescent information under wide-field illumination The experiments are carried out with a 4f system and a spatial light modulator in the Fourier (aperture) domain to display the codes.

Computational phase space measurements using multiplexed coded apertures (Conference Presentation)

Phase-space refers to simultaneous space-frequency information (e.g. Wigner functions, light fields), which is directly related to spatial coherence properties (e.g. Mutual Intensity). We introduce a binary pupil masking technique that allows us to computationally reconstruct the phase space distribution of optical beams from a series of images. Previous work has shown phase space to be useful for 3D imaging and localization in a multiple scattering environment. Binary masks are easy to implement compared to gray masks or phase masks and the proposed scheme requires no interferometry. After designing the masks with nonredundant arrays, we measure an intensity image for each aperture mask and reconstruct the phase space through an auxiliary coherence function. We demonstrate experimentally the reconstruction of the phase space of a collection of 3D incoherent sources.

SEAGLE: Robust Computational Imaging under Multiple Scattering

Multiple scattering of light as it passes through an object is a fundamental problem limiting the performance of imaging systems. We describe a new technique for robust imaging under multiple scattering based on a nonlinear scattering model and sparse-regularization.

Fast algorithm for 3D localization through scattering media: forward model and physics

We propose a physics model for volumetric scattering that leads to a fast algorithm for localizing point emitters in 3D. The input is a high-resolution 4D phase-space measurement. Our algorithm has small complexity for solving atomic norm based inverse problems with Terabyte-sized 4D datasets, enabling us to run our solver in almost real-time.

High-speed and high-resolution phase-space imaging with digital micromirror devices

We describe an imaging system for capturing 4D phase-space distributions of partially coherent and scattered light. The capture comprises a fast aperture coding in the Fourier space of the object, which is done by a digital micromirror device (DMD).