Jingshan Zhong

Experimental robustness of Fourier ptychography phase retrieval algorithms

Fourier ptychography is a new computational microscopy technique that provides gigapixel-scale intensity and phase images with both wide field-of-view and high resolution. By capturing a stack of low-resolution images under different illumination angles, an inverse algorithm can be used to computationally reconstruct the high-resolution complex field. Here, we compare and classify multiple proposed inverse algorithms in terms of experimental robustness. We find that the main sources of error are noise, aberrations and mis-calibration (i.e. model mis-match). Using simulations and experiments, we demonstrate that the choice of cost function plays a critical role, with amplitude-based cost functions performing better than intensity-based ones. The reason for this is that Fourier ptychography datasets consist of images from both brightfield and darkfield illumination, representing a large range of measured intensities. Both noise (e.g. Poisson noise) and model mis-match errors are shown to scale with intensity. Hence, algorithms that use an appropriate cost function will be more tolerant to both noise and model mis-match. Given these insights, we propose a global Newtons method algorithm which is robust and accurate. Finally, we discuss the impact of procedures for algorithmic correction of aberrations and mis-calibration.

Computational illumination for high-speed in vitro Fourier ptychographic microscopy

We demonstrate a new computational illumination technique that achieves large space-bandwidth-time product, for quantitative phase imaging of unstained live samples in vitro. Microscope lenses can have either large field of view (FOV) or high resolution, not both. Fourier ptychographic microscopy (FPM) is a new computational imaging technique that circumvents this limit by fusing information from multiple images taken with different illumination angles. The result is a gigapixel-scale image having both wide FOV and high resolution, i.e. large space-bandwidth product (SBP). FPM has enormous potential for revolutionizing microscopy and has already found application in digital pathology. However, it suffers from long acquisition times (on the order of minutes), limiting throughput. Faster capture times would not only improve imaging speed, but also allow studies of live samples, where motion artifacts degrade results. In contrast to fixed (e.g. pathology) slides, live samples are continuously evolving at various spatial and temporal scales. Here, we present a new source coding scheme, along with real-time hardware control, to achieve 0.8 NA resolution across a 4x FOV with sub-second capture times. We propose an improved algorithm and new initialization scheme, which allow robust phase reconstruction over long time-lapse experiments. We present the first FPM results for both growing and confluent in vitro cell cultures, capturing videos of subcellular dynamical phenomena in popular cell lines undergoing division and migration. Our method opens up FPM to applications with live samples, for observing rare events in both space and time.

3D imaging in volumetric scattering media using phase-space measurements

We demonstrate the use of phase-space imaging for 3D localization of multiple point sources inside scattering material. The effect of scattering is to spread angular (spatial frequency) information, which can be measured by phase space imaging. We derive a multi-slice forward model for homogenous volumetric scattering, then develop a reconstruction algorithm that exploits sparsity in order to further constrain the problem. By using 4D measurements for 3D reconstruction, the dimensionality mismatch provides significant robustness to multiple scattering, with either static or dynamic diffusers. Experimentally, our high-resolution 4D phase-space data is collected by a spectrogram setup, with results successfully recovering the 3D positions of multiple LEDs embedded in turbid scattering media.

Nonlinear Optimization Algorithm for Partially Coherent Phase Retrieval and Source Recovery

We propose a new algorithm for recovering both complex field (phase and amplitude) and source distribution (illumination spatial coherence) from a stack of intensity images captured through focus. The joint recovery is formulated as a nonlinear least-square-error optimization problem, which is solved iteratively by a modified Gauss-Newton method. We derive the gradient and Hessian of the cost function and show that our second-order optimization approach outperforms previously proposed phase retrieval algorithms, for datasets taken with both coherent and partially coherent illumination. The method is validated experimentally in a commercial microscope with both Köhler illumination and a programmable light-emitting diode dome.

4D phase-space multiplexing for fluorescent microscopy

Phase-space measurements enable characterization of second-order spatial coherence properties and can be used for digital aberration removal or 3D position reconstruction. Previous methods use a scanning aperture to measure the phase space spectrogram, which is slow and light inefficient, while also attenuating information about higher-order correlations. We demonstrate a significant improvement of speed and light throughput by incorporating multiplexing techniques into our phase-space imaging system. The scheme implements 2D coded aperture patterning in the Fourier (pupil) plane of a microscope using a Spatial Light Modulator (SLM), while capturing multiple intensity images in real space. We compare various multiplexing schemes to scanning apertures and show that our phase-space reconstructions are accurate for experimental data with biological samples containing many 3D fluorophores.

Multiplexed phase-space imaging for 3D fluorescence microscopy

Optical phase-space functions describe spatial and angular information simultaneously; examples of optical phase-space functions include light fields in ray optics and Wigner functions in wave optics. Measurement of phase-space enables digital refocusing, aberration removal and 3D reconstruction. High-resolution capture of 4D phase-space datasets is, however, challenging. Previous scanning approaches are slow, light inefficient and do not achieve diffraction-limited resolution. Here, we propose a multiplexed method that solves these problems. We use a spatial light modulator (SLM) in the pupil plane of a microscope in order to sequentially pattern multiplexed coded apertures while capturing images in real space. Then, we reconstruct the 3D fluorescence distribution of our sample by solving an inverse problem via regularized least squares with a proximal accelerated gradient descent solver. We experimentally reconstruct a 101 Megavoxel 3D volume (1010×510×500µm with NA 0.4), demonstrating improved acquisition time, light throughput and resolution compared to scanning aperture methods. Our flexible patterning scheme further allows sparsity in the sample to be exploited for reduced data capture.

Nonlinear optimization for partially coherent phase recovery with Abbe’s method

We extend our previous nonlinear optimization method for partially coherent phase retrieval from through-focus intensity images. The forward model is formulated as an Abbe extended source method, where each source point incurs different pupil filtering. The proposed method gives more accurate phase results beyond the coherent diffraction limit.

Large-scale phase retreival for metrology invited talk

This talk will describe two different microscope setups for achieving quantitative phase images in a commercial microscope using simple hardware. First, we discuss methods that use a stack of intensity images captured through focus in order to reconstruct phase, which can be related to surface height maps for applications in surface profiling and metrology. Second, we describe a new LED array microscope that enables phase profiling across a very large field-of-view (2mm) with high resolution (0.7 NA), by patterning the illumination angles with an LED array illuminator and implementing Fourier Ptychography. Both methods are implemented with simple hardware modifications to traditional microscopes and both exhibit stable, accurate and robust quantitative phase inversion algorithms.

Partially coherent phase imaging with source shapes estimation

We propose a method to recover quantitative phase from a stack of defocused intensity images illuminated with partially coherent light from a source of arbitrary shape in Köhler geometry. The algorithm uses a sparse Kalman filtering approach which is fast, accurate, and robust to noise. The proposed method is able to recover not only the phase, but also the source shape, which defines the spatial coherence of the illumination. We validate our algorithm experimentally in a commercial microscope with biological samples.

Experimental robustness of Fourier Ptychographic phase retrieval algorithms

We compare the results from several previously proposed phase retrieval algorithms using experimental Fourier Ptychography datasets and show importance of background subtraction and regularization for robustness with noisy imperfect data.