Xing Lin

Spatial-spectral encoded compressive hyperspectral imaging

This paper proposes a novel compressive hyperspectral (HS) imaging approach that allows for high-resolution HS images to be captured in a single image. The proposed architecture comprises three key components: spatial-spectral encoded optical camera design, over-complete HS dictionary learning and sparse-constraint computational reconstruction. Our spatial-spectral encoded sampling scheme provides a higher degree of randomness in the measured projections than previous compressive HS imaging approaches; and a robust nonlinear sparse reconstruction method is employed to recover the HS images from the coded projection with higher performance. To exploit the sparsity constraint on the nature HS images for computational reconstruction, an over-complete HS dictionary is learned to represent the HS images in a sparser way than previous representations. We validate the proposed approach on both synthetic and real captured data, and show successful recovery of HS images for both indoor and outdoor scenes. In addition, we demonstrate other applications for the over-complete HS dictionary and sparse coding techniques, including 3D HS images compression and denoising.

Dual-coded compressive hyperspectral imaging

This Letter presents a new snapshot approach to hyperspectral imaging via dual-optical coding and compressive computational reconstruction. We demonstrate that two high-speed spatial light modulators, located conjugate to the image and spectral plane, respectively, can code the hyperspectral datacube into a single sensor image such that the high-resolution signal can be recovered in postprocessing. We show various applications by designing different optical modulation functions, including programmable spatially varying color filtering, multiplexed hyperspectral imaging, and high-resolution compressive hyperspectral imaging.

Coded focal stack photography

We present coded focal stack photography as a computational photography paradigm that combines a focal sweep and a coded sensor readout with novel computational algorithms. We demonstrate various applications of coded focal stacks, including photography with programmable non-planar focal surfaces and multiplexed focal stack acquisition. By leveraging sparse coding techniques, coded focal stacks can also be used to recover a full-resolution depth and all-in-focus (AIF) image from a single photograph. Coded focal stack photography is a significant step towards a computational camera architecture that facilitates high-resolution post-capture refocusing, flexible depth of field, and 3D imaging.

Camera array based light field microscopy.

This paper proposes a novel approach for high-resolution light field microscopy imaging by using a camera array. In this approach, we apply a two-stage relay system for expanding the aperture plane of the microscope into the size of an imaging lens array, and utilize a sensor array for acquiring different sub-apertures images formed by corresponding imaging lenses. By combining the rectified and synchronized images from 5 × 5 viewpoints with our prototype system, we successfully recovered color light field videos for various fast-moving microscopic specimens with a spatial resolution of 0.79 megapixels at 30 frames per second, corresponding to an unprecedented data throughput of 562.5 MB/s for light field microscopy. We also demonstrated the use of the reported platform for different applications, including post-capture refocusing, phase reconstruction, 3D imaging, and optical metrology.

Transparent Object Reconstruction via Coded Transport of Intensity

Capturing and understanding visual signals is one of the core interests of computer vision. Much progress has been made w.r.t. many aspects of imaging, but the reconstruction of refractive phenomena, such as turbulence, gas and heat flows, liquids, or transparent solids, has remained a challenging problem. In this paper, we derive an intuitive formulation of light transport in refractive media using light fields and the transport of intensity equation. We show how coded illumination in combination with pairs of recorded images allow for robust computational reconstruction of dynamic two and three-dimensional refractive phenomena.

Computational Snapshot Multispectral Cameras: Toward dynamic capture of the spectral world

Multispectral cameras collect image data with a greater number of spectral channels than traditional trichromatic sensors, thus providing spectral information at a higher level of detail. Such data are useful in various fields, such as remote sensing, materials science, biophotonics, and environmental monitoring. The massive scale of multispectral data-at high resolutions in the spectral, spatial, and temporal dimensions-has long presented a major challenge in spectrometer design. With recent developments in sampling theory, this problem has become more manageable through use of undersampling and constrained reconstruction techniques. This article presents an overview of these state-of-the-art multispectral acquisition systems, with a particular focus on snapshot multispectral capture, from a signal processing perspective. We propose that undersampling-based multispectral cameras can be understood and compared by examining the efficiency of their sampling schemes, which we formulate as the spectral sensing coherence information between their sensing matrices and spectrum-specific bases learned from a large-scale multispectral image database. We analyze existing snapshot multispectral cameras in this manner, and additionally discuss their optical performance in terms of light throughput and system complexity.

Snapshot Hyperspectral Volumetric Microscopy.

The comprehensive analysis of biological specimens brings about the demand for capturing the spatial, temporal and spectral dimensions of visual information together. However, such high-dimensional video acquisition faces major challenges in developing large data throughput and effective multiplexing techniques. Here, we report the snapshot hyperspectral volumetric microscopy that computationally reconstructs hyperspectral profiles for high-resolution volumes of ~1000 μm × 1000 μm × 500 μm at video rate by a novel four-dimensional (4D) deconvolution algorithm. We validated the proposed approach with both numerical simulations for quantitative evaluation and various real experimental results on the prototype system. Different applications such as biological component analysis in bright field and spectral unmixing of multiple fluorescence are demonstrated. The experiments on moving fluorescent beads and GFP labelled drosophila larvae indicate the great potential of our method for observing multiple fluorescent markers in dynamic specimens.

Robust and accurate transient light transport decomposition via convolutional sparse coding

Ultrafast sources and detectors have been used to record the time-resolved scattering of light propagating through macroscopic scenes. In the context of computational imaging, decomposition of this transient light transport (TLT) is useful for applications, such as characterizing materials, imaging through diffuser layers, and relighting scenes dynamically. Here, we demonstrate a method of convolutional sparse coding to decompose TLT into direct reflections, inter-reflections, and subsurface scattering. The method relies on the sparsity composition of the time-resolved kernel. We show that it is robust and accurate to noise during the acquisition process.

Camera array based light field microscopy

We present a camera array based light field microscopy that utilizes a camera array to simultaneously capture the different perspectives formed by a two-stage relay system. The proposed method achieves high-resolution light field acquisition with high accuracy and facilitates various applications.

Coded aperture pair for quantitative phase imaging

This Letter proposes a novel quantitative phase-imaging approach by optically encoding light fields into a complementary image pair followed by computational reconstruction. We demonstrate that the axial intensity derivative for phase recovery can be well estimated by a coded-aperture image pair without z axial scanning. The experimental results demonstrate that our approach can achieve higher accuracy and robustness compared with conventional transport-of-intensity equation (TIE) based approaches under partial coherence illumination.