Lingfei Meng

Single-Shot Specular Surface Reconstruction with Gonio-Plenoptic Imaging

We present a gonio-plenoptic imaging system that realizes a single-shot shape measurement for specular surfaces. The system is comprised of a collimated illumination source and a plenoptic camera. Unlike a conventional plenoptic camera, our system captures the BRDF variation of the object surface in a single image in addition to the light field information from the scene, which allows us to recover very fine 3D structures of the surface. The shape of the surface is reconstructed based on the reflectance property of the material rather than the parallax between different views. Since only a single-shot is required to reconstruct the whole surface, our system is able to capture dynamic surface deformation in a video mode. We also describe a novel calibration technique that maps the light field viewing directions from the object space to subpixels on the sensor plane. The proposed system is evaluated using a concave mirror with known curvature, and is compared to a parabolic mirror scanning system as well as a multi-illumination photometric stereo approach based on simulations and experiments.

Light Field Otoscope

We introduce a light field otoscope that enables new functionalities for middle ear imaging, such as depth estimation, multi-view extraction, and multispectral rendering of the tympanic membrane. A prototype and experimental results are presented.

Evaluation of multispectral plenoptic camera

Plenoptic cameras enable capture of a 4D lightfield, allowing digital refocusing and depth estimation from data captured with a compact portable camera. Whereas most of the work on plenoptic camera design has been based a simplistic geometric-optics-based characterization of the optical path only, little work has been done of optimizing end-to-end system performance for a specific application. Such design optimization requires design tools that need to include careful parameterization of main lens elements, as well as microlens array and sensor characteristics. In this paper we are interested in evaluating the performance of a multispectral plenoptic camera, i.e. a camera with spectral filters inserted into the aperture plane of the main lens. Such a camera enables single-snapshot spectral data acquisition.1–3 We first describe in detail an end-to-end imaging system model for a spectrally coded plenoptic camera that we briefly introduced in.4 Different performance metrics are defined to evaluate the spectral reconstruction quality. We then present a prototype which is developed based on a modified DSLR camera containing a lenslet array on the sensor and a filter array in the main lens. Finally we evaluate the spectral reconstruction performance of a spectral plenoptic camera based on both simulation and measurements obtained from the prototype.

In Vivo Middle Ear Imaging with a Light Field Otoscope

We present a novel design of a light field otoscope for use in pediatric primary care settings to acquire diagnostic features such as 3D shape and multispectral information. Compared to prior art, the prototype improves speed, field-of-view, depth-of-field and illumination.

System Model and Performance Evaluation of Spectrally Coded Plenoptic Camera

We introduce an end-to-end imaging system model for a spectrally coded plenoptic camera. The model includes a system-dependent spectral demultiplexing algorithm and is used to evaluate spectral quality and classification performance of a spectrally coded plenoptic camera.

Parallax rectification for spectrally-coded plenoptic cameras

Spectrally-coded plenoptic cameras have been developed for single-snapshot spectral imaging. Whereas reconstruction of spectral images from plenoptic sensor data has been studied in the past, rectification of those images according to a depth distribution in the scene has not been investigated. In this paper we describe a spectral parallax rectification algorithm for plenoptic cameras. The algorithm is designed to handle flexible filter partitions in the aperture and the small baselines inherent to plenoptic cameras. We demonstrate the algorithms performance using checkerboard and captured scene data.

Single-snapshot 2D color measurement by plenoptic imaging system

Plenoptic cameras enable capture of directional light ray information, thus allowing applications such as digital refocusing, depth estimation, or multiband imaging. One of the most common plenoptic camera architectures contains a microlens array at the conventional image plane and a sensor at the back focal plane of the microlens array. We leverage the multiband imaging (MBI) function of this camera and develop a single-snapshot, single-sensor high color fidelity camera. Our camera is based on a plenoptic system with XYZ filters inserted in the pupil plane of the main lens. To achieve high color measurement precision of this system, we perform an end-to-end optimization of the system model that includes light source information, object information, optical system information, plenoptic image processing and color estimation processing. Optimized system characteristics are exploited to build an XYZ plenoptic colorimetric camera prototype that achieves high color measurement precision. We describe an application of our colorimetric camera to color shading evaluation of display and show that it achieves color accuracy of ΔE<0.01.