Yifan Peng

Encoded diffractive optics for full-spectrum computational imaging

Diffractive optical elements can be realized as ultra-thin plates that offer significantly reduced footprint and weight compared to refractive elements. However, such elements introduce severe chromatic aberrations and are not variable, unless used in combination with other elements in a larger, reconfigurable optical system. We introduce numerically optimized encoded phase masks in which different optical parameters such as focus or zoom can be accessed through changes in the mechanical alignment of a ultra-thin stack of two or more masks. Our encoded diffractive designs are combined with a new computational approach for self-calibrating imaging (blind deconvolution) that can restore high-quality images several orders of magnitude faster than the state of the art without pre-calibration of the optical system. This co-design of optics and computation enables tunable, full-spectrum imaging using thin diffractive optics.

Computational imaging using lightweight diffractive-refractive optics

Diffractive optical elements (DOE) show great promise for imaging optics that are thinner and more lightweight than conventional refractive lenses while preserving their light efficiency. Unfortunately, severe spectral dispersion currently limits the use of DOEs in consumer-level lens design. In this article, we jointly design lightweight diffractive-refractive optics and post-processing algorithms to enable imaging under white light illumination. Using the Fresnel lens as a general platform, we show three phase-plate designs, including a super-thin stacked plate design, a diffractive-refractive-hybrid lens, and a phase coded-aperture lens. Combined with cross-channel deconvolution algorithm, both spherical and chromatic aberrations are corrected. Experimental results indicate that using our computational imaging approach, diffractive-refractive optics is an alternative candidate to build light efficient and thin optics for white light imaging.

Grayscale performance enhancement for time-multiplexing light field rendering

One of the common approaches to compensate for the grayscale performance limitation in time-multiplexing light field displays is to employ a halftone technique. We propose an ordered-dithering halftone algorithm based on a 3-dimension super-mask to increase the gray levels of the time-multiplexing light field display. Our method makes full use of the overlapping perceived pixels which are caused by the time-multiplexing design, such that effectively trading-off the spatial resolution and color performance. A real-time rendering time-multiplexing display prototype is built to validate the proposed halftone algorithm. We conducted a user study to evaluate the quality of display scenes dithered by different super-mask configuration, which showed the consistency with the parameters we pre-calculated. The 3D ordered-dithering algorithm is able to present better visual perception than the conventional halftone algorithms with respect to grayscale representation, and flexible to be applied in different time-multiplexing light field display systems.

Optimized Image Synthesis for Multi-Projector-Type Light Field Display

Conventional light field displays suffer from the problem that severe distortion is perceived when the scenes are reconstructed far from the focused screen, due to the angular information loss in the reconstruction process. We introduce a weighted average optimization to the image synthesis process, aiming to tradeoff the reconstructed depth range and the image sharpness without changing the hardware configuration, particularly for multi-projector-type light field displays. The proposed optimization evaluates the lost angular information and reallocates them in the rendering process. The RMSE evaluation on the optimization performance is implemented. The experimental results show that after the optimization the display offers more accurate reconstruction than before. Besides, a subjective experiment is implemented to further validate the effectiveness of the optimization. We envision this optimization being applied to various multi-projector-type light field displays, despite of the arrangement of projectors and the shape of the screen.

Towards VR and AR: 360° light field display with mid-air interaction

We present a 360° light field display to visualize 3D objects above the table-top screen, and provide a graspable and measurable virtual or augmented reality environment. The display is constructed by a spinning flat-plate deflected diffuser screen, a high-speed light field projector, and the user tracking and interaction components. A rear-projection light propagation is specially designed to reconstruct the light field. By calibrating this levitated light field to the physical dimension, and capturing user gestures, we enable multiple users mid-air interactions based on their virtual measurement perception.

End-to-end Optimization of Optics and Image Processing for Achromatic Extended Depth of Field and Super-Resolution Imaging

In typical cameras the optical system is designed first; once it is fixed, the parameters in the image processing algorithm are tuned to get good image reproduction. In contrast to this sequential design approach, we consider joint optimization of an optical system (for example, the physical shape of the lens) together with the parameters of the reconstruction algorithm. We build a fully-differentiable simulation model that maps the true source image to the reconstructed one. The model includes diffractive light propagation, depth and wavelength-dependent effects, noise and nonlinearities, and the image post-processing. We jointly optimize the optical parameters and the image processing algorithm parameters so as to minimize the deviation between the true and reconstructed image, over a large set of images. We implement our joint optimization method using autodifferentiation to efficiently compute parameter gradients in a stochastic optimization algorithm. We demonstrate the efficacy of this approach by applying it to achromatic extended depth of field and snapshot super-resolution imaging.

Computational imaging with diffractive optics

Diffractive optical elements (DOEs) have been studied extensively in optics for decades, but have recently received a lot of renewed interests in the context of computational imaging and computational display because they can drastically reduce the size and weight of devices. However, the inherent strong dispersion is an obstacle that limits the use of DOEs in full spectrum imaging, causing unacceptable color fidelity loss in the captured or reconstructed images. Despite the benefits of facilitating compact form factors, DOEs have sufficient degrees of freedom that one can manipulate to encode the desirable light modulation functionality. In this dissertation we theoretically and experimentally investigate the practicability of introducing numerical optimization into the design procedure of optics, to enable a variety of diffractive optics subject to different application scenarios. Regarding imaging applications, we first validate the practicality of introducing diffractive-refractive hybrid elements as simplified optics in conventional computational imaging. We re-implement a cross-channel based deconvolution to correct the chromatic aberration. The full fabrication cycle of photolithography technique is developed, serving as the basis for all designs. Then, the desirable focal powers in both spectral and spatial domain are encoded onto the DOEs. Precisely, we develop the diffractive achromat that balances the focusing contributions in full visible spectrum. The color fidelity can thus be well preserved. Meanwhile, we develop the encoded lenses that provide focus tunable imaging ability and multi-focal sweep imaging compromise. The trade-off light loss and residual aberrations are tackled by a deconvolution step. In addition, we extend the design paradigm from image capture to image display, where the holograms of visualization of multiple narrowband spectra are en-

Revisiting Cross-Channel Information Transfer for Chromatic Aberration Correction

Image aberrations can cause severe degradation in image quality for consumer-level cameras, especially under the current tendency to reduce the complexity of lens designs in order to shrink the overall size of modules. In simplified optical designs, chromatic aberration can be one of the most significant causes for degraded image quality, and it can be quite difficult to remove in post-processing, since it results in strong blurs in at least some of the color channels. In this work, we revisit the pixel-wise similarity between different color channels of the image and accordingly propose a novel algorithm for correcting chromatic aberration based on this cross-channel correlation. In contrast to recent weak prior-based models, ours uses strong pixel-wise fitting and transfer, which lead to significant quality improvements for large chromatic aberrations. Experimental results on both synthetic and real world images captured by different optical systems demonstrate that the chromatic aberration can be significantly reduced using our approach.

Beam shaping for multicolour light-emitting diodes with diffractive optical elements

An improved particle swarm optimization method is proposed for the design of ultra-thin diffractive optical elements (DOEs) enabling multicolour beam shaping functionality. We employ pre-optimized initial structures and adaptive weight strategy in the algorithm to achieve better and identical shaping performance for multiple colours. Accordingly, a DOE for shaping light from green and blue LEDs has been designed and fabricated. Both experiment and numerical simulations have been conducted and the results agree well with each other. 15.66% average root mean square error (RMSE) and 0.22% RMSE difference are achieved. In addition, the parameters closely related to the performance of the optimization are analysed, which can provide insights for future application designs.

Highly efficient waveguide display with space-variant volume holographic gratings

We propose a highly efficient waveguide display based on space-variant volume holographic gratings (SVVHGs). The local period and slant angle of the SVVHG vary along the tangential direction, enabling variant incident angles to satisfy the Bragg condition of the local gratings. As a result, we enlarge the field of view (FOV) without using the conventional multiplexing scheme, while achieving high efficiency and large FOV at the same time. We experimentally record the SVVHGs on Bayfol HX200 films. We demonstrate that the proposed display can achieve 31.9% system efficiency for a broadband light source and 52.3% for a coherent light source, 20° FOV, and high brightness uniformity, making it a promising candidate for widespread applications in the augmented reality (AR) industry.