Ludovic Angot

A 2D to 3D video and image conversion technique based on a bilateral filter

A method to convert a 2D video or 2D still image into a depth map is presented. The depth map can consequently be converted into a pair of right and left images for viewing on a stereoscopic display. The depth map can be used for other applications such as multi-view generation. Our method is based on the use of an initial depth map and a bilateral filter approximated by a bilateral grid. For still images, the initial depth map is arbitrary set by default or can be chosen by the user, while for videos it is evaluated from motion estimation. Our method produces finely segmented depth maps; when converted to the appropriate format and seen on a stereoscopic display, the resulting image or video is both realistic and comfortable to watch.

Digital Decoding Design for Phase Coded Imaging System

This paper develops a digital decoding design for the imaging system with phase coded lens. The phase coded lens is employed to extend the depth of filed (DoF), and the proposed design is used to restore the special-purpose blur caused by the lens. Since in practice the imaging system inevitably contains manufacturing inaccuracy, it is often difficult to obtain precise point spread function (PSF) for image restoration. To deal with this problem, we develop a flow for designing filters without PSF information. The imaging system first takes a shot of a well-designed test chart to have a blur image of the chart. This blur image is then corrected by using the perspective transformation. We use both of the image of the test chart and the corrected blur image to calculate a minimum mean square error (MMSE) filter, so that the blur image processed by the filter can be very alike to the test chart image. The filter is applied to other images captured by the imaging system in order to verify its effectiveness in reducing the blur and for showing the capability of extending the DoF of the integrated system.

Single shot depth camera lens design optimization based on a blur metric

Computational imaging technology can capture extra information at the sensor and can be used for various photographic applications, including imaging with extended depth of field or depth extraction for 3D applications. The depth estimation from a single captured photograph can be achieved through a phase coded lens and image processing. In this paper, we propose a new method to design a phase coded lens, using a blur metric (BM) as the design criterion. Matlab and Zemax are used for the co-optimization of optical coding and digital image process. The purpose of the design is to find a curve for which the BM changes continuously and seriously within a distance range. We verified our approach by simulation, and got a axial symmetric phase mask as the coded lens. By using a pseudo-random pattern which contains uniform black and white patches as the input image, and the on-axis point spread function (PSF) calculated from Zemax, we can evaluate the BM of the simulated image which is convoluted by the pseudo-random pattern and PSF. In order to ensure the BM curve evaluated from the on-axis PSF represents the result of the whole field of view, the PSF is also optimized to get high off-axis similarity.

Comparison of the angular selectivity characteristics for an off-axis holographic storage system calculated by Born approximation and coupled-wave theory

The first Born approximation is applied to calculate the angular selectivity for different positions on the reconstructed image as a function of the object beams optical axis angle theta(ob) and reference beam angle theta(rw) for a holographic data storage system that records the Fourier transform holograms in a medium with an infinite plane-wave reference beam. Results are compared with those calculated by the coupled-wave theory.

31.2: Invited Paper: Enhanced Minimally Invasive Surgery by 2D to 3D Conversion

Minimally Invasive Surgery (MIS) is a growing surgical technique comparing to traditional open surgery since it dramatically reduces patient trauma and costs but MIS technique is difficult because of looking at a 2D screen to perform the surgery. Depth perception is thus lost due to 2D endoscopic capturing and display. Therefore, the paper proposes real-time 2D to 3D conversion for enhancing MIS to provide depth perception on 3D display for increasing surgical quality, improving in safety and speed of procedures, and also to lower learning curve. The proposed method to convert an endoscopic 2D video frame into a depth map and consequently into a pair of right and left image for viewing on a stereoscopic display is based on the use of a bilateral filter approximated by a bilateral grid. The architecture is to make edges sharp to enhance moving instruments and to preserve chamber geometry segmentation. Results of verification by simulated surgery and questionnaire for surgeons proof the feasibility of 2D to 3D conversion on MIS.

A real-time video 2D-to-3D with the bilateral grid

The new data structure, the bilateral grid, was presented by Jiawen et al. to make bilateral filter algorithm become simple implementation. Based on the data structure, the GPU CUDA-based optimization is proposed to have more efficiency in using GPU shared memory and massive multithreading. Meanwhile, a commercial application, the video 2d to 3d conversion which was presented by Ludovic et al. is also re-designed by applying the proposed CUDA-based bilateral grid three times to obtain better 3D quality in real-time. Depth map are created and modified by adjusting bilateral grid parameters.

Effect of the Recording Material Shrinkage on Holographic Reconstructed Image and Compensation Method by Simulation

We derive the formula for intensity distribution of a holographic reconstructed image related to the effect of the recording material shrinkage for an off-axis holographic storage system with a plane wave reference beam using the Born approximation method. It is shown that even under the Bragg condition, the intensity across the reconstructed image decreases significantly when the angle between write reference beam and object beam becomes large. We also introduce a simple method to compensate for the effect of the recording material shrinkage by shifting the read reference beam angle.

Evaluating distances using a coded lens camera and a blur metric

We propose a method and a system to measure distances to a target and at the same time obtaining an image of the target. The system is based on a wavefront coded lens and an image processing unit. The method requires a calibration phase where a blur metric is related to the distance of the target. The distance to the target for any position is then obtained by computing the blur metric of the target and using the calibration data. The method and system are easy to manufacture and provide an alternative to other distance measuring methods and devices, while also producing an image of the scene. The target is a printed image of pseudo random black and white elements which can be stick or placed nearby objects of which distance is to be evaluated.