Suren Vagharshakyan

Light Field Reconstruction Using Shearlet Transform

In this article we develop an image based rendering technique based on light field reconstruction from a limited set of perspective views acquired by cameras. Our approach utilizes sparse representation of epipolar-plane images (EPI) in shearlet transform domain. The shearlet transform has been specifically modified to handle the straight lines characteristic for EPI. The devised iterative regularization algorithm based on adaptive thresholding provides high-quality reconstruction results for relatively big disparities between neighboring views. The generated densely sampled light field of a given 3D scene is thus suitable for all applications which require light field reconstruction. The proposed algorithm compares favorably against state of the art depth image based rendering techniques and shows superior performance specifically in reconstructing scenes containing semi-transparent objects.

Image based rendering technique via sparse representation in shearlet domain

In this paper we propose a method for reconstructing a densely sampled light field from a given sparse set of perspective views from rectified cameras without an explicit estimation of the scene depth. The desired intermediate views are synthesized by inpainting of epipolar-plane images, utilizing their sparsity in the shearlet domain. For the purpose of shearlet-domain representation, compactly supported shearlets have been constructed using different directional filters for different scales in an attempt to provide better directional selectivity at lower scales. The reconstruction procedure with shearlet-domain sparsity condition is implemented through an iterative thresholding algorithm. The performance of the method is quantified by tests on synthetic and real visual data and compared favorably against depth-image based rendering.

Perceptual quality of reconstructed medical images on projection-based light field displays

With the appearance of light field displays, users may enjoy a much more natural sensation of 3D experience compared to prior technologies. This type of autostereoscopic, glasses-free visualization allows medical applications to improve both in usability and efficiency. The high angular resolution of medical images is resource-consuming, but can only be reduced while maintaining a sufficient level of overall quality through continuous parallax. A dense image set can also be achieved by applying the synthesis of intermediate views. In this paper we provide the analysis of the effect of reduced angular resolution and image synthesis on Quality of Experience in medical applications. Two separate series of subjective quality assessment measurements were conducted with 20 participants each, one focusing on angular resolution reduction and another one comparing the effect of such reductions with the quality of reconstructed images.

The Effect of Light Field Reconstruction and Angular Resolution Reduction on the Quality of Experience

The quality of visual contents displayed on 3D autostereoscopic displays – such as light field displays – essentially depend on factors that are not present in case of 3D stereoscopic or 2D ones, like angular resolution. A higher number of views in a given field of view enables a smoother, continuous motion parallax, but evidently requires more resources to transmit and display. However, in several cases a sufficiently high number of views might not even be available, thus light field reconstruction is required to increase the density of intermediate views. In this paper we introduce the results of a research aiming to measure the perceptual difference between light field reconstruction and different angular resolutions via a series of subjective image quality assessments. The analysis also calls attention to transmission requirements of content for light field displays.

Accelerated Shearlet-Domain Light Field Reconstruction

We consider the problem of reconstructing densely sampled light field (DSLF) from sparse camera views. In our previous work, the DSLF has been reconstructed by processing epipolar-plane images (EPI) employing sparse regularization in shearlet transform domain. With the aim to avoid redundant processing and reduce the overall reconstruction time, in this paper, we propose algorithm modifications in three directions. First, we modify the basic algorithm by offering a faster and more stable iterative procedure. Second, we elaborate on the proper use of color redundancy by studying the effect of reconstruction of an average intensity channel and its use as a guiding mode for colorizing the three color channels. Third, we explore similarities between EPIs by their grouping and joint processing or by effective decorrelation to get an initial estimate for the basic iterative procedure. We are specifically interested in GPU-based computations allowing an efficient implementation of the shearlet transform. We quantify our three main approaches to accelerated processing over a wide collection of horizontal as well as full-parallax datasets.

Tree-structured algorithm for efficient shearlet-domain light field reconstruction

This article considers techniques for accelerating a light field reconstruction algorithm operating in shearlet domain. In the proposed approach, an independent reconstruction of epipolar images (EPIs) is replaced with a consecutive tree-structured reconstruction. It aims at decreasing the number of iterations necessary for an EPI reconstruction by using already processed EPIs as initial values in the reconstruction stage. Two algorithms for structuring such processing trees are presented. The reconstruction performance of the proposed algorithms is illustrated on a real dataset. The underlying differences between the algorithms are discussed and numerical results of computation speeds are presented.

Signal Processing Methods for Light Field Displays

This chapter discusses the topic of emerging light field displays from a signal processing perspective. Light field displays are defined as devices which deliver continuous parallax along with the focus and binocular visual cues acting together in rivalry-free manner. In order to ensure such functionality, one has to deal with the light field, conceptualized by the plenoptic function and its adequate parametrization, sampling and reconstruction. The light field basics and the corresponding display technologies are overviewed in order to address the fundamental problems of analyzing light field displays as signal processing channels, and of capturing and representing light field visual content for driving such displays. Spectral analysis of multidimensional sampling operators is utilized to profile the displays in question, and modern sparsification approaches are employed to develop methods for high-quality light field reconstruction and rendering.

Shearlet-domain light field reconstruction for holographic stereogram generation

Holographic stereograms (HSs) constitute one of the most widely used types of computer-generated holograms. The scene information required to calculate the HSs can be acquired by conventional digital cameras. It is, however, usually required that the scene should be captured from dense set of view points. Therefore, relieving this requirement is critical in the sense of easing the capture process. In this paper, in the capture stage of holographic stereograms, we employ our previously presented light field reconstruction algorithm [1], where we utilize sparse representation of light fields in the shearlet domain and reconstruct dense light fields from their highly under-sampled versions. The simulation results demonstrate that we can relieve the dense view sampling requirement of HSs, e.g. by as high as 8 × 8 sub-sampling factor, and still keep the perceived image quality of holographic reconstructions at satisfactory levels. This enables, for example, replacing the scanning camera setups with the more convenient multi-camera arrangements.

Accuracy Evaluation of a Linear Positioning System for Light Field Capture

In this paper a method has been proposed for estimating the positions of a moving camera attached to a linear positioning system (LPS). By comparing the estimated camera positions with the expected positions, which were calculated based on the LPS specifications, the manufacturer specified accuracy of the system, can be verified. Having this data, one can more accurately model the light field sampling process. The overall approach is illustrated on an in-house assembled LPS.