Bastian Goldluecke

Variational Light Field Analysis for Disparity Estimation and Super-Resolution

We develop a continuous framework for the analysis of 4D light fields, and describe novel variational methods for disparity reconstruction as well as spatial and angular super-resolution. Disparity maps are estimated locally using epipolar plane image analysis without the need for expensive matching cost minimization. The method works fast and with inherent subpixel accuracy since no discretization of the disparity space is necessary. In a variational framework, we employ the disparity maps to generate super-resolved novel views of a scene, which corresponds to increasing the sampling rate of the 4D light field in spatial as well as angular direction. In contrast to previous work, we formulate the problem of view synthesis as a continuous inverse problem, which allows us to correctly take into account foreshortening effects caused by scene geometry transformations. All optimization problems are solved with state-of-the-art convex relaxation techniques. We test our algorithms on a number of real-world examples as well as our new benchmark data set for light fields, and compare results to a multiview stereo method. The proposed method is both faster as well as more accurate. Data sets and source code are provided online for additional evaluation.

Globally consistent depth labeling of 4D light fields

We present a novel paradigm to deal with depth reconstruction from 4D light fields in a variational framework. Taking into account the special structure of light field data, we reformulate the problem of stereo matching to a constrained labeling problem on epipolar plane images, which can be thought of as vertical and horizontal 2D cuts through the field. This alternative formulation allows to estimate accurate depth values even for specular surfaces, while simultaneously taking into account global visibility constraints in order to obtain consistent depth maps for all views. The resulting optimization problems are solved with state-of-the-art convex relaxation techniques. We test our algorithm on a number of synthetic and real-world examples captured with a light field gantry and a plenoptic camera, and compare to ground truth where available. All data sets as well as source code are provided online for additional evaluation.

Datasets and Benchmarks for Densely Sampled 4D Light Fields

We present a new benchmark database to compare and evaluate existing and upcoming algorithms which are tailored to light field processing. The data is characterised by a dense sampling of the light fields, which best fits current plenoptic cameras and is a characteristic property not found in current multi-view stereo benchmarks. It allows to treat the disparity space as a continuous space, and enables algorithms based on epipolar plane image analysis without having to refocus first. All datasets provide ground truth depth for at least the center view, while some have additional segmentation data available. Part of the light fields are computer graphics generated, the rest are acquired with a gantry, with ground truth depth established by a previous scanning of the imaged objects using a structured light scanner. In addition, we provide source code for an extensive evaluation of a number of previously published stereo, epipolar plane image analysis and segmentation algorithms on the database.

The natural vectorial total variation which arises from geometric measure theory

Several ways to generalize scalar total variation to vector-valued functions have been proposed in the past. In this paper, we give a detailed analysis of a variant we denote by

Spatial and angular variational super-resolution of 4d light fields

\text{TV}_J

Globally Consistent Multi-label Assignment on the Ray Space of 4D Light Fields

, which has not been previously explored as a regularizer. The contributions of the manuscript are twofold: on the theoretical side, we show that

On the Calibration of Focused Plenoptic Cameras

\text{TV}_J

The Variational Structure of Disparity and Regularization of 4D Light Fields

can be derived from the generalized Jacobians from geometric measure theory. Thus, within the context of this theory,

Bayesian View Synthesis and Image-Based Rendering Principles

\text{TV}_J

A Dataset and Evaluation Methodology for Depth Estimation on 4D Light Fields

is the most natural form of a vectorial total variation. As an important feature, we derive how