Christopher Schroers

Cyclic Schemes for PDE-Based Image Analysis

We investigate a class of efficient numerical algorithms for many partial differential equations (PDEs) in image analysis. They are applicable to parabolic or elliptic PDEs that have bounded coefficients and lead to space discretisations with symmetric matrices. Our schemes are easy to implement and well-suited for parallel implementations on GPUs, since they are based on the explicit diffusion scheme in the parabolic case, and the Jacobi method in the elliptic case. By supplementing these methods with cyclically varying time step sizes or relaxation parameters, we achieve efficiency gains of several orders of magnitude. We call the resulting algorithms Fast Explicit Diffusion (FED) and Fast Jacobi (FJ) methods. To achieve a good compromise between efficiency and accuracy, we show that one should use parameter cycles that result from factorisations of box filters. For these cycles we establish stability results in the Euclidean norm. Our schemes perform favourably in a number of applications, including isotropic nonlinear diffusion filters with widely varying diffusivities as well as anisotropic diffusion methods for image filtering, inpainting, and regularisation in computer vision. Moreover, they are equally suited for higher dimensional problems as well as higher order PDEs, and they can also be interpreted as efficient first order methods for smooth optimisation problems.

Anisotropic Range Image Integration

Obtaining high-quality 3D models of real world objects is an important task in computer vision. A very promising approach to achieve this is given by variational range image integration methods: They are able to deal with a substantial amount of noise and outliers, while regularising and thus creating smooth surfaces at the same time. Our paper extends the state-of-the-art approach of Zach et al.(2007) in several ways: (i) We replace the isotropic space-variant smoothing behaviour by an anisotropic (direction-dependent) one. Due to the directional adaptation, a better control of the smoothing with respect to the local structure of the signed distance field can be achieved. (ii) In order to keep data and smoothness term in balance, a normalisation factor is introduced. As a result, oversmoothing of locations that are seen seldom is prevented. This allows high quality reconstructions in uncontrolled capture setups, where the camera positions are unevenly distributed around an object. (iii) Finally, we use the more accurate closest signed distances instead of directional signed distances when converting range images into 3D signed distance fields. Experiments demonstrate that each of our three contributions leads to clearly visible improvements in the reconstruction quality.

Point Cloud Noise and Outlier Removal for Image-Based 3D Reconstruction

Point sets generated by image-based 3D reconstruction techniques are often much noisier than those obtained using active techniques like laser scanning. Therefore, they pose greater challenges to the subsequent surface reconstruction (meshing) stage. We present a simple and effective method for removing noise and outliers from such point sets. Our algorithm uses the input images and corresponding depth maps to remove pixels which are geometrically or photometrically inconsistent with the colored surface implied by the input. This allows standard surface reconstruction methods (such as Poisson surface reconstruction) to perform less smoothing and thus achieve higher quality surfaces with more features. Our algorithm is efficient, easy to implement, and robust to varying amounts of noise. We demonstrate the benefits of our algorithm in combination with a variety of state-of-the-art depth and surface reconstruction methods.

Introducing Maximal Anisotropy into Second Order Coupling Models

On the one hand, anisotropic diffusion is a well-established concept that has improved numerous computer vision approaches by permitting direction-dependent smoothing. On the other hand, recent applications have uncovered the importance of second order regularisation. The goal of this work is to combine the benefits of both worlds. To this end, we propose a second order regulariser that allows to penalise both jumps and kinks in a direction-dependent way. We start with an isotropic coupling model, and systematically introduce anisotropic concepts from first order approaches. We demonstrate the benefits of our model by experiments, and apply it to improve an existing focus fusion method.

Introducing More Physics into Variational Depth–from–Defocus

Given an image stack that captures a static scene with different focus settings, variational depth–from–defocus methods aim at jointly estimating the underlying depth map and the sharp image. We show how one can improve existing approaches by incorporating important physical properties. Most formulations are based on an image formation model (forward operator) that explains the varying amount of blur depending on the depth. We present a novel forward operator: It approximates the thin–lens camera model from physics better than previous ones used for this task, since it preserves the maximum–minimum principle w.r.t. the unknown image intensities. This operator is embedded in a variational model that is minimised with a multiplicative variant of the Euler–Lagrange formalism. This offers two advantages: Firstly, it guarantees that the solution remains in the physically plausible positive range. Secondly, it allows a stable gradient descent evolution without the need to adapt the relaxation parameter. Experiments with synthetic and real–world images demonstrate that our model is highly robust under different initialisations. Last but not least, the experiments show that the physical constraints are essential for obtaining more accurate solutions, especially in the presence of strong depth changes.

A Variational Taxonomy for Surface Reconstruction from Oriented Points

The problem of reconstructing a watertight surface from a finite set of oriented points has received much attention over the last decades. In this paper, we propose a general higher order framework for surface reconstruction. It is based on the idea that position and normal defined by each oriented point can be used to construct an implicit local description of the unknown surface. On the one hand, this allows us to systematically explain and relate several popular methods, for example implicit moving least squares, smooth signed distance surface reconstruction as well as (screened) Poisson surface reconstruction. On the other hand, it allows to derive and discuss a number of new approaches for reconstructing either the signed distance or the indicator function of the sought object. All of these approaches are able to achieve competitive results but one of them turns out to be especially promising. To improve reconstructions in difficult real world scenarios where point clouds have been estimated from colour images, we introduce a hull constraint that encourages the surface to stay within a given region. Our framework is implemented on the GPU using a recent cyclic scheme called Fast Jacobi, which combines low implementational effort with high efficiency.

Cross anisotropic cost volume filtering for segmentation

We study an advanced method for supervised multi-label image segmentation. To this end, we adopt a classic framework which recently has been revitalised by Rhemann et al. (2011). Instead of the usual global energy minimisation step, it relies on a mere evaluation of a cost function for every solution label, which is followed by a spatial smoothing step of these costs. While Rhemann et al. concentrate on efficiency, the goal of this paper is to equip the general framework with sophisticated subcomponents in order to develop a high-quality method for multi-label image segmentation: First, we present a substantially improved cost computation scheme which incorporates texture descriptors, as well as an automatic feature selection strategy. This leads to a high-dimensional feature space, from which we extract the label costs using a support vector machine. Second, we present a novel anisotropic diffusion scheme for the filtering step. In this PDE-based process, the smoothing of the cost volume is steered along the structures of the previously computed feature space. Experiments on widely used image databases show that our scheme produces segmentations of clearly superior quality.

Physically inspired depth-from-defocus

We propose a novel variational approach to the depth-from-defocus problem. The quality of such methods strongly depends on the modelling of the image formation (forward operator) that connects depth with out-of-focus blur. Therefore, we discuss different image formation models and design a forward operator that preserves essential physical properties such as a maximumminimum principle for the intensity values. This allows us to approximate the thin-lens camera model in a better way than previous approaches. Our forward operator refrains from any equifocal assumptions and fits well into a variational framework. Additionally, we extend our model to the multi-channel case and show the benefits of a robustification. To cope with noisy input data, we embed our method in a joint depth-from-defocus and denoising approach. For the minimisation of our energy functional, we show the advantages of a multiplicative EulerLagrange formalism in two aspects: First, it constrains our solution to the plausible positive range. Second, we are able to develop a semiimplicit gradient descent scheme with a higher stability range. While synthetic experiments confirm the achieved improvements, experiments on real data illustrate the applicability of the overall method. Novel forward operator preserving essential physical properties is pro-posed.No equifocal assumption is required.Joint handling of depth-from-defocus and denoising can improve the re-construction quality.Robustification leads to a better reconstruction at strong depth changes.Positivity constrained minimisation strategy increases the efficiency.

Multiview Depth Parameterisation with Second Order Regularisation

In this paper we consider the problem of estimating depth maps from multiple views within a variational framework. Previous work has demonstrated that multiple views improve the depth reconstruction, and that higher order regularisers model a good prior for typical real-world 3D scenes. We build on these findings and stress an important aspect that has not been considered in variational multiview depth estimation so far: We investigate several parameterisations of the unknown depth. This allows us to show, both analytically and experimentally, that directly working with depth values introduces an undesirable bias. As a remedy, we reveal that an inverse depth parameterisation is generally preferable. Our analysis clearly points out its benefits w.r.t. the data and the smoothness term. We verify these theoretical findings by means of experiments.

A Dense Pipeline for 3D Reconstruction from Image Sequences

We propose a novel pipeline for 3D reconstruction from image sequences that solely relies on dense methods. At no point sparse features are required. As input we only need a sequence of color images capturing a static scene while following a continuous path. Furthermore, we assume that an intrinsic camera calibration is known. Our pipeline comprises three steps: (1) First, we jointly estimate correspondences and stereo geometry for each two consecutive images. (2) Subsequently, we connect the individual pairwise estimates and globally refine them through bundle adjustment. As a result, all camera poses are merged into a consistent global model. This allows us to create accurate depth maps. (3) Finally, these depth maps are merged using variational range image integration techniques. Experiments show that our dense pipeline is an interesting alternative to sparse approaches. It yields accurate camera poses as well as 3D reconstructions.