Sebastian Höfer

Multiview specular stereo reconstruction of large mirror surfaces

In deflectometry, the shape of mirror objects is recovered from distorted images of a calibrated scene. While remarkably high accuracies are achievable, state-of-the-art methods suffer from two distinct weaknesses: First, for mainly constructive reasons, these can only capture a few square centimeters of surface area at once. Second, reconstructions are ambiguous i.e. infinitely many surfaces lead to the same visual impression. We resolve both of these problems by introducing the first multiview specular stereo approach, which jointly evaluates a series of overlapping deflectometric images. Two publicly available benchmarks accompany this paper, enabling us to numerically demonstrate viability and practicability of our approach.

Cavlectometry: Towards Holistic Reconstruction of Large Mirror Objects

We introduce a method based on the deflectometry principle for the reconstruction of specular objects exhibiting significant size and geometric complexity. A key feature of our approach is the deployment of an Automatic Virtual Environment (CAVE) as pattern generator. To unfold the full power of this experimental setup, an optical encoding scheme is developed which accounts for the distinctive topology of the CAVE. Furthermore, we devise an algorithm for detecting the object of interest in raw deflect metric images. The segmented foreground is used for single-view reconstruction, the background for estimation of the camera pose, necessary for calibrating the sensor system. Experiments suggest a significant gain of coverage in single measurements compared to previous methods.

Infrared deflectometry for the inspection of diffusely specular surfaces

Abstract Deflectometry is a full-field gradient technique that lends itself very well to testing specular surfaces. It uses the geometry of specular reflection to determine the gradient of the surface under inspection. In consequence, a necessary precondition to apply deflectometry is the presence of at least partially specular reflection. Surfaces with larger roughness have increasingly diffuse reflection characteristics, making them inaccessible to usual deflectometry. However, many industrially relevant surfaces exist that change their reflection characteristic during production and processing. An example is metal sheets that are used as car body parts. Whereas the molded but otherwise raw metal sheets show a mostly diffuse reflection without sufficient specular reflection, the final car body panels have a high specular reflectance due to the lacquering. In consequence, it would be advantageous to apply the same inspection approach both for the raw material and for the final product. To solve this challenge, specular reflection from rough surfaces can be achieved using light with a larger wavelength, as the specular reflectivity of a surface depends on the ratio of the surface roughness and the wavelength of the light applied. Wavelengths in the thermal infrared range create enough specular reflection to apply deflectometry on many visually rough metal surfaces. This contribution presents the principles of thermal deflectometry, its special challenges, and illustrates its use with examples from the inspection of industrially produced surfaces.

Pattern coding strategies for deflectometric measurement systems

In this paper we assess the impact of different error sources on the deflectometric measurement. We provide an overview of previous work in this field and fill the gaps to provide a unified measurement model. The focus is on the parameters of a deflectometric setup with the objective to give practice-oriented guidelines for optimizing the deflectometric data acquisition. We will differentiate between systematic error sources which can be anticipated and compensated for and errors which are intrinsic to the deflectometric measurement method itself. In the later case possible trade-offs between parameters are highlighted to enable the optimization of a setup to a specific application.

A novel region-based approach for the fusion of combined stereo and spectral series

This contribution proposes a novel approach for image fusion of combined stereo and spectral series acquired simultaneously with a camera array. To this purpose, nine cameras are equipped with spectral filters (50 nm spectral bandwidth) such that the visible and near infrared parts of the spectrum (400-900 nm) are observed. The resulting image series is fused in order to obtain two types of information: the 3D shape of the scene and its spectral properties. For the registration of the images, a novel region based registration approach which evaluates the gray value invariant features (e.g. edges) of regions in segmented images is proposed. The registration problem is formulated by means of energy functionals. The data term of our functional compares features of a region in one image with features of an area in another image, such that an additional independency of the form and size of the regions in the segmented images is obtained. As regularization, a smoothness term is proposed, which models the fact that disparity discontinuities should only occur at edges in the images. In order to minimize the energy functional, we use graph cuts. The minimization is carried out simultaneously over all image pairs in the series. Even though the approach is region based, a label (e.g. disparity) is assigned to each pixel. The result of the minimization approach consists of a disparity map. By means of calibration, we use the disparity map to compute a depth map. Once pixel depths are determined, the images can be warped to a common view, such that a pure spectral series is obtained. This can be used to classify different materials of the objects in the scene based on real spectral information, which cannot be acquired with a common RGB camera.

Optimization on shape curves with application to specular stereo

We state that a one-dimensional manifold of shapes in 3- space can be modeled by a level set function. Finding a minimizer of an independent functional among all points on such a shape curve has interesting applications in computer vision. It is shown how to replace the commonly encountered practice of gradient projection by a projection onto the curve itself. The outcome is an algorithm for constrained optimization, which, as we demonstrate theoretically and numerically, provides some important benefits in stereo reconstruction of specular surfaces.

Scannende Infrarotdeflektometrie für die Inspektion diffus spiegelnder Oberflächen

Zusammenfassung Unter Infrarotdeflektometrie versteht man die Anwendung des Messprinzips der Deflektometrie im thermischen Infrarotspektrum. Dadurch lässt sich ausnutzen, dass vor allem raue, metallische Oberflächen in diesem Spektrum spiegelnd erscheinen und so eine deflektometrische Inspektion ermöglicht wird. Diese Arbeit befasst sich mit dem Problem der Erzeugung der notwendigen thermischen Codemuster und deren Auswertung. Es werden verschiedene Ansätze vorgestellt, um Oberflächen mit Hilfe eines Industrieroboters oder im Durchlauf zu inspizieren. Darüber hinaus wird zur Auswertung der Muster zu einer Registrierung ein neues Verfahren vorgestellt. Die Ergebnisse werden an praktisch relevanten Beispielen demonstriert.

Flächenbasierte Registrierung kombinierter Stereo- und Spektralserien

Zusammenfassung Kamera-Arrays und kombinierte Bildserien werden verwendet, um zeitsparend heterogene Informationen über eine Szene zu erfassen. Als Beispiel beinhalten kombinierte Stereo- und Spektralserien sowohl räumliche als auch spektrale Information über eine Szene. Als Grundlage für eine Auswertung des Steroeffekts in solchen kombinierten Bildserien wird im vorliegenden Beitrag ein neuartiges flächenbasiertes Registrierungsverfahren dargestellt. Dabei werden die Bilder der Serie zunächst segmentiert und anschließend für die erhaltenen Regionen Merkmale extrahiert. Anhand dieser Merkmale, die im Wesentlichen Kanten in den Bildern beschreiben, werden Korrespondenzen zwischen Regionen bestimmt und daraus Tiefenkarten berechnet. Das Registrierungsproblem wird durch Energiefunktionale modelliert und mittels eines modifizierten Graph-Cuts-Verfahrens minimiert. Beispiele veranschaulichen die Vorgehensweise. Abstract The fast acquisition of heterogeneous information about a scene can be approached by image series with more than one varied parameter, also referred to as combined image series acquired with camera arrays. Examples of such series are combined stereo and spectral series which contain both spatial and spectral information about the scene. We propose in this contribution a new region-based registration method for evaluating the stereo effect in such combined image series. The images are first segmented and then features of the resulting regions are extracted. The features mainly describe edges of the regions and are used to determine correspondences between one or several regions. Energy functionals are employed to model the registration problem. The solution is found by minimizing the energy functional by means of a modified graph-cuts algorithm. Examples are presented to visualize the methods proposed.