Thomas Luhmann

GEOMETRIC CALIBRATION OF THERMOGRAPHIC CAMERAS

This chapter presents an overview of thermal imaging sensors for photogrammetric close-range applications. In particular, it presents results of the geometric calibration of thermographic cameras as they are used for building inspection and material testing. Geometric calibration becomes evident for all precise geometric image operations, e.g. mosaicking of two or more images or photogrammetric 3D modelling with thermal imagery. Two different test fields have been designed providing point targets that are visible in the thermal spectral band of the cameras.

Least-squares Matching with Advanced Geometric Transformation Models

The least-squares matching algorithm (LSM) for area-based image matching is a well known technique in photogrammetry and computer vision since more than two decades. Differences between two or more images can be modelled by estimating geometric and radiometric transformation functions within the functional model. Commonly the affine transformation is used as geometric transformation. Since this approach is not strict in terms of the projective imaging model, it is worthwhile to investigate alternative transformation models. This paper presents an advanced least-squares matching algorithm that uses the projective transformation model and polynomial transformations to handle geometric distortions between the images. The projective approach is geometrically strict as long as object surface and image sensors are planes. The polynomial approach is supposed to be geometrically strict for plane image sensors and non-plane object surfaces. The possibility of this kind of expansions has been mentioned in several papers but up to now no publicly available investigation is known. First results of the new geometric model have been published by the authors in 2008, showing promising effects on non-planar object patches. * Corresponding author

3D imaging: how to achieve highest accuracy

The generation of 3D information from images is a key technology in many different areas, e.g. in 3D modeling and representation of architectural or heritage objects, in human body motion tracking and scanning, in 3D scene analysis of traffic scenes, in industrial applications and many more. The basic concepts rely on mathematical representations of central perspective viewing as they are widely known from photogrammetry or computer vision approaches. The objectives of these methods differ, more or less, from high precision and well-structured measurements in (industrial) photogrammetry to fully-automated non-structured applications in computer vision. Accuracy and precision is a critical issue for the 3D measurement of industrial, engineering or medical objects. As state of the art, photogrammetric multi-view measurements achieve relative precisions in the order of 1:100000 to 1:200000, and relative accuracies with respect to retraceable lengths in the order of 1:50000 to 1:100000 of the largest object diameter. In order to obtain these figures a number of influencing parameters have to be optimized. These are, besides others: physical representation of object surface (targets, texture), illumination and light sources, imaging sensors, cameras and lenses, calibration strategies (camera model), orientation strategies (bundle adjustment), image processing of homologue features (target measurement, stereo and multi-image matching), representation of object or workpiece coordinate systems and object scale. The paper discusses the above mentioned parameters and offers strategies for obtaining highest accuracy in object space. Practical examples of high-quality stereo camera measurements and multi-image applications are used to prove the relevance of high accuracy in different applications, ranging from medical navigation to static and dynamic industrial measurements. In addition, standards for accuracy verifications are presented and demonstrated by practical examples and tests.

Skeleton Timed Up and Go

This paper presents a novel approach to fully automate the Timed Up and Go Assessment Test (TUG) in professional environments. The approach, called Skeleton Timed Up and Go (sTUG), is based on the usage of two Kinect for Xbox 360 sensors. sTUG supports the execution and documentation of the traditional TUG assessment test and furthermore calculates nine events, which demarcate the five main components during a run. On two days we conducted an experiment with five elderly aged 70-84 and four males aged 29-31 in the activity laboratory of the OFFIS Institute, Oldenburg to proof the reliability and feasibility of the system. Results demonstrate that sTUG can precisely measure the total duration of traditional TUG and is capable of detecting accurately nine motion events which demarcate the components during a run.

Measurement of Rotor Blade Deformations of Wind Energy Converters with Laser Scanners

Wind energy converters in operation are exposed to high stresses which result in large deformations of the rotor blades. In this paper a method for determination of deformations of rotating rotor blades is presented using multiple synchronous laser scanners and cameras. In a first step, multiple scanners in 1D mode are used which record cross sections at different positions along the rotor blades. By comparing the recorded cross sections with a CAD model of the rotor blade, the deformations in out-of-plane and torsional direction can be derived. In order to ensure that the positions of the cross sections are defined in the coordinate system of the wind energy converter, the nacelle is pre-scanned and a 3D transformation is performed using known coordinates from the manufacturer. To account for the relatively slow movement of the nacelle, it is observed by a photogrammetric camera. The results of the nacelles motion are considered in the analysis of the 1D data. First test recordings were carried out with different measurement frequencies to enable comparisons of accuracy. Furthermore, first results of the cross-section measurements are presented. For the next step the 3D scans will be evaluated which have been acquired using a further instrument simultaneously with the 1D scans. In the same way as before the 3D points will be transferred to the reference system of the nacelle, and then combined with the 1D data.

Multi-image semi-global matching in object space

Semi-Global Matching (SGM) is a widespread algorithm for image matching which is used for very different applications, ranging from real-time applications (e.g. for generating 3D data for driver assistance systems) to aerial image matching. Originally developed for stereo-image matching, several extensions have been proposed to use more than two images within the matching process (multi-baseline matching, multi-view stereo). These extensions still perform the image matching in (rectified) stereo images and combine the pairwise results afterwards to create the final solution. This paper proposes an alternative approach which is suitable for the introduction of an arbitrary number of images into the matching process and utilizes image matching by using non-rectified images. The new method differs from the original SGM method mainly in two aspects: Firstly, the cost calculation is formulated in object space within a dense voxel raster by using the grey (or colour) values of all images instead of pairwise cost calculation in image space. Secondly, the semi-global (path-wise) minimization process is transferred into object space as well, so that the result of semi-global optimization leads to index maps (instead of disparity maps) which directly indicate the 3D positions of the best matches. Altogether, this yields to an essential simplification of the matching process compared to multi-view stereo (MVS) approaches. After a description of the new method, results achieved from two different datasets (close-range and aerial) are presented and discussed.

Image recording systems for close-range photogrammetry

The state-of -the-art in sensor techniques for close-range photogrammetry is characterized by two major trends. In the field of photographic sensors reseau cameras are gradually superseding conventional metric systems and leading to camera systems of very high accuracy. Fully digital imaging systems are also advancing, and particular use is made of CCD cameras with standard video output. Higher resolutions can currently be only achieved if the object is recorded sequentially by a moving CCD sensor in the image space. Additional illumination devices and object targeting have to be involved in the photogrammetric recording process. Image recording systems are only one component of a total system. In order to judge the efficiency of the system the complete object recording and data processing procedure must be taken into account.

Development and Evaluation of a Miniature Trinocular Camera System for Surgical Measurement Applications

Stereo endoscopes for minimally invasive surgery are well established in different medical applications. They may reduce the intervention time due to a better visual support of the surgeon by stereoscopic viewing. Due to rather difficult image analysis conditions in surgical environments, stereoscopic cameras are usually not used for 3D measurements during conventional operations. Especially for computer-assisted surgery, a highly reliable, accurate and dense surface real-time measurement could be of high importance. For that reason, a miniaturised trinocular camera system has been developed and is presented in this paper. To process the trinocular image sequences, a rectification approach for image triplets and an efficient trinocular semi-global matching approach have been implemented. The system is evaluated using two exemplary image triplets with very different matching conditions. The results from the developed trinocular matching are compared with reference models and against the results from OpenCV semi-global block matching using the left and right stereo images only. The proposed trinocular approach consistently provides more complete and error-free point clouds. The greatest improvement due to introducing the third image is achieved without any optimisations such as semi-global matching. The proposed trinocular approach achieves at least the same result quality as the binocular approach, but allows smaller matching windows. Due to fewer arithmetic operations in three small windows than in two large ones, the trinocular analysis outperforms the binocular analysis in terms of speed.ZusammenfassungEntwicklung und Evaluation einesminiaturisierten Dreikamerasystems für chirurgischeAnwendungen. Stereoendoskope sind in einigen Bereichen der minimalinvasiven Chirurgie gut etabliert. Sie können die Interventionszeit durch eine bessere Wahrnehmung aufgrund des stereoskopischen Sehens verkürzen. Bedingt durch recht komplexe Aufnahmebedingungen im chirurgischen Umfeld werden die Stereobilder gewöhnlich nicht für 3D Messungen während konventioneller Operationen eingesetzt. Besonders für die computerassistierte Chirurgie sind eine hoch zuverlässige, genaue sowie dichte Oberflächenmessung in Echtzeit besonders wertvoll. Deshalb wurde ein miniaturisiertes Dreikamerasystem entwickelt, das in dieser Veröffentlichung vorgestellt wird. Zur Verarbeitung der Bildtripel wurden effiziente Verfahren zur Erstellung von Normalbildern und ein trinokulares Semi-Global Matching entwickelt. Mithilfe von zwei exemplarischen Bildtripeln mit sehr unterschiedlichen Bildanalysebedingungen wird das System näher untersucht. Die Ergebnisse des entwickelten trinokularen Matching werden mit Referenzmodellen und den Ergebnissen des OpenCV Semi-Global Block Matching unter Verwendung des linken und rechten Stereobildes verglichen. Der vorgeschlagene trinokulare Ansatz liefert durchweg vollständigere und störungsfreiere Punktwolken. Die größte Verbesserung durch die dritte Kamera wurde ohne Einsatz von Optimierungen, wie z.B. Semi-Global Matching, erzielt. Der vorgeschlagene trinokulare Ansatz erzielt mindestens dieselbe Ergebnisqualität wie der binokulare Ansatz und erlaubt dabei kleinere Matchingfenster. Durch eine verringerte Anzahl arithmetischer Operationen bei drei kleinen Matchingfenstern ergibt sich außerdem ein Geschwindigkeitsvorteil.

Object-Based Semi-global Multi-image Matching

Semi-global matching (SGM) is a widespread algorithm for dense image matching which is used for very different applications, ranging from real-time applications (e.g., for generating 3D data for driver assistance systems) to aerial image matching. Originally developed for stereo-image matching, several extensions have been proposed to use more than two images within the matching process (multi-baseline matching and multi-view stereo). These extensions perform the image matching in (rectified) stereo images and combine the pairwise results afterwards to create the final solution. This paper proposes an alternative approach that is suitable for the introduction of an arbitrary number of unrectified images into the matching process. The new method differs from the original SGM method mainly in two aspects: first, the cost calculation is formulated in object space within a dense voxel raster using the grey (or colour) values of all images instead of pairwise cost calculation in image space. Second, the semi-global (pathwise) minimization process is transferred into object space as well, so that the result of semi-global optimization leads to index maps (instead of disparity maps) which directly indicate the 3D positions of the best matches. Altogether, this yields a simplification of the matching process compared to multi-view stereo (MVS) approaches. After a description of the new method, results achieved from different data sets (close-range and aerial, nadir and oblique) are presented and discussed.ZusammenfassungObjektbasierte Semiglobale Mehrbildzuordnung. Semi-Global Matching (SGM) ist heute ein weit verbreiteter Algorithmus für die dichte Bildzuordnung (dense image matching), der in zahlreichen verschiedenen Anwendungsgebieten zum Einsatz kommt, die von der Echtzeitanwendung (z.B. zur Erzeugung von 3D-Daten in Fahrerassistenzsystemen) bis zur Luftbildauswertung reichen. Ursprünglich für reine Stereobilddaten entwickelt, existieren mittlerweile zahlreiche Erweiterungen, in denen mehr als zwei Bilder für den Matchingprozess verwendet werden (multi-baseline matching, multi-view stereo MVS). In diesen Ansätzen wird das Matching in (rektifizierten) Stereobildern durchgeführt, deren Ergebnisse anschließend zu einem Gesamtergebnis kombiniert werden. In diesem Beitrag wird dagegen ein Verfahren vorgeschlagen, das eine beliebige Anzahl nicht rektifizierter Bilder simultan in den Matchingprozess integriert. Die neue Methode unterscheidet sich vom SGM vor allem in zwei Aspekten: Erstens wird die Kostenfunktion mit einem dichten Voxelraster im Objektraum formuliert, wobei die Grau- oder Farbwerte aller Bilder simultan verwendet werden, anstatt paarweise im Bildraum zu arbeiten. Zweitens wird der semi-globale (pfadbasierte) Minimierungsprozess ebenfalls in den Objektraum übertragen, so dass das Ergebnis der semi-globalen Optimierung zu Indexkarten (im Gegensatz zu Disparitätskarten) führt, die direkt der 3D-Position der besten Zuordnung entsprechen. Zusammengefasst führt dies im Vergleich zu MVS zu einer wesentlichen Vereinfachung des Matchingprozesses. Nach einer Beschreibung des neuen Verfahrens werden die Ergebnisse für unterschiedliche Datensätze (Nahbereichsfall und Luftbildanwendung mit Nadir- und Schrägluftbildern) präsentiert und diskutiert.

Generation of synthetic image sequences for the verification of matching and tracking algorithms for deformation analysis

This paper reports on a method for the generation of synthetic image data for almost arbitrary static or dynamic 3D scenarios. Image data generation is based on pre-defined 3D objects, object textures, camera orientation data and their imaging properties. The procedure does not focus on the creation of photo-realistic images under consideration of complex imaging and reflection models as they are used by common computer graphics programs. In contrast, the method is designed with main emphasis on geometrically correct synthetic images without radiometric impact. The calculation process includes photogrammetric distortion models, hence cameras with arbitrary geometric imaging characteristics can be applied. Consequently, image sets can be created that are consistent to mathematical photogrammetric models to be used as sup-pixel accurate data for the assessment of high-precision photogrammetric processing methods. In the first instance the paper describes the process of image simulation under consideration of colour value interpolation, MTF/PSF and so on. Subsequently the geometric quality of the synthetic images is evaluated with ellipse operators. Finally, simulated image sets are used to investigate matching and tracking algorithms as they have been developed at IAPG for deformation measurement in car safety testing.