Jan Boehm

Automated 3D Reconstruction of Interiors from Point Clouds

We present a new technique for the fully automated 3D modelling of indoor environments from a point cloud. The point cloud is acquired with several scans and is afterwards processed in order to segment planar structures, which have a noticeable architectural meaning (floor, ceiling and walls) in the interior. The basic approach to data segmentation is plane sweeping based on a hypothesis-and-test strategy. From the segmentation results, the ground plan is created through cell decomposition by trimming the two-dimensional ground space using half-space primitives. An extension in height of the ground contours makes the generation of the 3D model possible. The so-reconstructed indoor model is saved in CAD format for analysis and further applications or, simply, as a record of the interior geometry.

Automatic Geometry Generation from Point Clouds for BIM

The need for better 3D documentation of the built environment has come to the fore in recent years, led primarily by city modelling at the large scale and Building Information Modelling (BIM) at the smaller scale. Automation is seen as desirable as it removes the time-consuming and therefore costly amount of human intervention in the process of model generation. BIM is the focus of this paper as not only is there a commercial need, as will be shown by the number of commercial solutions, but also wide research interest due to the aspiration of automated 3D models from both Geomatics and Computer Science communities. The aim is to go beyond the current labour-intensive tracing of the point cloud to an automated process that produces geometry that is both open and more verifiable. This work investigates what can be achieved today with automation through both literature review and by proposing a novel point cloud processing process. We present an automated workflow for the generation of BIM data from 3D point clouds. We also present quality indicators for reconstructed geometry elements and a framework in which to assess the quality of the reconstructed geometry against a reference.

Photogrammetric calibration and accuracy evaluation of a cross-pattern stripe projector

Presently there is a growing demand for fast and precise 3D computer vision systems for a wide variety of industrial applications like reverse engineering, quality control and industrial gauging. One important aspect of any vision system is the data acquisition. If the principle of triangulation is used the correspondence problem is to be solved. The coded light approach offers a fast way to overcome this problem and to provide dense range data. In order to get high accuracy range images the system needs to be calibrated. In this paper, we compare two calibration techniques: polynomial depth calibration and photogrammetric calibration. We have carried out both methods independently. To obtain results about the accuracy in object space, we measured the surface of a plane- table.

Image selection in photogrammetric multi-view stereo methods for metric and complete 3D reconstruction

Multi-View Stereo (MVS) as a low cost technique for precise 3D reconstruction can be a rival for laser scanners if the scale of the model is resolved. A fusion of stereo imaging equipment with photogrammetric bundle adjustment and MVS methods, known as photogrammetric MVS, can generate correctly scaled 3D models without using any known object distances. Although a huge number of stereo images (e.g. 200 high resolution images from a small object) captured of the object contains redundant data that allows detailed and accurate 3D reconstruction, the capture and processing time is increased when a vast amount of high resolution images are employed. Moreover, some parts of the object are often missing due to the lack of coverage of all areas. These problems demand a logical selection of the most suitable stereo camera views from the large image dataset. This paper presents a method for clustering and choosing optimal stereo or optionally single images from a large image dataset. The approach focusses on the two key steps of image clustering and iterative image selection. The method is developed within a software application called Imaging Network Designer (IND) and tested by the 3D recording of a gearbox and three metric reference objects. A comparison is made between IND and CMVS, which is a free package for selecting vantage images. The final 3D models obtained from the IND and CMVS approaches are compared with datasets generated with an MMDx Nikon Laser scanner. Results demonstrate that IND can provide a better image selection for MVS than CMVS in terms of surface coordinate uncertainty and completeness.

Towards fully automatic reliable 3D acquisition

This paper describes a novel system for accurate 3D digitization of complex objects. Its main novelties can be seen in the new approach, which brings together different systems and tools in a unique platform capable of automatically generating an accurate and complete model for an object of interest. This is performed through generating an approximate model of the object, designing a stereo imaging network for the object with this model and capturing the images at the designed postures through exploiting an inverse kinematics method for a non-standard six degree of freedom robot. The images are then used for accurate and dense 3D reconstruction using photogrammetric multi-view stereo method in two modes, including resolving scale with baseline and with control points. The results confirm the feasibility of using Particle Swarm Optimization in solving inverse kinematics for this non-standard robot. The system provides this opportunity to test the effect of incidence angle on imaging network design and shows that the matching algorithms work effectively for incidence angle of 10?. The accuracy of the final point cloud generated with the system was tested in two modes through a comparison with a dataset generated with a close range 3D colour laser scanner. We designed a 6-DOF robot for accurate and dense 3D reconstruction using images.Particle Swarm Optimization was evaluated for inverse kinematic of the robot.A software package, Imaging Network Designer, was tested for this robot.The accuracy of the robot for 3D reconstruction is around 200 µ m .

Automatic Image Selection in Photogrammetric Multi-view Stereo Methods

This paper brings together a team of specialists in optical metrology, museum curation, collection digitization and 3D development to describe and illustrate by example a method for the selection of the most suitable camera views, vantage viewpoints, from a large image dataset intended for metric 3D artefact reconstruction. The presented approach is capable of automatically identifying and processing the most appropriate images from a multi-image photogrammetric network captured by an imaging specialist. The aim is to produce a 3D model suited to a wide range of museum uses, including visitor interactives. The approach combines off-the-shelf imaging equipment with rigorous photogrammetric bundle adjustment and multi-view stereo (MVS), supported by an image selection process that is able to take into account range-related and visibility-related constraints. The paper focusses on the two key steps of image clustering and iterative image selection. The developed method is illustrated by the 3D recording of four ancient Egyptian artefacts from the Petrie Museum of Egyptian Archaeology at UCL, with an analysis taking into account completeness, coordination uncertainty and required number of images. Comparison is made against the baseline of the established CMVS (Clustering Views for Multi-view Stereo), which is a free package for selecting vantage images within a huge image collection. For the museum, key outputs from the 3D recording process are visitor interactives which are built around high quality textured mesh models. The paper therefore considers the quality of the output from each process as input to texture model generation. Results demonstrate that whilst both methods can provide high quality records, our new method, Image Network Designer (IND), can provide a better image selection for MVS than CMVS in terms of coordination uncertainty and completeness of the final model for the museum recording of artefacts. Furthermore, the improvements gained, particularly in model completeness, minimise the significant overhead in mesh editing needed to provide a more direct and economical route to 3D model output.

Accuracy Investigation for Structured-light Based Consumer 3D Sensors

This work focuses on the performance investigation of consumer 3D sensors with respect to their repeatability and accuracy. It explores currently available sensors based on the 3D sensing technology developed by PrimeSense and introduced to the market in the form of the Microsoft Kinect. Accuracy and repeatability can be crucial criteria for the use of these sensors outside their intended use for home entertainment. The test strategies for the study are motivated by the VDI/VDE 2634 guideline. At the core of the work is the investigation of several units of the Asus Xtion Pro and the PrimeSense Developer Kit and a comparison of their performance. Altogether eighteen sensor units were tested. The results of the proposed test scenario for the sensor units show excellent repeatability at a few millimetres. However, absolute accuracy is worse and can be up to a few centimetres. Sensor performance varies greatly both for sensors of the same manufacturer and in-between manufacturers.

Curvature-based range image classification for object recognition

This work focuses on the extraction of features from dense range images for object recognition. The object recognition process is based on a CAD model of the subject. Curvature information derived from the CAD model is used to support the feature extraction process. We perform a curvature based classification of the range image to achieve a segmentation into meaningful surface patches, which are later to be matched with the surfaces of the CAD model.

Experimental measurement system for industrial inspection of 3D parts

A research group at the University of Stuttgart has set up an experimental measurement robot for industrial close range inspection. During a test run, the feasibility of a multi- sensor/actor system has been shown. The system uses optical sensors to perform different tasks including object recognition, localization and gauging. It is a step towards systems which are able to inspect and gauge several parts from a set of parts stored in a 3D model database. This paper describes the results which have been obtained so far and were demonstrated during a test run. It then focuses on our latest developments concerning 3D data acquisition, registration, segmentation, model generation from CAD data and object recognition.

A webcam photogrammetric method for robot calibration

This paper describes a strategy for accurate robot calibration using close range photogrammetry. A 5-DoF robot has been designed for placement of two web cameras relative to an object. To ensure correct camera positioning, the robot is calibrated using the following strategy. First, a Denavit-Hartenberg method is used to generate a general kinematic robot model. A set of reference frames are defined relative to each joint and each of the cameras, transformation matrices are then produced to represent change in position and orientation between frames in terms of joint positions and unknown parameters. The complete model is extracted by multiplying these matrices. Second, photogrammetry is used to estimate the postures of both cameras. A set of images are captured of a calibration fixture from different robot poses. The camera postures are then estimated using bundle adjustment. Third, the kinematic parameters are estimated using weighted least squares. For each pose a set of equations are extracted from the model and the unknown parameters are estimated in an iterative procedure. Finally these values are substituted back into the original model. This final model is tested using forward kinematics by comparing the model’s predicted camera postures for given joint positions to the values obtained through photogrammetry. Inverse kinematics is performed using both least squares and particle swarm optimisation and these techniques are contrasted. Results demonstrate that this photogrammetry approach produces a reliable and accurate model of the robot that can be used with both least squares and particle swarm optimisation for robot control.