Simon Kriegel

Efficient next-best-scan planning for autonomous 3D surface reconstruction of unknown objects

This work focuses on autonomous surface reconstruction of small-scale objects with a robot and a 3D sensor. The aim is a high-quality surface model allowing for robotic applications such as grasping and manipulation. Our approach comprises the generation of next-best-scan (NBS) candidates and selection criteria, error minimization between scan patches and termination criteria. NBS candidates are iteratively determined by a boundary detection and surface trend estimation of the acquired model. To account for both a fast and high-quality model acquisition, that candidate is selected as NBS, which maximizes a utility function that integrates an exploration and a mesh-quality component. The modeling and scan planning methods are evaluated on an industrial robot with a high-precision laser striper system. While performing the new laser scan, data are integrated on-the-fly into both, a triangle mesh and a probabilistic voxel space. The efficiency of the system in fast acquisition of high-quality 3D surface models is proven with different cultural heritage, household and industrial objects.

A surface-based Next-Best-View approach for automated 3D model completion of unknown objects

The procedure of manually generating a 3D model of an object is very time consuming for a human operator. Next-best- view (NBV) planning is an important aspect for automation of this procedure in a robotic environment. We propose a surface-based NBV approach, which creates a triangle surface from a real-time data stream and determines viewpoints similar to human intuition. Thereby, the boundaries in the surface are detected and a quadratic patch for each boundary is estimated. Then several viewpoint candidates are calculated, which look perpendicular to the surface and overlap with previous sensor data. A NBV is selected with the goal to fill areas which are occluded. This approach focuses on the completion of a 3D model of an unknown object. Thereby, the search space for the viewpoints is not restricted to a cylinder or sphere. Our NBV determination proves to be very fast, and is evaluated in an experiment on test objects, applying an industrial robot and a laser range scanner.

Combining object modeling and recognition for active scene exploration

Active scene exploration incorporates object recognition methods for analyzing a scene of partially known objects and exploration approaches for autonomous modeling of unknown parts. In this work, recognition, exploration, and planning methods are extended and combined in a single scene exploration system, enabling advanced techniques such as multi-view recognition from planned view positions and iterative recognition by integration of new objects from a scene. Here, a geometry based approach is used for recognition, i.e. matching objects from a database. Unknown objects are autonomously modeled and added to the recognition database. Next-Best-View planning is performed both for recognition and modeling. Moreover, 3D measurements are merged in a Probabilistic Voxel Space, which is utilized for planning collision free paths, minimal occlusion views, and verifying the poses of the recognized objects against all previous information. Experiments on an industrial robot with attached 3D sensors are shown for scenes with household and industrial objects.

Next-best-scan planning for autonomous 3D modeling

We present a next-best-scan (NBS) planning approach for autonomous 3D modeling. The system successively completes a 3D model from complex shaped objects by iteratively selecting a NBS based on previously acquired data. For this purpose, new range data is accumulated in-the-loop into a 3D surface (streaming reconstruction) and new continuous scan paths along the estimated surface trend are generated. Further, the space around the object is explored using a probabilistic exploration approach that considers sensor uncertainty. This allows for collision free path planning in order to completely scan unknown objects. For each scan path, the expected information gain is determined and the best path is selected as NBS. The presented NBS approach is tested with a laser striper system, attached to an industrial robot. The results are compared to state-of-the-art next-best-view methods. Our results show promising performance with respect to completeness, quality and scan time.

Information-gain view planning for free-form object reconstruction with a 3d ToF camera

Active view planning for gathering data from an unexplored 3D complex scenario is a hard and still open problem in the computer vision community. In this paper, we present a general task-oriented approach based on an information-gain maximization that easily deals with such a problem. Our approach consists of ranking a given set of possible actions, based on their task-related gains, and then executing the best-ranked action to move the required sensor. An example of how our approach behaves is demonstrated by applying it over 3D raw data for real-time volume modelling of complex-shaped objects. Our setting includes a calibrated 3D time-of-flight (ToF) camera mounted on a 7 degrees of freedom (DoF) robotic arm. Noise in the sensor data acquisition, which is too often ignored, is here explicitly taken into account by computing an uncertainty matrix for each point, and refining this matrix each time the point is seen again. Results show that, by always choosing the most informative view, a complete model of a 3D free-form object is acquired and also that our method achieves a good compromise between speed and precision.

Multi-view orientation estimation using Bingham mixture models

This paper describes a multi-view pose estimation system, that is exploiting the mobility of a depth sensor through mounting it onto a robotic manipulator. Given a pose estimation algorithm that performs feature extraction and matching to a model database, we investigate the probabilistic modeling of the pose space as well as the measurement uncertainty, to be used in a sequential state estimation approach. Uncertainties in 3d position can be modeled in a parametric way by 3d Gaussians, but the space of rotations in 3d - the special orthogonal group SO(3) - requires approaches from directional statistics. A convenient representation for orientations are unit quaternions over which the Bingham distribution defines a parametric probability density function. The Bingham distribution also correctly accounts for the sign symmetry of orientation quaternions and leave degrees of freedom unconstrained (which is especially useful if an object is rotationally symmetric, with no unique quaternion describing its orientation). In our experiments we test different sequential fusion methods, optimize their parameters, and investigate how the derived filter performs in a case with high uncertainties.

Autonomous pick and place operations in industrial production

Pick and place applications are very common in industrial environments. Autonomous execution of this task is desirable, as it is simple and repetitive for a human. To this end, we employ a mobile robot containing a light weight robot (LWR) arm, a pan-tilt unit (PTU) and a multitude of sensors (see Fig. 1).

Experience-based optimization of robotic perception

As the performance of key perception tasks heavily depends on their parametrization, deploying versatile robots to different application domains will also require a way to tune these changing scenarios by their operators. As many of these tunings are found by trial and error basically by experts as well, and the quality criteria change from application to application, we propose a Pipeline Optimization Framework that helps overcoming lengthy setup times by largely automating this process. When deployed, fine-tuning optimizations as presented in this paper can be initiated on pre-recorded data, dry runs, or automatically during operation. Here, we quantified the performance gains for two crucial modules based on ground truth annotated data. We release our challenging THR dataset, including evaluation scenes for two application scenarios.

Monte Carlo Registration and Its Application with Autonomous Robots

This work focuses on Monte Carlo registration methods and their application with autonomous robots. A streaming and an offline variant are developed, both based on a particle filter. The streaming registration is performed in real-time during data acquisition with a laser striper allowing for on-the-fly pose estimation. Thus, the acquired data can be instantly utilized, for example, for object modeling or robot manipulation, and the laser scan can be aborted after convergence. Curvature features are calculated online and the estimated poses are optimized in the particle weighting step. For sampling the pose particles, uniform, normal, and Bingham distributions are compared. The methods are evaluated with a high-precision laser striper attached to an industrial robot and with a noisy Time-of-Flight camera attached to service robots. The shown applications range from robot assisted teleoperation, over autonomous object modeling, to mobile robot localization.

On-the-fly particle filter registration for laser data

This work is focused on streaming particle filter registration of surface models such as homogeneous triangle meshes and point clouds. Part of the approach is a streaming curvature feature calculation. The investigated approach utilizes a particle filter to incrementally update pose estimates during data acquisition. The method is evaluated in real data experiments with a high-precision laser striper system attached to an industrial robot. During the laser scan, the data is integrated on-the-fly in order to calculate features and based on these to estimate the objects pose. Experiments show the methods competitiveness in accuracy and reliability compared to state-of-the-art offline algorithms.