Dirk Holz

Real-time plane segmentation using RGB-D cameras

Real-time 3D perception of the surrounding environment is a crucial precondition for the reliable and safe application of mobile service robots in domestic environments. Using a RGB-D camera, we present a system for acquiring and processing 3D (semantic) information at frame rates of up to 30Hz that allows a mobile robot to reliably detect obstacles and segment graspable objects and supporting surfaces as well as the overall scene geometry. Using integral images, we compute local surface normals. The points are then clustered, segmented, and classified in both normal space and spherical coordinates. The system is tested in different setups in a real household environment. The results show that the system is capable of reliably detecting obstacles at high frame rates, even in case of obstacles that move fast or do not considerably stick out of the ground. The segmentation of all planes in the 3D data even allows for correcting characteristic measurement errors and for reconstructing the original scene geometry in far ranges.

Robust 3D-mapping with time-of-flight cameras

Time-of-flight cameras constitute a smart and fast technology for 3D perception but lack in measurement precision and robustness. The authors present a comprehensive approach for 3D environment mapping based on this technology. Imprecision of depth measurements are properly handled by calibration and application of several filters. Robust registration is performed by a novel extension to the Iterative Closest Point algorithm. Remaining registration errors are reduced by global relaxation after loop-closure and surface smoothing. A laboratory ground truth evaluation is provided as well as 3D mapping experiments in a larger indoor environment.

Fast Range Image Segmentation and Smoothing Using Approximate Surface Reconstruction and Region Growing

Decomposing sensory measurements into relevant parts is a fundamental prerequisite for solving complex tasks, e.g., in the field of mobile manipulation in domestic environments. In this paper, we present a fast approach to surface reconstruction in range images by means of approximate polygonal meshing. The obtained local surface information and neighborhoods are then used to 1) smooth the underlying measurements, and 2) segment the image into planar regions and other geometric primitives. An evaluation using publicly available data sets shows that our approach does not rank behind state-of-the-art algorithms while allowing to process range images at high frame rates.

Registration with the Point Cloud Library: A Modular Framework for Aligning in 3-D

Registration is an important step when processing three-dimensional (3-D) point clouds. Applications for registration range from object modeling and tracking, to simultaneous localization and mapping (SLAM). This article presents the open-source point cloud library (PCL) and the tools available for point cloud registration. The PCL incorporates methods for the initial alignment of point clouds using a variety of local shape feature descriptors, as well as methods for refining initial alignments using different variants of the well-known iterative closest point (ICP) algorithm. This article provides an overview on registration algorithms, usage examples of their PCL implementations, and tips for their application. Since the choice and parameterization of the right algorithm for a particular type of data is one of the biggest problems in 3-D point cloud registration, we present three complete examples of data (and applications) and the respective registration pipeline in the PCL. These examples include dense red-green-blue-depth (RGB-D) point clouds acquired by consumer color and depth cameras, high-resolution laser scans from commercial 3-D scanners, and low-resolution sparse point clouds captured by a custom lightweight 3-D scanner on a microaerial vehicle (MAV).

Johnny: An Autonomous Service Robot for Domestic Environments

In this article we describe the architecture, algorithms and real-world benchmarks performed by Johnny Jackanapes, an autonomous service robot for domestic environments. Johnny serves as a research and development platform to explore, develop and integrate capabilities required for real-world domestic service applications. We present a control architecture which allows to cope with various and changing domestic service robot tasks. A software architecture supporting the rapid integration of functionality into a complete system is as well presented. Further, we describe novel and robust algorithms centered around multi-modal human robot interaction, semantic scene understanding and SLAM. Evaluation of the complete system has been performed during the last years in the RoboCup@Home competition where Johnnys outstanding performance led to successful participation. The results and lessons learned of these benchmarks are explained in more detail.

Mobile bin picking with an anthropomorphic service robot

Grasping individual objects from an unordered pile in a box has been investigated in static scenarios so far. In this paper, we demonstrate bin picking with an anthropomorphic mobile robot. To this end, we extend global navigation techniques by precise local alignment with a transport box. Objects are detected in range images using a shape primitive-based approach. Our approach learns object models from single scans and employs active perception to cope with severe occlusions. Grasps and arm motions are planned in an efficient local multiresolution height map. All components are integrated and evaluated in a bin picking and part delivery task.

Multi-frequency Phase Unwrapping for Time-of-Flight cameras

Time-of-Flight (ToF) cameras gain depth information by emitting amplitude-modulated near-infrared light and measuring the phase shift between the emitted and the reflected signal. The phase shift is proportional to the objects distance modulo the wavelength of the modulation frequency. This results in a distance ambiguity. Distances larger than the wavelength are wrapped into the sensors non-ambiguity range and cause spurious distance measurements. We apply Phase Unwrapping to reconstruct these wrapped measurements. Our approach is based on a probabilistic graphical model. We use loopy belief propagation to detect and infer the position of wrapped measurements. Besides depth discontinuities, our method utilizes multiple modulation frequencies to identify wrapped measurements. In experiments, we show that wrapped measurements are identified and corrected, even in situations where the scene shows steep slopes in the depth measurements.

Sancta simplicitas - on the efficiency and achievable results of SLAM using ICP-based incremental registration

This paper presents an efficient combination of algorithms for SLAM in dynamic environments. The overall approach is based on range image registration using the ICP algorithm. Different extensions to this algorithm are used to incrementally construct point models of the robots workspace. A simple heuristic allows for determining which points in a newly acquired range image are already contained in the point model and for adding only those points that provide new information. Furthermore, the means for dealing with environment dynamics are presented which allow for continuously conducting SLAM and updating the point model according to changes in a dynamic environment. The achievable results of the overall approach are compared to Rao-Blackwellized Particle Filters as a state-of-the-art solution to the SLAM problem and evaluated using a recently published benchmark by Burgard et al. (2009).

Multimodal obstacle detection and collision avoidance for micro aerial vehicles

Reliably perceiving obstacles and avoiding collisions is key for the fully autonomous application of micro aerial vehicles (MAVs), Limiting factors for increasing autonomy and complexity of MAVs are limited onboard sensing and limited onboard processing power. In this paper, we propose a complete system with a multimodal sensor setup for omnidirectional obstacle perception. We developed a lightweight 3D laser scanner and visual obstacle detection using wide-angle stereo cameras. Detected obstacles are aggregated in egocentric grid maps. We implemented a fast reactive collision avoidance approach for safe operation in the vicinity of structures like buildings or vegetation.

RoboCup@Home: Demonstrating Everyday Manipulation Skills in RoboCup@Home

As benchmarking robotics research is inherently difficult, robot competitions are increasingly popular. They bring together researchers, students, and enthusiasts who are in the pursuit of a technological challenge. Prominent examples for such competitions include the Defense Advanced Research Projects Agencys (DARPAs) Grand and Urban Challenges [1], the International Aerial Robotics Competition (IARC) [2], the European Land-Robot Trial (ELROB) [3], and, last but not least, RoboCup [4], [5].