Christian Lenz

NimbRo Rescue: Solving Disaster-response Tasks with the Mobile Manipulation Robot Momaro

Robots that solve complex tasks in environments too dangerous for humans to enter are desperately needed, e.g., for search and rescue applications. We describe our mobile manipulation robot Momaro, with which we participated successfully in the DARPA Robotics Challenge. It features a unique locomotion design with four legs ending in steerable wheels, which allows it both to drive omnidirectionally and to step over obstacles or climb. Furthermore, we present advanced communication and teleoperation approaches, which include immersive three-dimensional 3D visualization, and 6D tracking of operator head and arm motions. The proposed system is evaluated in the DARPA Robotics Challenge, the DLR SpaceBot Cup Qualification, and lab experiments. We also discuss the lessons learned from the competitions.

NimbRo picking: Versatile part handling for warehouse automation

Part handling in warehouse automation is challenging if a large variety of items must be accommodated and items are stored in unordered piles. To foster research in this domain, Amazon holds picking challenges. We present our system which achieved second and third place in the Amazon Picking Challenge 2016 tasks. The challenge required participants to pick a list of items from a shelf or to stow items into the shelf. Using two deep-learning approaches for object detection and semantic segmentation and one item model registration method, our system localizes the requested item. Manipulation occurs using suction on points determined heuristically or from 6D item model registration. Parametrized motion primitives are chained to generate motions. We present a full-system evaluation during the APC 2016 and component-level evaluations of the perception system on an annotated dataset.

High-precision 3D detection and reconstruction of grapes from laser range data for efficient phenotyping based on supervised learning

Display Omitted Fully-automated 3D reconstruction of grape bunches for phenotyping.Combining supervised learning and object recognition methods.Parameter initialization and optimization based on known statistical values.In-depth evaluation of all steps of the pipeline. In this contribution, we present an automated approach to the phenotyping of grape bunches. To do so, our method analyses high-resolution sensor data taken from grape bunches and generates complete 3D reconstructions of the observed grape bunches. We extend a previous work from our group to earlier development stages with mostly visible stem structure, using an enhanced pre-classification of the sensor data into specific categories, i.e., berries and stems, yielding high precision and recall rates for the reconstruction of the berries of more than 98% and 94%, respectively. The same quality of results can be achieved by training a classification model on one grape bunch and applying it to the other grape bunches. Furthermore, we describe important observations concerning parameter initialization and optimization techniques resulting in a guideline for people working in the area.

Supervised Autonomy for Exploration and Mobile Manipulation in Rough Terrain with a Centaur-Like Robot

Planetary exploration scenarios illustrate the need for autonomous robots that are capable to operate in unknown environments without direct human interaction. At the DARPA Robotics Challenge, we demonstrated that our Centaur-like mobile manipulation robot Momaro can solve complex tasks when teleoperated. Motivated by the DLR SpaceBot Cup 2015, where robots should explore a Mars-like environment, find and transport objects, take a soil sample, and perform assembly tasks, we developed autonomous capabilities for Momaro. Our robot perceives and maps previously unknown, uneven terrain using a 3D laser scanner. Based on the generated height map, we assess drivability, plan navigation paths, and execute them using the omnidirectional drive. Using its four legs, the robot adapts to the slope of the terrain. Momaro perceives objects with cameras, estimates their pose, and manipulates them with its two arms autonomously. For specifying missions, monitoring mission progress, on-the-fly reconfiguration, and teleoperation, we developed a ground station with suitable operator interfaces. To handle network communication interruptions and latencies between robot and ground station, we implemented a robust network layer for the ROS middleware. With the developed system, our team NimbRo Explorer solved all tasks of the DLR SpaceBot Camp 2015. We also discuss the lessons learned from this demonstration.

Team NimbRo Rescue at DARPA Robotics Challenge Finals

Summary form only given. The video shows the compacted first-day run of team NimbRo Rescue at the DARPA Robotics Challenge Finals in Pomona, CA. It features the mobile manipulation robot Momaro which has a flexible base with four legs that end in steerable wheels. Momaro can drive omnidirectionally and step over obstacles. The robot is equipped with an anthropomorphic upper body with two 7 DoF arms that end in four-finger grippers. A 3D laser scanner and multiple cameras capture the environment. Operator interfaces include a steering wheel and a gas pedal for car driving, a joystick for omnidirectional locomotion, and a head-mounted 3D immersive display with two 6 DoF magnetic hand trackers for solving complex manipulation tasks. Through Momaro, our team solved seven of the eight tasks of the DARPA Robotics Challenge: driving a car, egressing the car, opening a door, turning a valve, cutting a hole into a drywall, traversing debris, and a surprise task, which was to operate a big switch. All this was done in only 34 minutes. Team NimbRo Rescue was the best European team, coming in 4th in the overall ranking.

DRC Team NimbRo Rescue: Perception and Control for Centaur-Like Mobile Manipulation Robot Momaro

Robots that solve complex tasks in environments too dangerous for humans to enter are desperately needed, e.g. for search and rescue applications. We describe our mobile manipulation robot Momaro, with which we participated successfully in the DARPA Robotics Challenge. It features a unique locomotion design with four legs ending in steerable wheels, which allows it both to drive omnidirectionally and to step over obstacles or climb. Furthermore, we present advanced communication and teleoperation approaches, which include immersive 3D visualization, and 6D tracking of operator head and arm motions. The proposed system is evaluated in the DARPA Robotics Challenge, the DLR SpaceBot Camp 2015, and lab experiments. We also discuss the lessons learned from the competitions and present initial steps towards autonomous operator assistance functions.

Team NimbRo at MBZIRC 2017: Autonomous valve stem turning using a wrench: SCHWARZ et al.

The Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2017 has defined ambitious new benchmarks to advance the state-of-the-art in autonomous operation of ground-based and flying robots. In this article, we describe our winning entry to MBZIRC Challenge 2: the mobile manipulation robot Mario. It is capable of autonomously solving a valve manipulation task using a wrench tool detected, grasped, and finally employed to turn a valve stem. Marios omnidirectional base allows both fast locomotion and precise close approach to the manipulation panel. We describe an efficient detector for medium-sized objects in 3D laser scans and apply it to detect the manipulation panel. An object detection architecture based on deep neural networks is used to find and select the correct tool from grayscale images. Parametrized motion primitives are adapted online to percepts of the tool and valve stem in order to turn the stem. We report in detail on our winning performance at the challenge and discuss lessons learned.

Supervised Autonomy for Exploration and Mobile Manipulation in Rough Terrain

Planetary exploration scenarios illustrate the need for robots that are capable to operate in unknown environments without direct human interaction. Motivated by the DLR SpaceBot Cup 2015, where robots should explore a Mars-like environment, find and transport objects, take a soil sample, and perform assembly tasks, we developed autonomous capabilities for our mobile manipulation robot Momaro. The robot perceives and maps previously unknown, uneven terrain using a 3D laser scanner. We assess drivability and plan navigation for the omnidirectional drive. Using its four legs, Momaro adapts to the slope of the terrain. It perceives objects with cameras, estimates their pose, and manipulates them with its two arms autonomously. For specifying missions, monitoring mission progress, and on-the-fly reconfiguration, we developed suitable operator interfaces. With the developed system, our team NimbRo Explorer solved all tasks of the DLR SpaceBot Camp 2015.