Tobias Fromm

Robust 3D object modeling with a low-cost RGBD-sensor and AR-markers for applications with untrained end-users

Abstract An approach for generating textured 3D models of objects without the need for complex infrastructure such as turn-tables or high-end sensors on precisely controlled rails is presented. The method is inexpensive as it uses only a low-cost RGBD sensor, e.g., Microsoft Kinect or ASUS Xtion, and Augmented Reality (AR) markers printed on paper sheets. The sensor can be moved by hand by an untrained person and the AR-markers can be arbitrarily placed in the scene, thus allowing the modeling of objects of a large range of sizes. Due to the use of the simple AR markers, the method is significantly more robust than just using the RGBD sensor or a monocular camera alone and it hence avoids the typical need for manual post-processing of alternative approaches like Kinect-Fusion, 123D Catch, Photosynth, or similar. This article has two main contributions: First, the development of a simple, inexpensive method for the quick and easy digitization of physical objects is presented. Second, the development of an uncertainty model for AR-marker pose estimation is introduced. The latter is of interest beyond the object modeling application presented here. The uncertainty model is used in a graph-based relaxation method to improve model-consistency. Realistic modeling of various objects, such as parcels, sport balls, coffee sacks, human dolls, etc., is experimentally demonstrated. Good model-accuracy is shown for several ground-truth objects with simple geometries and known dimensions. Furthermore, it is shown that the models obtained using the uncertainty model have fewer errors than the ones obtained without it.

No More Heavy Lifting: Robotic Solutions to the Container Unloading Problem

This article discusses the scientifically and industrially important problem of automating the process of unloading goods from standard shipping containers. We outline some of the challenges barring further adoption of robotic solutions to this problem, ranging from handling a vast variety of shapes, sizes, weights, appearances, and packing arrangements of the goods, through hard demands on unloading speed and reliability, to ensuring that fragile goods are not damaged. We propose a modular and reconfigurable software framework in an attempt to efficiently address some of these challenges. We also outline the general framework design and the basic functionality of the core modules developed. We present two instantiations of the software system on two different fully integrated demonstrators: (1) coping with an industrial scenario, i.e., the automated unloading of coffee sacks with an already economically interesting performance; and (2) a scenario used to demonstrate the capabilities of our scientific and technological developments in the context of medium- to long-term prospects of automation in logistics. We performed evaluations that allowed us to summarize several important lessons learned and to identify future directions of research on autonomous robots for the handling of goods in logistics applications.

Physics-based damage-aware manipulation strategy planning using Scene Dynamics Anticipation

We present a damage-aware planning approach which determines the best sequence to manipulate a number of objects in a scene. This works on task-planning level, abstracts from motion planning and anticipates the dynamics of the scene using a physics simulation. Instead of avoiding interaction with the environment, we take unintended motion of other objects into account and plan manipulation sequences which minimize the potential damage. Our method can also be used as a validation measure to judge planned motions for their feasibility in terms of damage avoidance. We evaluate our approach on one industrial scenario (autonomous container unloading) and one retail scenario (shelf replenishment).

Robotic bridge inspection within strategic flood evacuation planning

Bridges are fragile transport infrastructure elements which require special attention in evacuation planning within flood disaster scenarios. In this paper we present work on a bridge inspection procedure with a marine vehicle in which a bridge statics analysis expert assesses the condition of the bridge from a safe operating station. Given bridge construction knowledge beforehand, our work shows how the assessment can benefit from an unmanned marine vehicle which provides information about the current structural condition of the bridge. Due to quality degradation of standard RGB camera readings caused by varying visibility conditions under water, visual information is captured from an imaging sonar. In contrary to conventional approaches, the assessment of bridges gains quality by considering live stream of sonar images which reveal bridge structure information even under rough water conditions. Additionally, an image registration method is applied which creates a map of the live stream and provides a valuable information source for further bridge assessment. We describe the deployment of the system in a real test field.

Robotic bridge statics assessment within strategic flood evacuation planning using low-cost sensors

Scenario: A rescue team needs to cross a partially damaged bridge in a flooded area. It is unknown whether the construction is still able to carry a vehicle. Assessing the constructions integrity can be accomplished by the analysis of the bridges eigenfrequencies. Rather than using proprietary expensive Vibration Measurement Systems (VMS) we propose to utilize off-the-shelf smartphones as sensors - which still require to be placed at the spot on the bridge best suited for picking up vibrations. Within this work, we use an Unmanned Ground Vehicle (UGV) featuring a robotic manipulator. It allows a non-technician operator to optimally place the device semi- automatically. We evaluate our approach in a real-life scenario. Demo video: https://youtu.be/u_3pe0nZ5tw

Efficient continuous system integration and validation for deep-sea robotics applications

Deep-sea operations of remotely-operated vehicles (ROV) need robust testing and deployment strategies beyond the traditional pre-deployment validation on real hardware. Seamless integration of simulated components into the validation pipeline allows for rapid development of components and validation under controlled conditions. We describe the benefits arising from such a continuous integration and validation approach as well as an example setup in the EU project DexROV.