Hannes Bistry

A cloud computing approach to complex robot vision tasks using smart camera systems

In this paper we show our work on enabling service robot systems to distribute parts of the image processing functions to different off-board computer systems in the working environment of the robot. Thus complex algorithms can be carried out on high performance systems circumventing the restrictions considering space and power consumption that a mobile platform imposes. As high resolution cameras provide a huge amount of image data and the bandwidth of a wireless network connection is strongly limited, we are using intelligent camera systems on the mobile robot platform to execute parts of the image processing functions directly on the robot. This way only preprocessed image information will be transmitted instead of raw image data. We are using a flexible modular software framework that allows us to split image processing tasks into a pipeline of modular functions that can run on different systems. We show how our approach can be used to enable a service robot system to speed up high resolution SIFT-based object detection.

Architecture and Software Design for a Service Robot in an Elderly-Care Scenario

ABSTRACT Systems for ambient assisted living (AAL) that integrate service robots with sensor networks and user monitoring can help elderly people with their daily activities, allowing them to stay in their homes and live active lives for as long as possible. In this paper, we outline the AAL system currently developed in the European project Robot-Era, and describe the engineering aspects and the service-oriented software architecture of the domestic robot, a service robot with advanced manipulation capabilities. Based on the robot operating system (ROS) middleware, our software integrates a large set of advanced algorithms for navigation, perception, and manipulation. In tests with real end users, the performance and acceptability of the platform are evaluated.

A smart interface-unit for the integration of pre-processed laser range measurements into robotic systems and sensor networks

In this paper we propose a novel method of connecting laser range finders to a service robot or to an arbitrary sensor network. The developed unit can be connected to two laser range finders and provides a standard Ethernet port. Autonomous computing power enables the unit to carry out data manipulation tasks. Every device in the network will be able to access both the raw measurement data and pre- processed measurements calculated by the smart interface unit. The paper will discuss the technical aspects of the developed unit as well as the implemented novel processing algorithms. These algorithms are specific to tasks many service robots have to carry out. We present a use case where the developed unit has been applied successfully.

PEIS, MIRA, and ROS: Three frameworks, one service robot — A tale of integration

Designing and building a mobile service robot is a complex task, involving many design decisions regarding hardware components and software architecture. A robot design always constitutes a delicate compromise between capabilities, performance, safety, reliability and costs. Only the availability of high-quality robot middleware frameworks like ROS has allowed several university groups to succeed in this task. In this paper, we describe the hardware and software design of the Domestic Robot of project Robot-Era, a medium cost mobile manipulation platform designed to help elderly people with their daily routines in an assisted living environment. Assembled from standard hardware components, the robot is characterized by its services-based software architecture, which integrates three different software frameworks, exploiting the strengths of each. We highlight the development process as well as some problems that occurred during the design, implementation and testing phases.

Task oriented control of smart camera systems in the context of mobile service robots

In this paper we present our work on enabling smart cameras to act as autonomous vision systems for mobile service robots. The smart cameras will adapt their functionality to the current task of the robot system. Instead of transferring raw image data, higher level image information or regions-of-interest are transmitted. This way, the amount of data is reduced and computationally intensive image interpretation will not affect other tasks of the robot system. We are developing a flexible software solution to integrate smart camera systems to the architecture of our robot system. Each image processing task is constructed by a composition of modular functions that can be distributed over different systems. Timing aspects of the flow of data can be analyzed with different tools to evaluate the performance. For this work we are using a commercially available smart camera, but due to the use of a modular architecture the porting to other camera models is easy. We show the advantages of our approach in a setup where image regions containing a face are detected and extracted for further processing steps. This task is accomplished using different setups, where more or fewer subtasks are assigned to the camera system. The performance of the overall system is evaluated with respect to processor load, network load and latency of image data.

Fast and robust detection and tracking of multiple persons on RGB-D data fusing spatio-temporal information

In this paper, we present an efficient and adaptive method for detecting and tracking multiple persons while providing real-time capability and high robustness to outlier noise. Given an RGB-D image data sequence, our algorithm combines two independent approaches for person detection. First, a cluster-based segmentation and classification on RGB-D point clouds and second a face detection on RGB images, where each method itself is post-processed by spatio-temporal filtering for tracking and sensitivity purposes. Our analysis and experimental results prove that the combined approach performs significantly better than the individual solutions and greatly reduces the number of false positives in situations where one detector fails.

Development of an Intelligent Omnivision Surveillance System

This publication describes an innovative intelligent omnivision video system, that stitches the images of four cameras together, resulting in one seamless image. This way the system generates a 360° view of the scene. An additional automatically controlled pan-tilt-zoom-camera provides a high resolution view of user defined regions of interest (ROI). In addition to the fusion of multiple camera images, the system has intelligent features like object detection and region-of-interest detection. The software architecture features configurable pipelines of image processing functions. All different steps in the pipeline like decoding, feature extraction, encoding, and visualization are implemented as modules inside this pipeline. It is easily possible to rearrange the pipeline and add new functions to the overall system. The pan-tilt-zoom camera is controlled by an embedded system that has been developed for this system. GPU-accelerated processing elements allows real-time panorama stitching. We show the application of our system in the field of maritime surveillance, but the system can also be used for robots.

An approach for automated scale invariant STM-scan matching using SIFT

In this paper, we present our research on automated matching of Scanning Tunneling Microscope (STM) images acquired at different scales and resolutions. Scale Invariant Feature Transform (SIFT [1]) is a method of selecting, describing, and matching key points of images. We apply SIFT to images generated with an STM to detect corresponding points and calculate exact transformation matrices. The matching procedure is modified in order to improve the results in this context. The long term goal of our research is to perform automated atom manipulation with an STM. For this purpose, the abilities to build a map of the surface, to recognize the position of the STM tip, and to handle spatial uncertainties are important for sophisticated control algorithms. SIFT has several possible applications in this context, for instance drift measurements and hierarchical maps for tip navigation.