Florian Lier

The Cognitive Interaction Toolkit Improving Reproducibility of Robotic Systems Experiments

Research on robot systems either integrating a large number of capabilities in a single architecture or displaying outstanding performance in a single domain achieved considerable progress over the last years. Results are typically validated through experimental evaluation or demonstrated live, e.g., at robotics competitions. While common robot hardware, simulation and programming platforms yield an improved basis, many of the described experiments still cannot be reproduced easily by interested researchers to confirm the reported findings. We consider this a critical challenge for experimental robotics. Hence, we address this problem with a novel process which facilitates the reproduction of robotics experiments. We identify major obstacles to experiment replication and introduce an integrated approach that allows (i) aggregation and discovery of required research artifacts, (ii) automated software build and deployment, as well as (iii) experiment description, repeatable execution and evaluation.We explain the usage of the introduced process along an exemplary robotics experiment and discuss our approach in the context of current ecosystems for robot programming and simulation.

Towards automated execution and evaluation of simulated prototype HRI experiments

Autonomous robots are highly relevant targets for interaction studies, but can exhibit behavioral variability that confounds experimental validity. Currently, testing on real systems is the only means to prevent this, but remains very labour-intensive and often happens too late. To improve this situation, we are working towards early testing by means of partial simulation, with automated assessment, and based upon continuous software integration to prevent regressions. We will introduce the concept and describe a proof-of-concept that demonstrates fast feedback and coherent experiment results across repeated trials. Categories and Subject Descriptors D.2.5 [Software Engineering]: Testing and Debugging— Designs, Program Verification

Simulation and HRI Recent Perspectives with the MORSE Simulator

Simulation in robotics is often a love-hate relationship: while simulators do save us a lot of time and effort compared to regular deployment of complex software architectures on complex hardware, simulators are also known to evade many of the real issues that robots need to manage when they enter the real world. Because humans are the paragon of dynamic, unpredictable, complex, real world entities, simulation of human-robot interactions may look condemn to fail, or, in the best case, to be mostly useless. This collective article reports on five independent applications of the MORSE simulator in the field of human-robot interaction: It appears that simulation is already useful, if not essential, to successfully carry out research in the field of HRI, and sometimes in scenarios we do not anticipate.

Continuous integration for iterative validation of simulated robot models

Simulated environments often provide the first, and are usually the most frequent, test environment for robotic systems, primarily due to their cost and safety advantages. Unfortunately, changing aspects of both, the simulation and the real robot, as well as actuator control algorithms are often not taken into account when relying on simulation results. In this paper we present a continuous integration approach to verify simulated robot models in an integrated and frequent manner, comprising a simulated and a real robot for comparison. The central aspect of our concept is to iteratively assess the fidelity of simulated robot models. In an exemplary case study we distilled a first set of requirements and metrics, which can be used by developers to verify their algorithms and to automatically detect further system changes.

Welcome to the Future – How Naïve Users Intuitively Address an Intelligent Robotics Apartment

The purpose of this Wizard-of-Oz study was to explore the intuitive verbal and non-verbal goal-directed behavior of naive participants in an intelligent robotics apartment. Participants had to complete seven mundane tasks, for instance, they were asked to turn on the light. Participants were explicitly instructed to consider nonstandard ways of completing the respective tasks. A multi-method approach revealed that most participants favored speech and interfaces like switches and screens to communicate with the intelligent robotics apartment. However, they required instructions to use the interfaces in order to perceive them as competent targets for human-machine interaction. Hence, first important steps were taken to investigate how to design an intelligent robotics apartment in a user-centered and user-friendly manner.

Towards automated system and experiment reproduction in robotics

Even though research on autonomous robots and human-robot interaction accomplished great progress in recent years, and reusable soft- and hardware components are available, many of the reported findings are only hardly reproducible by fellow scientists. Usually, reproducibility is impeded because required information, such as the specification of software versions and their configuration, required data sets, and experiment protocols are not mentioned or referenced in most publications. In order to address these issues, we recently introduced an integrated tool chain and its underlying development process to facilitate reproducibility in robotics. In this contribution we instantiate the complete tool chain in a unique user study in order to assess its applicability and usability. To this end, we chose three different robotic systems from independent institutions and modeled them in our tool chain, including three exemplary experiments. Subsequently, we asked twelve researchers to reproduce one of the formerly unknown systems and the associated experiment. We show that all twelve scientists were able to replicate a formerly unknown robotics experiment using our tool chain.

Facilitating research cooperation through linking and sharing of heterogenous research artefacts: cross platform linking of semantically enriched research artefacts

Researchers and other knowledge workers frequently produce and use diverse research artefacts such as papers, data sets, experiment specifications, software, etc. In this, they are often faced with unclear relationships (e.g., which version of a software was in use for a particular paper), creating unnecessary work and potentially errors. Semantic web technologies can provide metadata as well as explicit, specific links between the artefacts. However, data acquisition and perceived utility are potential stumbling blocks for adoption. Therefore, we propose a system which is focused on integrating and augmenting existing data (thus protecting the existing investment), and examine it using an interaction-oriented perspective, on users without semantic web experience. Specifically, we first study requirements of the target group and then present an exploratory study of managing research artefacts related to software-centric projects. The results confirm that diverse data sources are in common use, that re-using existing repositories is perceived as efficient (e.g., more convenient, shorter cycle time), and that the experimented aggregates are perceived as functionally relevant. Furthermore, the integration of quality assurance mechanisms, such as continuous integration, is perceived as beneficial, despite some added effort.

ToBI – Team of Bielefeld: Enhancing Robot Behaviors and the Role of Multi-robotics in RoboCup@Home

In this paper, we describe the joint effort of the Team of Bielefeld (ToBI) which won the RoboCup@Home competition in Leipzig 2016. RoboCup@Home consists of a defined set of benchmarking tests that cover multiple skills needed by service robots. We present the robotic platforms, technical contributions, and lessons learned from previous events that led to the final success this year. This includes a framework for behavior modeling and communication employed on two human-sized robots Floka and Biron as well as on the small robotic device AMiRo. These were used for a multi-robot collaboration scenario in the Finals. We describe our main contributions in automated testing, error handling, memorization and reporting, robot-robot coordination, and flexible grasping that considers object shape.

RoboBench: Towards sustainable robotics system benchmarking

We present RoboBench, a novel platform for sharing robot full-system simulations for benchmarking. The creation of this platform and benchmark suite is motivated by a need for reproducible research. A challenge in creating a full-system benchmarks are incompatibilities in software created by different groups and the difficulty of reproducing software environments. We solve this problem by using software containers, an emerging virtualization technology. RoboBench enables sharing robot software in a runnable state, capturing the software behavior of robots carrying out missions. These simulations make clear the performance impact and resource usage of programs and algorithms relative to other software involved in the mission. These containers are integrated with the CITK platform for reproducible research, which automates generation and publishing of the containers. We present an overview of the system, a description of our prototype set of benchmark missions, along with a validation study comparing the computational load profile of a mission performed on a real and simulated robot. Additionally, we present preliminary results of an overall analysis of the benchmarks in the RoboBench suite, showing where computational work is expended in robotics common robotics tasks. RoboBench is extensible, and is the first step toward a robust, quantitative approach to engineering computationally-efficient robots.

Humotion: A Human Inspired Gaze Control Framework for Anthropomorphic Robot Heads

In recent years, an attempt is being made to control robots more intuitive and intelligible by exploiting and integrating anthropomorphic features to boost social human-robot interaction. The design and construction of anthropomorphic robots for this kind of interaction is not the only challenging issue -- smooth and expectation-matching motion control is still an unsolved topic. In this work we present a highly configurable, portable, and open control framework that facilitates anthropomorphic motion generation for humanoid robot heads by enhancing state-of-the-art neck-eye coordination with human-like eyelid saccades and animation. On top of that, the presented framework supports dynamic neck offset angles that allow animation overlays and changes in alignment to the robots communication partner whileretaining visual focus on a given target. In order to demonstrate the universal applicability of the proposed ideas we used this framework to control the Flobi and the iCub robot head, both in simulation and on the physical robot. In order to foster further comparative studies of different robot heads, we will release all software, based on this contribution, under an open-source license.