Alberto Finzi

The SHERPA project: Smart collaboration between humans and ground-aerial robots for improving rescuing activities in alpine environments

The goal of the paper is to present the foreseen research activity of the European project “SHERPA” whose activities will start officially on February 1th 2013. The goal of SHERPA is to develop a mixed ground and aerial robotic platform to support search and rescue activities in a real-world hostile environment, like the alpine scenario that is specifically targeted in the project. Looking into the technological platform and the alpine rescuing scenario, we plan to address a number of research topics about cognition and control. What makes the project potentially very rich from a scientific viewpoint is the heterogeneity and the capabilities to be owned by the different actors of the SHERPA system: the human rescuer is the “busy genius”, working in team with the ground vehicle, as the “intelligent donkey”, and with the aerial platforms, i.e. the “trained wasps” and “patrolling hawks”. Indeed, the research activity focuses on how the “busy genius” and the “SHERPA animals” interact and collaborate with each other, with their own features and capabilities, toward the achievement of a common goal.

Human-Robot interaction through mixed-initiative planning for rescue and search rovers

We present an approach to human-robot interaction in Urban Search and Rescue (USAR) domains based on reactive mixed-initiative planning. A model-based executive monitoring system is used to coordinate the operators interventions and the concurrent activities of a rescue rover. In this setting, the users and the robots activities are coordinated by a continuos reactive planning process. We show the advantages of this approach for both the operator situation awareness and human-robot interaction during rescue missions. We present the implementation of the control architecture on a robotic system (DORO) providing some experimental results obtained from testing in rescue arenas.

An extensible architecture for robust multimodal human-robot communication

Human safety and effective human-robot communication are main concerns in HRI applications. In order to achieve such goals, a system should be very robust, allowing little chance for misunderstanding the users commands. Moreover, the system should permit natural interaction reducing the time and the effort needed to achieve tasks. The main purpose of this work is to develop a general framework for flexible and multimodal human-robot communication. The proposed architecture should be easy to modify and expand, adding or modifying input channels and changing the multimodal fusion strategies. In this paper, we introduce our general approach and provide a case study with two modalities (gesture and speech).

A dialogue system for multimodal human-robot interaction

This paper presents a POMDP-based dialogue system for multimodal human-robot interaction (HRI). Our aim is to exploit a dialogical paradigm to allow a natural and robust interaction between the human and the robot. The proposed dialogue system should improve the robustness and the flexibility of the overall interactive system, including multimodal fusion, interpretation, and decision-making. The dialogue is represented as a Partially Observable Markov Decision Process (POMDPs) to cast the inherent communication ambiguity and noise into the dialogue model. POMDPs have been used in spoken dialogue systems, mainly for tourist information services, but their application to multimodal human-robot interaction is novel. This paper presents the proposed model for dialogue representation and the methodology used to compute a dialogue strategy. The whole architecture has been integrated on a mobile robot platform and has bee n tested in a human-robot interaction scenario to assess the overall performances with respect to baseline controllers.

Attentional modulation of mutually dependent behaviors

In this paper, we investigate simple attentional mechanisms suitable for sensing rate regulation and action coordination in the presence of mutually dependent behaviors. We present our architecture along with a case study where a real robotic system is to manage and harmonize conflicting tasks. This research focuses on attentional mechanisms for regulating the frequencies of sensor readings and action activations in a behavior-based robotic system. Such mechanisms are to direct sensors toward the most salient sources of information and filter the available sensory data to prevent unnecessary information processing.

Validation and verification issues in a timeline-based planning system

To foster effective use of artificial intelligence planning and scheduling (PS moreover, they employ resolution processes designed to optimize the solution with respect to non-trivial evaluation functions. Knowledge engineering environments aim at simplifying direct access to the technology for people other than the original system designers, while the integration of validation and verification (V&V) capabilities in such environments may potentially enhance the users’ trust in the technology. Somehow, V&V techniques may represent a complementary technology, with respect to P&S, that contributes to developing richer software environments to synthesize a new generation of robust problem-solving applications. The integration of V&V and P&S techniques in a knowledge engineering environment is the topic of this paper. In particular, it analyzes the use of state-of-the-art V&V technology to support knowledge engineering for a timeline-based planning system called MrSPOCK. The paper presents the application domain for which the automated solver has been developed, introduces the timeline-based planning ideas, and then describes the different possibilities to apply V&V to planning. Hence, it continues by describing the step of adding V&V functionalities around the specialized planner, MrSPOCK. New functionalities have been added to perform both model validation and plan verification. Lastly, a specific section describes the benefits as well as the performance of such functionalities.

TGA-based controllers for flexible plan execution

Plans synthesized by Temporal Planning and Scheduling systems may be temporally flexible hence they identify an envelope of possible solutions. Such flexibility can be exploited by an executive systems for robust on-line execution. Recent works have addressed aspects of plan execution using a quite general approach grounded on formal modeling and formal methods. The present work extends such an approach by presenting the formal synthesis of a plan controller associated to a flexible temporal plan. In particular, the controller synthesis exploits Timed Game Automata (TGA) for formal modeling and UPPAAL-TIGA as a model checker. After presenting a formal extension, the paper introduces a detailed experimental analysis on a real-world case study that demonstrates the viability of the approach. In particular, it is shown how the controller synthesis overhead is compatible with the performance expected from a short-horizon planner.

Continuous gesture recognition for flexible human-robot interaction

In this work, we present a reliable and continuous gesture recognition method that supports a natural and flexible interaction between the human and the robot. The aim is to provide a system that can be trained online with few samples and can cope with intra user variability during the gesture execution. The proposed approach relies on the generation of an ad-hoc Hidden Markov Model (HMM) for each gesture exploiting a direct estimation of the parameters. Each model represents the best prototype candidate from the associated gesture training set. The generated models are then employed within a continuous recognition process that provides the probability of each gesture at each step. The proposed method is evaluated in two case studies: a hand-performed letters recognizer and a natural gesture recognizer. Finally, we show the overall system at work in a simple human-robot interaction scenario.

Augmenting situation awareness via model-based control in rescue robots

In this work we describe a model-based approach to the executive control of a rescue rover. We show how this control architecture naturally supports human-robot interaction in the diverse activities needed in rescue and search. We illustrate the approach by considering human-robot interaction in the domain of the RoboCup rescue competition. We discuss the implementation and tests done both during RoboCup contests and in the laboratory, to show performances according to different working modalities such as fully operated, supervised, fully autonomous.

A mixed-initiative control system for an Aerial Service Vehicle supported by force feedback

We present an approach to mixed initiative control for unmanned aerial vehicles (UAVs) where sliding autonomy is supported by mixed-initiative planning and haptic feedback. In the proposed framework, we assume that an autonomous system can plan and execute robotic tasks while a human operator can provide interventions when necessary receiving a force feedback. The haptic feedback is associated with the sensation about how the system is diverging from the planned operations. We tested the system at work in virtual and real environments considering simple navigation tasks. We compared the performance of human operators with or without the assistance of the force feedback. The collected results support the hypothesis that the proposed approach enables effective and intuitive mixed-initiative control.