João Filipe Ferreira

Bayesian real-time perception algorithms on GPU

In this text we present the real-time implementation of a Bayesian framework for robotic multisensory perception on a graphics processing unit (GPU) using the Compute Unified Device Architecture (CUDA). As an additional objective, we intend to show the benefits of parallel computing for similar problems (i.e. probabilistic grid-based frameworks), and the user-friendly nature of CUDA as a programming tool. Inspired by the study of biological systems, several Bayesian inference algorithms for artificial perception have been proposed. Their high computational cost has been a prohibitory factor for real-time implementations. However in some cases the bottleneck is in the large data structures involved, rather than the Bayesian inference per se. We will demonstrate that the SIMD (single-instruction, multiple-data) features of GPUs provide a means for taking a complicated framework of relatively simple and highly parallelisable algorithms operating on large data structures, which might take up to several minutes of execution with a regular CPU implementation, and arrive at an implementation that executes in the order of tenths of a second. The implemented multimodal perception module (including stereovision, binaural sensing and inertial sensing) builds an egocentric representation of occupancy and local motion, the Bayesian Volumetric Map (BVM), based on which gaze shift decisions are made to perform active exploration and reduce the entropy of the BVM. Experimental results show that the real-time implementation successfully drives the robotic system to explore areas of the environment mapped with high uncertainty.

Probabilistic Approaches to Robotic Perception

This book tries to address the following questions: How should the uncertainty and incompleteness inherent to sensing the environment be represented and modelled in a way that will increase the autonomy of a robot? How should a robotic system perceive, infer, decide and act efficiently? These are two of the challenging questions robotics community and robotic researchers have been facing. The development of robotic domain by the 1980s spurred the convergence of automation to autonomy, and the field of robotics has consequently converged towards the field of artificial intelligence (AI). Since the end of that decade, the general publics imagination has been stimulated by high expectations on autonomy, where AI and robotics try to solve difficult cognitive problems through algorithms developed from either philosophical and anthropological conjectures or incomplete notions of cognitive reasoning. Many of these developments do not unveil even a few of the processes through which biological organisms solve these same problems with little energy and computing resources. The tangible results of this research tendency were many robotic devices demonstrating good performance, but only under well-defined and constrained environments. The adaptability to different and more complex scenarios was very limited. In this book, the application of Bayesian models and approaches are described in order to develop artificial cognitive systems that carry out complex tasks in real world environments, spurring the design of autonomous, intelligent and adaptive artificial systems, inherently dealing with uncertainty and the irreducible incompleteness of models.

A Bayesian framework for active artificial perception

In this paper, we present a Bayesian framework for the active multimodal perception of 3-D structure and motion. The design of this framework finds its inspiration in the role of the dorsal perceptual pathway of the human brain. Its composing models build upon a common egocentric spatial configuration that is naturally fitting for the integration of readings from multiple sensors using a Bayesian approach. In the process, we will contribute with efficient and robust probabilistic solutions for cyclopean geometry-based stereovision and auditory perception based only on binaural cues, modeled using a consistent formalization that allows their hierarchical use as building blocks for the multimodal sensor fusion framework. We will explicitly or implicitly address the most important challenges of sensor fusion using this framework, for vision, audition, and vestibular sensing. Moreover, interaction and navigation require maximal awareness of spatial surroundings, which, in turn, is obtained through active attentional and behavioral exploration of the environment. The computational models described in this paper will support the construction of a simultaneously flexible and powerful robotic implementation of multimodal active perception to be used in real-world applications, such as human-machine interaction or mobile robot navigation.

Implementation and calibration of a Bayesian binaural system for 3D localisation

In this text we present a Bayesian system of auditory localisation in distance, azimuth and elevation using binaural cues only; we focus mainly on implementation details and the calibration procedure, and present experimental results. This binaural system is also integrated in a spatial representation framework for multimodal perception of 3D structure and motion — the Bayesian Volumetric Map (BVM). This solution will enable the implementation of an active perception system with great potential in applications as diverse as social robots or even robotic navigation.

Attentional Mechanisms for Socially Interactive Robots–A Survey

This review intends to provide an overview of the state of the art in the modeling and implementation of automatic attentional mechanisms for socially interactive robots. Humans assess and exhibit intentionality by resorting to multisensory processes that are deeply rooted within low-level automatic attention-related mechanisms of the brain. For robots to engage with humans properly, they should also be equipped with similar capabilities. Joint attention, the precursor of many fundamental types of social interactions, has been an important focus of research in the past decade and a half, therefore providing the perfect backdrop for assessing the current status of state-of-the-art automatic attentional-based solutions. Consequently, we propose to review the influence of these mechanisms in the context of social interaction in cutting-edge research work on joint attention. This will be achieved by summarizing the contributions already made in these matters in robotic cognitive systems research, by identifying the main scientific issues to be addressed by these contributions and analyzing how successful they have been in this respect, and by consequently drawing conclusions that may suggest a roadmap for future successful research efforts.

A hierarchical Bayesian framework for multimodal active perception

In this article, we present a hierarchical Bayesian framework for multimodal active perception, devised to be emergent, scalable and adaptive. This framework, while not strictly neuromimetic, finds its roots in the role of the dorsal perceptual pathway of the human brain. Its composing models build upon a common spatial configuration that is naturally fitting for the integration of readings from multiple sensors using a Bayesian approach devised in previous work. The framework presented in this article is shown to adequately model human-like active perception behaviours, namely by exhibiting the following desirable properties: high-level behaviour results from low-level interaction of simpler building blocks; seamless integration of additional inputs is allowed by the Bayesian Programming formalism; initial ‘genetic imprint’ of distribution parameters may be changed ‘on the fly’ through parameter manipulation, thus allowing for the implementation of goal-dependent behaviours (i.e. top-down influences).

Designing an artificial attention system for social robots

In this paper, we introduce the main components comprising the action-perception loop of an overarching framework implementing artificial attention, designed to fulfil the requirements of social interaction (i.e., reciprocity, and awareness), with strong inspiration on current theories in functional neuroscience. We demonstrate the potential of our framework, by showing how it exhibits coherent behaviour without any inbuilt prior expectations regarding the experimental scenario. Current research in cognitive systems for social robots has suggested that automatic attention mechanisms are essential to social interaction. In fact, we hypothesise that enabling artificial cognitive systems with middleware implementing these mechanisms will empower robots to perform adaptively and with a higher degree of autonomy in complex and social environments. However, this type of assumption is yet to be convincingly and systematically put to the test. The ultimate goal will be to test our working hypothesis and the role of attention in adaptive, social robotics.

Active Exploration Using Bayesian Models for Multimodal Perception

In this text we will present a novel solution for active perception built upon a probabilistic framework for multimodal perception of 3D structure and motion -- the Bayesian Volumetric Map (BVM). This solution applies the notion of entropy to promote gaze control for active exploration of areas of high uncertainty on the BVM so as to dynamically build a spatial map of the environment storing the largest amount of information possible. Moreover, entropy-based exploration is shown to be an efficient behavioural strategy for active multimodal perception.

Touch attention Bayesian models for robotic active haptic exploration of heterogeneous surfaces

This work contributes to the development of active haptic exploration strategies of surfaces using robotic hands in environments with an unknown structure. The architecture of the proposed approach consists two main Bayesian models, implementing the touch attention mechanisms of the system. The model πper perceives and discriminates different categories of materials (haptic stimulus) integrating compliance and texture features extracted from haptic sensory data. The model πtar actively infers the next region of the workspace that should be explored by the robotic system, integrating the task information, the permanently updated saliency and uncertainty maps extracted from the perceived haptic stimulus map, as well as, inhibition-of-return mechanisms. The experimental results demonstrate that the Bayesian model πper can be used to discriminate 10 different classes of materials with an average recognition rate higher than 90%. The generalization capability of the proposed models was demonstrated experimentally. The ATLAS robot, in the simulation, was able to perform the following of a discontinuity between two regions made of different materials with a divergence smaller than 1cm (30 trials). The tests were performed in scenarios with 3 different configurations of the discontinuity. The Bayesian models have demonstrated the capability to manage the uncertainty about the structure of the surfaces and sensory noise to make correct motor decisions from haptic percepts.

Robotic implementation of biological Bayesian models for visuo-inertial image stabilization and gaze control

Robotic implementations of gaze control and image stabilization have been previously proposed, that rely on fusing inertial and visual sensing modalities. Human and biological system also combine the two sensing modalities for the same goal. In this work we build upon these previous results and, with the contribution of psychophysical studies, attempt a more bio-inspired approach to the robotic implementation. Since Bayesian models have been successfully used to explain psychophysical experimental findings, we propose a robotic implementation using Bayesian inference.