Alberto Montebelli

On Cognition as Dynamical Coupling: An Analysis of Behavioral Attractor Dynamics

The interaction of brain, body, and environment can result in complex behavior with rich dynamics, even for relatively simple agents. Such dynamics are, however, often difficult to analyze. In this article, we explore the case of a simple simulated robotic agent, equipped with a reactive neurocontroller and an energy level, which the agent has been evolved to recharge. A dynamical systems analysis shows that a non-neural internal state (energy level), despite its simplicity, dynamically modulates the behavioral attractors of the agent—environment system, such that the robots behavioral repertoire is continually adapted to its current situation and energy level. What emerges is a dynamic, non-deterministic, and highly self-organized action selection mechanism, originating from the dynamical coupling of four systems (non-neural internal states, neurocontroller, body, and environment) operating at very different timescales.

Simultaneous kinesthetic teaching of positional and force requirements for sequential in-contact tasks

This paper demonstrates a method for simultaneous transfer of positional and force requirements for in-contact tasks from a human instructor to a robotic arm through kinesthetic teaching. This is achieved by a specific use of the sensory configuration, where a force/torque sensor is mounted between the tool and the flange of a robotic arm endowed with integrated torque sensors at each joint. The human demonstration is modeled using Dynamic Movement Primitives. Following human demonstration, the robot arm is provided with the capacity to perform sequential in-contact tasks, for example writing on a notepad a previously demonstrated sequence of characters. During the reenactment of the task, the system is not only able to imitate and generalize from demonstrated trajectories, but also from their associated force profiles. In fact, the implemented framework is extended to successfully recover from perturbations of the trajectory during reenactment and to cope with dynamic environments.

Incrementally Assisted Kinesthetic Teaching for Programming by Demonstration

Kinesthetic teaching is an established method of teaching robots new skills without requiring robotics or programming knowledge. However, the inertia and uncoordinated motions of individual joints decrease the intuitiveness and naturalness of interaction and impair the quality of the learned skill. This paper proposes a method to ease kinesthetic teaching by combining the idea of incremental learning through warping several demonstrations into a common frame with virtual tool dynamics to assist the user during teaching. In fact, during a sequence of demonstrations the stiffness of the robot under Cartesian impedance control is gradually increased, to provide stronger assistance to the user based on the demonstrations accumulated up to that moment. Therefore, the operator has the opportunity to progressively refine the tasks model while the robot more docilely follows the learned action. Robot experiments and a user study performed on 25 novice users show that the proposed approach improves both usability as well as resulting skill quality.

On handing down our tools to robots: Single-phase kinesthetic teaching for dynamic in-contact tasks

We present a (generalizable) method aimed to simultaneously transfer positional and force requirements encoded in a physical human skill (wood planing) from a human instructor to a robotic arm through kinesthetic teaching. We achieve our goal through a novel use of a common sensory configuration, constituted by a force/torque sensor mounted between the tool and the flange of a robotic arm. The robotic arm is endowed with integrated torque sensors at each joint. The mathematical model used to capture the general dynamic of the interaction between the human user and the wood surface is based on Dynamic Movement Primitives. During reenactment of the task, the system can imitate and generalize the demonstrated spatial requirements, as well as their associated force profiles. Therefore, the robotic arm acquires the capacity to reproduce the dynamic profile for in-contact tasks requiring an articulated coordination in the distribution of forces. For example, the capacity to effectively operate the plane on a wood plank over multiple strokes, according to the demonstration of the human instructor.

The Cognitive Body: From Dynamic Modulation to Anticipation

Starting from the situated and embodied perspective on the study of cognition as a source of inspiration, this paper programmatically outlines a path towards an experimental exploration of the role of the body in a minimal anticipatory cognitive architecture. Cognition is here conceived and synthetically analyzed as a broadly extended and distributed dynamic process emerging from the interplay between a body, a nervous system and their environment. Firstly, we show how a non-neural internal state, crucially characterized by slowly changing dynamics, can modulate the activity of a simple neurocontroller. The result, emergent from the use of a standard evolutionary robotic simulation, is a self-organized, dynamic action selection mechanism, effectively operating in a context dependent way. Secondly, we show how these characteristics can be exploited by a novel minimalist anticipatory cognitive architecture. Rather than a direct causal connection between the anticipation process and the selection of the appropriate behavior, it implements a model for dynamic anticipation that operates via bodily mediation (bodily-anticipation hypothesis ). This allows the system to swiftly scale up to more complex tasks never experienced before, achieving flexible and robust behavior with minimal adaptive cost.

An analysis of behavioral attractor dynamics

The interaction of brain, body and environment can result in complex behavior with rich dynamics even for relatively simple agents. Such dynamics are, however, often notoriously difficult to analyze. In this paper we explore the case of a simple simulated robotic agent, equipped with a reactive neurocontroller and an energy level, that the agent has been evolved to re-charge. A dynamical systems analysis, shows that a non-neural internal state (energy level), despite its simplicity, dynamically modulates the agent-environment systems behavioral attractors, such that the robots behavioral repertoire is continually adapted to its current situation and energy level.

The Search for Beauty: Evolution of Minimal Cognition in an Animat Controlled by a Gene Regulatory Network and Powered by a Metabolic System

We have created a model of a hybrid system in which a gene regulatory network (GRN) controls the search for resources (fuel/food and water) necessary to allow an artificial metabolic system (simulated microbial fuel cell) to produce energy. We explore the behaviour of simple animats in a two-dimensional simulated environment requiring minimal cognition. In our system control evolves in a biologically-realistic manner under tight energy constraints. We use a model of GRN in which there is no limit on the size of the network, and the concentration of regulatory substances (transcriptional factors, TFs) change in a continuous fashion. Externally driven concentrations of selected TFs provide the sensory information to the animat, while the concentration of selected internally produced TFs is interpreted as the signal for actuators. We use a genetic algorithm to obtain diverse evolved strategies in ecologically grounded animats with motivational autonomy, even though they lack a dedicated motivational circuit. There are three motivations (or drives) in the system: thirst, hunger, and reproduction. The animats need to search for food and water, but also to perform work. Because the value of such work is arbitrary (in the eye of the beholder), but affects the chances of reproduction, we suggest that the term beauty is more appropriate, and we name the task the Search for Beauty. The results obtained provide a step towards realizing a biologically realistic system with respect to: the way the control is exercised, the way it evolves, and the way the metabolism provides energy.

The Role of Internal States in the Emergence of Motivation and Preference: A Robotics Approach

In order to explain and model emotion we need to attend to the role internal states play in the generation of behavior. We argue that motivational and perceptual roles emerge from the dynamical interaction between physiological processes, sensory-motor processes and the environment. We investigate two aspects inherent to emotion appraisal and response which rely on physiological process: the ability to categorize relations with the environment and to modulate response generating different action tendencies.

Learning in-contact control strategies from demonstration

Learning to perform tasks like pulling a door handle or pushing a button, inherently easy for a human, can be surprisingly difficult for a robot. A crucial problem in these kinds of in-contact tasks is the context specificity of pose and force requirements. In this paper, a robot learns in-contact tasks from human kinesthetic demonstrations. To address the need to balance between the position and force constraints, we propose a model based on the hidden semi-Markov model (HSMM) and Cartesian impedance control. The model captures uncertainty over time and space and allows the robot to smoothly satisfy a tasks position and force constraints by online modulation of impedance controller stiffness according to the HSMM state belief. In experiments, a KUKA LWR 4+ robotic arm equipped with a force/torque sensor at the wrist successfully learns from human demonstrations how to pull a door handle and push a button.

Learning movement synchronization in multi-component robotic systems

Imitation learning of tasks in multi-component robotic systems requires capturing concurrency and synchronization requirements in addition to task structure. Learning time-critical tasks depends furthermore on the ability to model temporal elements in demonstrations. This paper proposes a modeling framework based on Petri nets capable of modeling these aspects in a programming by demonstration context. In the proposed approach, models of tasks are constructed from segmented demonstrations as task Petri nets, which can be executed as discrete controllers for reproduction. We present algorithms that automatically construct models from demonstrations, showing how elements of time-critical tasks can be mapped into task Petri net elements. The approach is validated by an experiment in which a robot plays a musical passage on a keyboard.