Frédéric Kaplan

Intrinsic Motivation Systems for Autonomous Mental Development

Exploratory activities seem to be intrinsically rewarding for children and crucial for their cognitive development. Can a machine be endowed with such an intrinsic motivation system? This is the question we study in this paper, presenting a number of computational systems that try to capture this drive towards novel or curious situations. After discussing related research coming from developmental psychology, neuroscience, developmental robotics, and active learning, this paper presents the mechanism of Intelligent Adaptive Curiosity, an intrinsic motivation system which pushes a robot towards situations in which it maximizes its learning progress. This drive makes the robot focus on situations which are neither too predictable nor too unpredictable, thus permitting autonomous mental development. The complexity of the robots activities autonomously increases and complex developmental sequences self-organize without being constructed in a supervised manner. Two experiments are presented illustrating the stage-like organization emerging with this mechanism. In one of them, a physical robot is placed on a baby play mat with objects that it can learn to manipulate. Experimental results show that the robot first spends time in situations which are easy to learn, then shifts its attention progressively to situations of increasing difficulty, avoiding situations in which nothing can be learned. Finally, these various results are discussed in relation to more complex forms of behavioral organization and data coming from developmental psychology

What is Intrinsic Motivation? A Typology of Computational Approaches.

Intrinsic motivation, centrally involved in spontaneous exploration and curiosity, is a crucial concept in developmental psychology. It has been argued to be a crucial mechanism for open-ended cognitive development in humans, and as such has gathered a growing interest from developmental roboticists in the recent years. The goal of this paper is threefold. First, it provides a synthesis of the different approaches of intrinsic motivation in psychology. Second, by interpreting these approaches in a computational reinforcement learning framework, we argue that they are not operational and even sometimes inconsistent. Third, we set the ground for a systematic operational study of intrinsic motivation by presenting a formal typology of possible computational approaches. This typology is partly based on existing computational models, but also presents new ways of conceptualizing intrinsic motivation. We argue that this kind of computational typology might be useful for opening new avenues for research both in psychology and developmental robotics.

Bootstrapping grounded word semantics

The paper reports on experiments with a population of visually grounded robotic agents capable of bootstrapping their own ontology and shared lexicon without prior design nor other forms of human intervention. The agents do so while playing a particular language game called the guessing game. We show that synonymy and ambiguity arise as emergent properties in the lexicon, due to the situated grounded character of the agent-environment interaction, but that there are also tendencies to dampen them so as to make the language more coherent and thus more optimal from the viewpoints of communicative success, cognitive complexity, and learnability.

Robotic clicker training

Abstract In this paper, we want to propose the idea that some techniques used for animal training might be helpful for solving human–robot interaction problems in the context of entertainment robotics. We present a model for teaching complex actions to an animal-like autonomous robot based on “clicker training”, a method used efficiently by professional trainers for animals of different species. After describing our implementation of clicker training on an enhanced version of AIBO, Sony’s four-legged robot, we argue that this new method can be a promising technique for teaching unusual behavior and sequences of actions to a pet robot.

In search of the neural circuits of intrinsic motivation.

Children seem to acquire new know-how in a continuous and open-ended manner. In this paper, we hypothesize that an intrinsic motivation to progress in learning is at the origins of the remarkable structure of childrens developmental trajectories. In this view, children engage in exploratory and playful activities for their own sake, not as steps toward other extrinsic goals. The central hypothesis of this paper is that intrinsically motivating activities correspond to expected decrease in prediction error. This motivation system pushes the infant to avoid both predictable and unpredictable situations in order to focus on the ones that are expected to maximize progress in learning. Based on a computational model and a series of robotic experiments, we show how this principle can lead to organized sequences of behavior of increasing complexity characteristic of several behavioral and developmental patterns observed in humans. We then discuss the putative circuitry underlying such an intrinsic motivation system in the brain and formulate two novel hypotheses. The first one is that tonic dopamine acts as a learning progress signal. The second is that this progress signal is directly computed through a hierarchy of microcortical circuits that act both as prediction and metaprediction systems.

An Interactive Table for Supporting Participation Balance in Face-to-Face Collaborative Learning

We describe an interactive table designed for supporting face-to-face collaborative learning. The table, Reflect, addresses the issue of unbalanced participation during group discussions. By displaying on its surface, a shared visualization of member participation, Reflect, is meant to encourage participants to avoid the extremes of over and underparticipation. We report on a user study that validates some of our hypotheses on the effect the table would have on its users. Namely, we show that Reflect leads to more balanced collaboration, but only under certain conditions. We also show different effects the table has on over and underparticipators.

Maximizing Learning Progress: An Internal Reward System for Development

This chapter presents a generic internal reward system that drives an agent to increase the complexity of its behavior. This reward system does not reinforce a predefined task. Its purpose is to drive the agent to progress in learning given its embodiment and the environment in which it is placed. The dynamics created by such a system are studied first in a simple environment and then in the context of active vision.

Collective Learning and Semiotic Dynamics

We report on a case study in the emergence of a lexicon in a group of autonomous distributed agents situated and grounded in an open environment. Because the agents are autonomous, grounded, and situated, the possible words and possible meanings are not fixed but continuously change as the agents autonomously evolve their communication system and adapt it to novel situations. The case study shows that a complex semiotic dynamics unfolds and that generalisations present in the language are due to processes outside the agent.

Social behaviour of dogs encountering AIBO, an animal-like robot in a neutral and in a feeding situation

The use of animal-like autonomous robots might offer new possibilities in the study of animal interactions, if the subject recognises it as a social partner. In this paper we investigate whether AIBO, a dog-like robot of the Sony Corp. can be used for this purpose. Twenty-four adult and sixteen 4-5 months old pet dogs were tested in two situations where subjects encountered one of four different test-partners: (1) a remote controlled car; (2) an AIBO robot; (3) AIBO with a puppy-scented furry cover; and (4) a 2-month-old puppy. In the neutral situation the dog could interact freely with one of the partners for 1 min in a closed arena in the presence of its owner. In the feeding situation the encounters were started while the dog was eating food. Our results show that age and context influence the social behaviour of dogs. Further, we have found that although both age groups differentiated the living and non-living test-partners for some extent, the furry AIBO evoked significantly increased responses in comparison to the car. These experiments show the first steps towards the application of robots in behavioural studies, notwithstanding that at present AIBOs limited ability to move constrains its effectiveness as social partner for dogs.

Living With a Vacuum Cleaning Robot - A 6-month Ethnographic Study

Little is known about the usage, adoption process and long-term effects of domestic service robots in people’s homes. We investigated the usage, acceptance and process of adoption of a vacuum cleaning robot in nine households by means of a six month ethnographic study. Our major goals were to explore how the robot was used and integrated into daily practices, whether it was adopted in a durable way, and how it impacted its environment. We studied people’s perception of the robot and how it evolved over time, kept track of daily routines, the usage patterns of cleaning tools, and social activities related to the robot. We integrated our results in an existing framework for domestic robot adoption and outlined similarities and differences to it. Finally, we identified several factors that promote or hinder the process of adopting a domestic service robot and make suggestions to further improve human-robot interactions and the design of functional home robots toward long-term acceptance.