Olivier L. Georgeon

An intrinsically-motivated schema mechanism to model and simulate emergent cognition

We introduce an approach to simulate the early mechanisms of emergent cognition based on theories of enactive cognition and on constructivist epistemology. The agent has intrinsic motivations implemented as inborn proclivities that drive the agent in a proactive way. Following these drives, the agent autonomously learns regularities afforded by the environment, and hierarchical sequences of behaviors adapted to these regularities. The agent represents its current situation in terms of perceived affordances that develop through the agents experience. This situational representation works as an emerging situation awareness that is grounded in the agents interaction with its environment and that in turn generates expectations and activates adapted behaviors. Through its activity and these aspects of behavior (behavioral proclivity, situation awareness, and hierarchical sequential learning), the agent starts to exhibit emergent sensibility, intrinsic motivation, and autonomous learning. Following theories of cognitive development, we argue that this initial autonomous mechanism provides a basis for implementing autonomously developing cognitive systems.

Supporting activity modelling from activity traces

We present a new method and tool for activity modelling through qualitative sequential data analysis. In particular, we address the question of constructing a symbolic abstract representation of an activity from an activity trace. We use knowledge engineering techniques to help the analyst build an ontology of the activity, that is, a set of symbols and hierarchical semantics that supports the construction of activity models. The ontology construction is pragmatic, evolutionist and driven by the analyst in accordance with their modelling goals and their research questions. Our tool helps the analyst define transformation rules to process the raw trace into abstract traces based on the ontology. The analyst visualizes the abstract traces and iteratively tests the ontology, the transformation rules and the visualization format to confirm the models of activity. With this tool and this method, we found innovative ways to represent a car-driving activity at different levels of abstraction from activity traces collected from an instrumented vehicle. As examples, we report two new strategies of lane changing on motorways that we have found and modelled with this approach.

Cognitive Modelling and Computational Simulation of Drivers Mental Activities

Several car driver models are available in the literature. From an historical point of view, it is possible to identify three main phases, for the last 40 years, in this research area. The works of (1970), centred on task analysis, are typically representative of the studies carried out during the 1970s. The authors proposed a taxonomy of the main driving tasks (e.g. accelerating, steering, overtaking, lane changing) organised in nine categories (e.g. basic control tasks, tasks related to traffic condition). This work closely parallels the research of Allen et al. (1970), who divided the driving task in three levels: the microperformance, the situational performance and the macroperformance. These levels differ both according to their time scale and with regard to the kind of cognitive activity required. At the microperformance level, most of the actions are automated skills. Steering and speed control are the main subtasks. Feedback loops, concerning driving action implemented at this level, are very short (on the order of seconds). The macroperformance concerns the trip planning and the route finding during the trip. It corresponds to slow conscious processes requiring cognitive resources. The time scale can be hours at this level. Between these two levels, situational performance corresponds to the analysis of the road environment and to the selection of relevant behaviour in the current situation and traffic conditions. Performance at this level is determined by the driver’s perception and understanding of the driving context.

Interactional Motivation in artificial systems: Between extrinsic and intrinsic motivation

This paper introduces Interactional Motivation (IM) as a way to implement self-motivation in artificial systems. An interactionally motivated agent selects behaviors for the sake of enacting the behavior itself rather than for the value of the behaviors outcome. IM contrasts with extrinsic motivation by the fact that it defines the agents motivation independently from the environments state. Because IM does not refer to the environments states, we argue that IM is a form of self-motivation on the same level as intrinsic motivation. IM, however, differs from intrinsic motivation by the fact that IM allows specifying the agents inborn value system explicitly. This paper introduces a formal definition of the IM paradigm and compares it to the reinforcement-learning paradigm as traditionally implemented in Partially Observable Markov Decision Processes (POMDPs).

Analyzing Behaviorial Data for Refining Cognitive Models of Operator

We present a methodology and a tool for analyzing the activity of an operator interacting with a complex technical device. The goal is of refining cognitive models of the operator by relating them to patterns of behavior in real situations. The activity is observed to he modeled as a trace having a graph structure. The trace is transformed according to metadata in order to become meaningful in the context of modeling theories. Our trace based system thus gathers both a representation of the activity and a representation of the analysts expertise, for facilitating the discovery of knowledge in the field of cognitive psychology. The approach is illustrated by its application for car driver cognitive modeling

Demonstrating sensemaking emergence in artificial agents: A method and an example

We propose an experimental method to study the possible emergence of sensemaking in artificial agents. This method involves analyzing the agents behavior in a test bed environment that presents regularities in the possibilities of interaction afforded to the agent, while the agent has no presuppositions about the underlying functioning of the environment that explains such regularities. We propose a particular environment that permits such an experiment, called the Small Loop Problem. We argue that the agents behavior demonstrates sensemaking if the agent learns to exploit regularities of interaction to fulfill its self-motivation as if it understood (at least partially) the underlying functioning of the environment. As a corollary, we argue that sensemaking and self-motivation come together. We propose a new method to generate self-motivation in an artificial agent called interactional motivation. An interactionally motivated agent seeks to perform interactions with predefined positive values and avoid interactions with predefined negative values. We applied the proposed sensemaking emergence demonstration method to an agent implemented previously, and produced example reports that suggest that this agent is capable of a rudimentary form of sensemaking.

A Model and Simulation of Early-Stage Vision as a Developmental Sensorimotor Process

Theories of embodied cognition and active vision suggest that perception is constructed through interaction and becomes meaningful because it is grounded in the agent’s activity. We developed a model to illustrate and implement these views. Following its intrinsic motivation, the agent autonomously learns to coordinate its motor actions with the information received from its sensory system. Besides illustrating theories of active vision, this model suggests new ways to implement vision and intrinsic motivation in artificial systems. Specifically, we coupled an intrinsically motivated schema mechanism with a visual system. To connect vision with sequences, we made the visual system react to movements in the visual field rather than merely transmitting static patterns.

Modeling Biological Agents Beyond the Reinforcement-learning Paradigm☆

Abstract It is widely acknowledged that biological beings (animals) are not Markov: modelers generally do not model them as agents receiving a complete representation of their environments state in input (except perhaps in simple controlled tasks). In this paper, we claim that biological beings generally cannot recognize rewarding Markov states of their environment either. Therefore, we model them as agents trying to perform rewarding interactions with their environment (interaction-driven tasks), but not as agents trying to reach rewarding states (state-driven tasks). We review two interaction-driven tasks: the AB and AABB task, and implement a non-Markov Reinforcement-Learning (RL) algorithm based upon historical sequences and Q-learning. Results show that this RL algorithm takes significantly longer than a constructivist algorithm implemented previously by Georgeon, Ritter, & Haynes (2009). This is because the constructivist algorithm directly learns and repeats hierarchical sequences of interactions, whereas the RL algorithm spends time learning Q-values. Along with theoretical arguments, these results support the constructivist paradigm for modeling biological agents.

Inverting the Interaction Cycle to Model Embodied Agents

Cognitive architectures should make explicit the conceptual begin and end points of the agent/environment interaction cycle. Most architectures begin with the agent receiving input data representing the environment, and end with the agent sending output data. This paper suggests inverting this cycle: the agent sends output data that specifies an experiment, and receives input data that represents the result of this experiment. This complies with the embodiment paradigm because the input data does not directly represent the environment and does not amount to the agents perception. We illustrate this in an example and propose an assessment method based upon activity-trace analysis.

Autonomous object modeling based on affordances for spatial organization of behavior

We present an architecture for self-motivated agents to organize their behaviors in space according to possibilities of interactions afforded by initially unknown objects. The long-term goal is to design agents that construct their own knowledge of objects through experience, rather than exploiting pre-coded knowledge. Self-motivation is defined here as a tendency to experiment and to respond to behavioral opportunities afforded by the environment. Some interactions have predefined valences that specify inborn behavioral preferences. Over time, the agent learns the relation between its perception of objects and the interactions that they afford, in the form of data structures, called signatures of interaction, which encode the minimal spatial configurations that afford an interaction. The agent keeps track of enacted interactions in a topological spatial memory, to recognize and localize subsequent possibilities of interaction (through their signatures) afforded by surrounding objects. Experiments with a simulated agent and a robot show that they learn to navigate in their environment, taking into account multiple surrounding objects, reaching or avoiding objects according to the valence of the interactions that they afford.