Serge Thill

Theories and computational models of affordance and mirror systems: An integrative review

Neuroscientific and psychological data suggest a close link between affordance and mirror systems in the brain. However, we still lack a full understanding of both the individual systems and their interactions. Here, we propose that the architecture and functioning of the two systems is best understood in terms of two challenges faced by complex organisms, namely: (a) the need to select among multiple affordances and possible actions dependent on context and high-level goals and (b) the exploitation of the advantages deriving from a hierarchical organisation of behaviour based on actions and action-goals. We first review and analyse the psychological and neuroscientific literature on the mechanisms and processes organisms use to deal with these challenges. We then analyse existing computational models thereof. Finally we present the design of a computational framework that integrates the reviewed knowledge. The framework can be used both as a theoretical guidance to interpret empirical data and design new experiments, and to design computational models addressing specific problems debated in the literature.

Sentence processing: linking language to motor chains

A growing body of evidence in cognitive science and neuroscience points towards the existence of a deep interconnection between cognition, perception and action. According to this embodied perspective language is grounded in the sensorimotor system and language understanding is based on a mental simulation process (Jeannerod, 2007; Gallese, 2008; Barsalou, 2009). This means that during action words and sentence comprehension the same perception, action, and emotion mechanisms implied during interaction with objects are recruited. Among the neural underpinnings of this simulation process an important role is played by a sensorimotor matching system known as the mirror neuron system (Rizzolatti and Craighero, 2004). Despite a growing number of studies, the precise dynamics underlying the relation between language and action are not yet well understood. In fact, experimental studies are not always coherent as some report that language processing interferes with action execution while others find facilitation. In this work we present a detailed neural network model capable of reproducing experimentally observed influences of the processing of action-related sentences on the execution of motor sequences. The proposed model is based on three main points. The first is that the processing of action-related sentences causes the resonance of motor and mirror neurons encoding the corresponding actions. The second is that there exists a varying degree of crosstalk between neuronal populations depending on whether they encode the same motor act, the same effector or the same action-goal. The third is the fact that neuronal populations’ internal dynamics, which results from the combination of multiple processes taking place at different time scales, can facilitate or interfere with successive activations of the same or of partially overlapping pools.

Robot-assisted therapy for autism spectrum disorders with (partially) autonomous control: Challenges and outlook

Robot-assisted therapy (RAT) is an emerging field that has already seen some success and is likely to develop in the future. One particular application area is within therapies for autism spectrum disorders, in which the viability of the approach has been demonstrated.The present paper is a vision paper with the aim of identifying research directions in the near future of RAT. Specifically, we argue that the next step in such therapeutic scenarios is the development of more substantial levels of autonomy which would allow the robot to adapt to the individual needs of children over longer periods of time (while remaining under the ultimate supervision of a therapist). We argue that this requires new advances on the level of robot controllers as well as the ability to infer and classify intentions, goals and emotional states of the robot’s interactants. We show that the state of the art in a number of relevant disciplines is now at the point at which such an endeavour can be approached in earnest.

Affordances, Adaptive Tool Use and Grounded Cognition

The role of affordances In our opinion the model would be improved if affordance for action mechanisms were integrated: The potentiation of motor interactions consistent with the conventional use of a perceived tool (Bub et al., 2008). Affordances would easily explain the mechanism underlying the quick choice of the most appropriate tool when an action goal is given. In Figure 1A, bidirectional connections between available tools and usage contexts represent the automatic activation of stable affordances (Borghi and Riggio, 2009) emerging from invariant features of the tool (mainly its functional meaning) incorporated into the object representation in long term memory (e.g., a spoon typically stirs and scoops, a knife cuts and spreads). Activation of stable affordances is independent of context or task to be carried out. Furthermore, for each stable affordance, variable affordances emerging from temporary characteristics of the tools such as their current handle orientation are also activated. When an action goal is given (stir the coffee), attention is restricted to a specific action context (to stir) and oriented to available and action-consistent tools (spoon and stirrer). Thus, activation of a restricted number of stable and variable affordances is privileged (e.g., “spoon + to stir” and “stirrer + to stir” with the right hand) over those not consistent with the action goal. Here, the controller will select “spoon + to stir” instead of “stirrer + to stir” and execute the required action with the hand corresponding to the handle orientation. Selection between two equally appropriate tools is assumed to be determined by stronger stable and variable affordances being activated. A commentary on

Embodied cognition and circular causality : On the role of constitutive autonomy in the reciprocal coupling of perception and action

The reciprocal coupling of perception and action in cognitive agents has been firmly established: perceptions guide action but so too do actions influence what is perceived. While much has been said on the implications of this for the agents external behavior, less attention has been paid to what it means for the internal bodily mechanisms which underpin cognitive behavior. In this article, we wish to redress this by reasserting that the relationship between cognition, perception, and action involves a constitutive element as well as a behavioral element, emphasizing that the reciprocal link between perception and action in cognition merits a renewed focus on the system dynamics inherent in constitutive biological autonomy. Our argument centers on the idea that cognition, perception, and action are all dependent on processes focussed primarily on the maintenance of the agents autonomy. These processes have an inherently circular nature—self-organizing, self-producing, and self-maintaining—and our goal is to explore these processes and suggest how they can explain the reciprocity of perception and action. Specifically, we argue that the reciprocal coupling is founded primarily on their endogenous roles in the constitutive autonomy of the agent and an associated circular causality of global and local processes of self-regulation, rather than being a mutual sensory-motor contingency that derives from exogenous behavior. Furthermore, the coupling occurs first and foremost via the internal milieu realized by the agents organismic embodiment. Finally, we consider how homeostasis and the related concept of allostasis contribute to this circular self-regulation.

How to Build a Supervised Autonomous System for Robot-Enhanced Therapy for Children with Autism Spectrum Disorder

Abstract Robot-Assisted Therapy (RAT) has successfully been used to improve social skills in children with autism spectrum disorders (ASD) through remote control of the robot in so-called Wizard of Oz (WoZ) paradigms.However, there is a need to increase the autonomy of the robot both to lighten the burden on human therapists (who have to remain in control and, importantly, supervise the robot) and to provide a consistent therapeutic experience. This paper seeks to provide insight into increasing the autonomy level of social robots in therapy to move beyond WoZ. With the final aim of improved human-human social interaction for the children, this multidisciplinary research seeks to facilitate the use of social robots as tools in clinical situations by addressing the challenge of increasing robot autonomy.We introduce the clinical framework in which the developments are tested, alongside initial data obtained from patients in a first phase of the project using a WoZ set-up mimicking the targeted supervised-autonomy behaviour. We further describe the implemented system architecture capable of providing the robot with supervised autonomy.

What's on the Inside Counts: A Grounded Account of Concept Acquisition and Development

Understanding the factors which affect the age of acquisition (AoA) of words and concepts is fundamental to understanding cognitive development more broadly. Traditionally, studies of AoA have taken two approaches, either exploring the effect of linguistic variables such as input frequency (e.g., Naigles and Hoff-Ginsberg, 1998) or the semantics of the underlying concept, such as concreteness or imageability (e.g., Bird et al., 2001). Embodied theories of cognition, meanwhile, assume that concepts, even relatively abstract ones, can be grounded in the embodied experience. While the focus of such discussions has been mainly on grounding in external modalities, more recently some have argued for the importance of interoceptive features, or grounding in complex modalities such as social interaction. In this paper, we argue for the integration and extension of these two strands of research. We demonstrate that the psycholinguistic factors traditionally considered to determine AoA are far from sufficient to account for the variability observed in AoA data. Given this gap, we propose groundability as a new conceptual tool that can measure the degree to which concepts are grounded both in external and, critically, internal modalities. We then present a mechanistic theory of conceptual representation that can account for groundability in addition to the existing variables argued to influence concept acquisition in both the developmental and embodied cognition literatures, and discuss its implications for future work in concept and cognitive development.

Dreaming of electric sheep? Exploring the functions of dream-like mechanisms in the development of mental imagery simulations

According to the simulation hypothesis, mental imagery can be explained in terms of predictive chains of simulated perceptions and actions, i.e., perceptions and actions are reactivated internally by our nervous system to be used in mental imagery and other cognitive phenomena. Our previous research shows that it is possible but not trivial to develop simulations in robots based on the simulation hypothesis. While there are several previous approaches to modelling mental imagery and related cognitive abilities, the origin of such internal simulations has hardly been addressed. The inception of simulation (InSim) hypothesis suggests that dreaming has a function in the development of simulations by forming associations between experienced, non-experienced but realistic, and even unrealistic perceptions. Here, we therefore develop an experimental set-up based on a simple simulated robot to test whether such dream-like mechanisms can be used to instruct research into the development of simulations and mental imagery-like abilities. Specifically, the hypothesis is that ‘dreams’ informing the construction of simulations lead to faster development of good simulations during waking behaviour. The paper presents initial results in favour of the hypothesis.

Modeling the Development of Goal-Specificity in Mirror Neurons

Neurophysiological studies have shown that parietal mirror neurons encode not only actions but also the goal of these actions. Although some mirror neurons will fire whenever a certain action is perceived (goal-independently), most will only fire if the motion is perceived as part of an action with a specific goal. This result is important for the action-understanding hypothesis as it provides a potential neurological basis for such a cognitive ability. It is also relevant for the design of artificial cognitive systems, in particular robotic systems that rely on computational models of the mirror system in their interaction with other agents. Yet, to date, no computational model has explicitly addressed the mechanisms that give rise to both goal-specific and goal-independent parietal mirror neurons. In the present paper, we present a computational model based on a self-organizing map, which receives artificial inputs representing information about both the observed or executed actions and the context in which they were executed. We show that the map develops a biologically plausible organization in which goal-specific mirror neurons emerge. We further show that the fundamental cause for both the appearance and the number of goal-specific neurons can be found in geometric relationships between the different inputs to the map. The results are important to the action-understanding hypothesis as they provide a mechanism for the emergence of goal-specific parietal mirror neurons and lead to a number of predictions: (1) Learning of new goals may mostly reassign existing goal-specific neurons rather than recruit new ones; (2) input differences between executed and observed actions can explain observed corresponding differences in the number of goal-specific neurons; and (3) the percentage of goal-specific neurons may differ between motion primitives.

Learning new motion primitives in the mirror neuron system: a self-organising computational model

Computational models of the mirror (neuron) system are attractive in robotics as they may inspire novel approaches to implement e.g. action understanding. Here, we present a simple self-organising map which forms the first part of larger ongoing work in building such a model. We show that minor modifications to the standard implementation of such a map allows it to continuously learn new motor concepts. We find that this learning is facilitated by an initial motor babbling phase, which is in line with an embodied view of cognition. Interestingly, we also find that the map is capable of reproducing neurophysiological data on goal-encoding mirror neurons. Overall, our model thus fulfils the crucial requirement of being able to learn new information throughout its lifetime. Further, although conceptually simple, its behaviour has interesting parallels to both cognitive and neuroscientific evidence.