Encarni Marcos

Neural Variability in Premotor Cortex Is Modulated by Trial History and Predicts Behavioral Performance

In the study of decision making, emphasis is placed on different forms of perceptual integration, while the influence of other factors, such as memory, is ignored. In addition, it is believed that the information underlying decision making is carried in the rate of the neuronal response, while its variability is considered unspecific. Here we studied the influence of recent experience on motor decision making by analyzing the activity of neurons in the dorsal premotor area of two monkeys performing a countermanding arm task. We observe that the across-trial variability of the neural response strongly correlates with trial history-dependent changes in reaction time. Using a theoretical model of decision making, we show that a trial history-monitoring signal can explain the observed behavioral and neural modulation. Our study reveals that, in the neural processes that culminate in motor plan maturation, the evidence provided by perception and memory is reflected in mean rate and variance respectively.

Evidence against perfect integration of sensory information during perceptual decision making

Perceptual decision making is often modeled as perfect integration of sequential sensory samples until the accumulated total reaches a fixed decision bound. In that view, the buildup of neural activity during perceptual decision making is attributed to temporal integration. However, an alternative explanation is that sensory estimates are computed quickly with a low-pass filter and combined with a growing signal reflecting the urgency to respond and it is the latter that is primarily responsible for neural activity buildup. These models are difficult to distinguish empirically because they make similar predictions for tasks in which sensory information is constant within a trial, as in most previous studies. Here we presented subjects with a variant of the classic constant-coherence motion discrimination (CMD) task in which we inserted brief motion pulses. We examined the effect of these pulses on reaction times (RTs) in two conditions: 1) when the CMD trials were blocked and subjects responded quickly and 2) when the same CMD trials were interleaved among trials of a variable-motion coherence task that motivated slower decisions. In the blocked condition, early pulses had a strong effect on RTs but late pulses did not, consistent with both models. However, when subjects slowed their decision policy in the interleaved condition, later pulses now became effective while early pulses lost their efficacy. This last result contradicts models based on perfect integration of sensory evidence and implies that motion signals are processed with a strong leak, equivalent to a low-pass filter with a short time constant.

Distributed Adaptive Control: A Proposal on the Neuronal Organization of Adaptive Goal Oriented Behavior

In behavioral motor coordination and interaction it is a fundamental challenge how an agent can learn to perceive and act in unknown and dynamic environments. At present, it is not clear how an agent can – without any explicitly predefined knowledge – acquire internal representations of the world while interacting with the environment. To meet this challenge, we propose a biologically based cognitive architecture called Distributed Adaptive Control (DAC). DAC is organized in three different, tightly coupled, layers of control: reactive, adaptive and contextual. DAC based systems are self-contained and fully grounded, meaning that they autonomously generate representations of their primary sensory inputs, hence bootstrapping their behavior form simple to advance interactions. Following this approach, we have previously identified a novel environmentally mediated feedback loop in the organization of perception and behavior, i.e. behavioral feedback. Additionally, we could demonstrated that the dynamics of the memory structure of DAC, acquired during a foraging task, are equivalent to a Bayesian description of foraging. In this chapter we present DAC in a concise form and show how it is allowing us to extend the different subsystems to more biophysical detailed models. These further developments of the DAC architecture, not only allow to better understand the biological systems, but moreover advance DACs behavioral capabilities and generality.

The Hierarchical Accumulation of Knowledge in the Distributed Adaptive Control Architecture

Animals acquire knowledge as they interact with the world. Several authors define this acquisition as a chain of transformations: data is acquired and converted into information that is converted into knowledge. Moreover, theories on cumulative learning suggest that different cognitive layers accumulate this knowledge, building highly complex skills from low complexity ones. The biologically based Distributed Adaptive Control cognitive architecture (DAC) has been proposed as a cumulative learning system. DAC contains different layers of control: reactive, adaptive and contextual. This hierarchical organization allows for acquisition of knowledge in a bottom-up interaction, i.e. sampled data is transformed into knowledge. DAC has already been used as a framework to investigate fundamental problems encountered in biology. Here we describe the DAC architecture and present some studies focused on its highest cognitive layer where knowledge is constructed and used. We investigate the roles of reactive and contextual control depending on the characteristics and complexity of the tasks. We also show how multi-sensor information could be integrated in order to acquire and use knowledge optimally. Finally, we discuss the possible problems of cumulative learning and the adopted solutions in the context of DAC.

Motor cost influences perceptual decisions

Perceptual decision making has been widely studied using tasks in which subjects are asked to discriminate a visual stimulus and instructed to report their decision with a movement. In these studies, performance is measured by assessing the accuracy of the participants’ choices as a function of the ambiguity of the visual stimulus. Typically, the reporting movement is considered as a mere means of reporting the decision with no influence on the decision-making process. However, recent studies have shown that even subtle differences of biomechanical costs between movements may influence how we select between them. Here we investigated whether this purely motor cost could also influence decisions in a perceptual discrimination task in detriment of accuracy. In other words, are perceptual decisions only dependent on the visual stimulus and entirely orthogonal to motor costs? Here we show the results of a psychophysical experiment in which human subjects were presented with a random dot motion discrimination task and asked to report the perceived motion direction using movements of different biomechanical cost. We found that the pattern of decisions exhibited a significant bias towards the movement of lower cost, even when this bias reduced performance accuracy. This strongly suggests that motor costs influence decision making in visual discrimination tasks for which its contribution is neither instructed nor beneficial.

The neuronal substrate underlying order and interval representations in sequential tasks: A biologically based robot study

Sequence learning tasks depend on the ability to acquire and control the order of actions and their proper timing. Several studies have shown that in sequence learning different areas of the brain are involved when recalling the order of actions and their proper interval. One hypothesis proposes that two separate areas of the brain interact with each other, one computes order while the other would compute the interval. A second hypothesis proposes that one area computes both, order and interval. To better understand how this computation of order and interval might be realized by the brain, we developed a robot based architecture and investigated the behavioral and architectural implications of these two hypothesis: one or two neuronal areas computing order and interval. Using a sequence learning foraging task we show that performance is enhanced in case of distributed processes. However, we show that as a drawback, explicit interval information can not be reconstructed.

Moth-Like Chemo-Source Localization and Classification on an Indoor Autonomous Robot

Olfaction is a crucial sense for many living organisms. Many animals, especially insects, rely heavily on the olfactory sense for encoding and processing different chemical cues in order to perform several tasks such as foraging, predator avoidance, mate finding, communication etc.(22). Yet, olfaction has not been as widely studied as vision or the auditory system in insects. At the same time, robotic platforms capable of searching, locating and classifying odor sources in wind turbulence and in the presence of complex odors have diverse applications ranging from environmental monitoring (21), detection of explosives and other hazardous substances (19), land mine detection (2) to human search and rescue operations. The main challenge thereby is the stable and fast coding and decoding of odors and the localization of the sources (17). In our own recent work, we have proposed an insect-like mapless navigation mechanism which integrates surge-and-cast chemo search, path integration, wind detection and visual landmark navigation on an indoor mobile robot (28). Also, we have proposed a model based on insect navigation that is capable of navigating in highly dynamic environments and our model was compared directly to ant navigational data, with strikingly similar navigational behaviors (26). The problem of ambiguous information, particularly in the navigational context, is also addressed in our recent work (27). Beyond that, we have contributed significantly to modeling insect navigation and designing robotic systems such as: a model of the locust Lobula Giant Movement Detector (LGMD) tested on a high speed robot (29), moth-like odor localization for robots (30), control of an unmanned aerial vehicle using a neuronal model of a fly-locust brain (31; 32), moth-like optomotor anemotactic chemical search for robots (33), and a blimp flight control using a biologically inspired flight control system (34). Despite these advances, several biological systems with relatively simple nervous systems solve the odor localization and classification problem much more efficiently than their artificial counterparts: bees use odor to localize nests, ants use pheromone trails to organize foraging in swarms, lobsters use odor to locate food, the Escherichia bacteria use odors to locate nutrients, male moths use olfaction to locate female mates etc. The odor localization

The complementary roles of allostatic and contextual control systems in foraging tasks

To survive in an unknown environment an animal has to learn how to reach specific goal states. The animal is firstly guided by its reactive behavior motivated by its internal needs. After exploring the environment, contextual information can be used to optimally fulfill these internal needs. However, how a reactive and a contextual control system complement each other is still a fundamental question. Here, we address this problem from the perspective of the Distributed Adaptive Control architecture (DAC). We extend DACs reactive layer with an allostatic control system and integrate it with its contextual control layer. Through robot foraging tasks we test the properties of the allostatic and contextual cozntrol systems and their interaction. We assess how they scale with task complexity. In particular, we show that the behavior generated by the contextual control layer is of particular importance when the system is facing conflict situations.

Integrating neuroscience-based models towards an autonomous biomimetic Synthetic Forager

Foraging can be described as goal-oriented exploration for resources. It exemplifies how animals coordinate complex sensory and effector systems under varying environmental conditions. To emulate the foraging capabilities of natural systems is a major goal for robotics. Therefore, foraging is an excellent paradigm to benchmark novel autonomous control strategies. Here we describe the biomimetic control architecture of the Synthetic Forager (SF), an effort to integrate multiple biologically constrained models of specific perceptual and cognitive processes pertaining to foraging into one general autonomous robot controller. This proposal is built upon the well-established Distributed Adaptive Control (DAC) framework and brings together neuroscience-based models of decision-making, multi-modal sensory processing, localization and mapping and allostatic behavioral control. To show the potential of the SF model we used it to control a high-mobility wheeled robotic platform in three behavioral tasks similar to experimental protocols applied to rodents. We show that the robot can reliably perform cue detection, rule learning and goal-oriented navigation in open environments. We propose that this approach to robotics allows both the study of embodied neuroscience models and the transfer of brain based principles to robotic systems.

Generalization of Integrator Models to Foraging: A Robot Study Using the DAC9 Model

Experimental research on decision making has been mainly focused on binary perceptual tasks. The generally accepted models describing the decision process in these tasks are the integrator models. These models suggest that perceptual evidence is accumulated over time until a decision is made. Therefore, the final decision is based solely on recent perceptual information. In behaviorally more relevant tasks such as foraging, it is however probable, that the current choice also depends on previous experience. To understand the implications of considering previous experience in an integrator model we investigate it using a cognitive architecture (DAC9) with a robot performing foraging tasks. Compared to an instantaneous decision making model we show that an integrator model improves performance and robustness to task complexity. Further we show that it compresses the information stored in memory. This result suggests a change in the way actions are retrieved from memory leading to self-generated actions.