Zenon Mathews

PASAR: An integrated model of prediction, anticipation, sensation, attention and response for artificial sensorimotor systems

A wide range of neuroscientific studies suggest the existence of cognitive mechanisms like attention, prediction, anticipation and strong vertical interactions between different hierarchical layers of the brain while performing complex tasks. Despite advances in both cognitive brain research and in the development of brain-inspired artificial cognitive systems, the interplay of these key ingredients of cognition remain largely elusive and unquantified in complex real-world tasks. Furthermore, it has not yet been demonstrated how a self-contained hierarchical cognitive system acting under limited resource constraints can quantifiably benefit from the incorporation of top-down and bottom-up attentional mechanisms. In this context, an open fundamental question is how a data association mechanism can integrate bottom-up sensory information and top-down knowledge. Here, building on the Distributed Adaptive Control (DAC) architecture, we propose a single framework for integrating these different components of cognition and demonstrate the frameworks performance in solving real-world and simulated robot tasks. Using the model we quantify the interactions between prediction, anticipation, attention and memory. Our results support the strength of a complete system that incorporates attention, prediction and anticipation mechanisms compared to incomplete systems for real-world and complex tasks. We unveil the relevance of transient memory that underlines the utility of the above mechanisms for intelligent knowledge management in artificial sensorimotor systems. These findings provide concrete predictions for physiological and psychophysical experiments to validate our model in biological cognitive systems.

The eXperience Induction Machine: A New Paradigm for Mixed-Reality Interaction Design and Psychological Experimentation

The eXperience Induction Machine (XIM) is one of the most advanced mixed-reality spaces available today. XIM is an immersive space that consists of physical sensors and effectors and which is conceptualized as a general-purpose infrastructure for research in the field of psychology and human–artifact interaction. In this chapter, we set out the epistemological rational behind XIM by putting the installation in the context of psychological research. The design and implementation of XIM are based on principles and technologies of neuromorphic control. We give a detailed description of the hardware infrastructure and software architecture, including the logic of the overall behavioral control. To illustrate the approach toward psychological experimentation, we discuss a number of practical applications of XIM. These include the so-called, persistent virtual community, the application in the research of the relationship between human experience and multi-modal stimulation, and an investigation of a mixed-reality social interaction paradigm.

Insect-Like mapless navigation based on head direction cells and contextual learning using chemo-visual sensors

We present a novel biomimetic approach to mapless autonomous navigation based on insect neuroethology. We implemented and tested a real-time neuronal model based on the Distributed Adaptive Control framework. The model unifies different aspects of insect navigation and foraging including landmark recognition, chemical search, path integration and optimal memory usage. Consistent with recent findings the model supports navigation using heading direction information, thus precluding the use of global information. We tested our model using a mobile robot performing a foraging task. While foraging for chemical sources in a wind tunnel, the robot memorizes the followed trajectories, using information from landmarks and heading direction accumulators. After foraging, landmark navigation is tested with the odor source turned off. Our results show stability against robot kidnapping and generalization of homing behavior to stable mapless landmark navigation. This demonstrates that allocentric and efficient goal-oriented navigation strategies can be generated by relying on purely local information.

An insect-based method for learning landmark reliability using expectation reinforcement in dynamic environments

Navigation in unknown dynamic environments still remains a major challenge in robotics. Whereas insects like the desert ant with very limited computing and memory capacities solve this task with great efficiency. Thus, the understanding of the underlying neural mechanisms of insect navigation can inform us on how to build simpler yet robust autonomous robots. Based on recent developments in insect neuroethology and cognitive psychology, we propose a method for landmark navigation in dynamic environments. Our method enables the navigator to learn the reliability of landmarks using an expectation reinforcement method. For that end, we implemented a real-time neuronal model based on the Distributed Adaptive Control framework. The results demonstrate that our model is capable of learning the stability of landmarks by reinforcing its expectations. Also, the proposed mechanism allows the navigator to optimally restore its confidence when its expectations are violated. We also perform navigational experiments with real ants to compare with the results of our model. The behavior of the proposed autonomous navigator closely resembles real ant navigational behavior. Moreover, our model explains navigation in dynamic environments as a memory consolidation process, harnessing expectations and their violations.

Visual anticipation biases conscious decision making but not bottom-up visual processing

Prediction plays a key role in control of attention but it is not clear which aspects of prediction are most prominent in conscious experience. An evolving view on the brain is that it can be seen as a prediction machine that optimizes its ability to predict states of the world and the self through the top-down propagation of predictions and the bottom-up presentation of prediction errors. There are competing views though on whether prediction or prediction errors dominate the formation of conscious experience. Yet, the dynamic effects of prediction on perception, decision making and consciousness have been difficult to assess and to model. We propose a novel mathematical framework and a psychophysical paradigm that allows us to assess both the hierarchical structuring of perceptual consciousness, its content and the impact of predictions and/or errors on conscious experience, attention and decision-making. Using a displacement detection task combined with reverse correlation, we reveal signatures of the usage of prediction at three different levels of perceptual processing: bottom-up fast saccades, top-down driven slow saccades and consciousnes decisions. Our results suggest that the brain employs multiple parallel mechanism at different levels of perceptual processing in order to shape effective sensory consciousness within a predicted perceptual scene. We further observe that bottom-up sensory and top-down predictive processes can be dissociated through cognitive load. We propose a probabilistic data association model from dynamical systems theory to model the predictive multi-scale bias in perceptual processing that we observe and its role in the formation of conscious experience. We propose that these results support the hypothesis that consciousness provides a time-delayed description of a task that is used to prospectively optimize real time control structures, rather than being engaged in the real-time control of behavior itself.

Intelligent motor decision: From selective attention to a Bayesian world model

Intelligent sensor/motor allocation is gaining in importance in many areas of robotics and autonomous systems. It allows the autonomous entity to allocate its resources for solving the currently most critical task depending on the entitypsilas current state, its sensory input and its acquired knowledge of the world. Such architectures which support dynamic motor allocation are invaluable for systems with limited resources. Biological systems also build and maintain a world-model to enable intelligent motor decision making. Based on recent advances in attention research and psychophysiology we propose a general purpose push-pull selective attention mechanism for building a world model and intelligent motor action control. We implement and test an architecture called A-BID, which is guided by a neural network implementation of a selective attention mechanism that is used to build a probabilistic world model. Using A-BID, the system performs at each time step the action that is optimal in the Bayesian sense.

Differential neural mechanisms for early and late prediction error detection

Emerging evidence indicates that prediction, instantiated at different perceptual levels, facilitate visual processing and enable prompt and appropriate reactions. Until now, the mechanisms underlying the effect of predictive coding at different stages of visual processing have still remained unclear. Here, we aimed to investigate early and late processing of spatial prediction violation by performing combined recordings of saccadic eye movements and fast event-related fMRI during a continuous visual detection task. Psychophysical reverse correlation analysis revealed that the degree of mismatch between current perceptual input and prior expectations is mainly processed at late rather than early stage, which is instead responsible for fast but general prediction error detection. Furthermore, our results suggest that conscious late detection of deviant stimuli is elicited by the assessment of prediction error’s extent more than by prediction error per se. Functional MRI and functional connectivity data analyses indicated that higher-level brain systems interactions modulate conscious detection of prediction error through top-down processes for the analysis of its representational content, and possibly regulate subsequent adaptation of predictive models. Overall, our experimental paradigm allowed to dissect explicit from implicit behavioral and neural responses to deviant stimuli in terms of their reliance on predictive models.

Moth-Like Chemo-Source Localization and Classification on an Indoor Autonomous Robot

Olfaction is a crucial sense for many living organisms. Many animals, especially insects, rely heavily on the olfactory sense for encoding and processing different chemical cues in order to perform several tasks such as foraging, predator avoidance, mate finding, communication etc.(22). Yet, olfaction has not been as widely studied as vision or the auditory system in insects. At the same time, robotic platforms capable of searching, locating and classifying odor sources in wind turbulence and in the presence of complex odors have diverse applications ranging from environmental monitoring (21), detection of explosives and other hazardous substances (19), land mine detection (2) to human search and rescue operations. The main challenge thereby is the stable and fast coding and decoding of odors and the localization of the sources (17). In our own recent work, we have proposed an insect-like mapless navigation mechanism which integrates surge-and-cast chemo search, path integration, wind detection and visual landmark navigation on an indoor mobile robot (28). Also, we have proposed a model based on insect navigation that is capable of navigating in highly dynamic environments and our model was compared directly to ant navigational data, with strikingly similar navigational behaviors (26). The problem of ambiguous information, particularly in the navigational context, is also addressed in our recent work (27). Beyond that, we have contributed significantly to modeling insect navigation and designing robotic systems such as: a model of the locust Lobula Giant Movement Detector (LGMD) tested on a high speed robot (29), moth-like odor localization for robots (30), control of an unmanned aerial vehicle using a neuronal model of a fly-locust brain (31; 32), moth-like optomotor anemotactic chemical search for robots (33), and a blimp flight control using a biologically inspired flight control system (34). Despite these advances, several biological systems with relatively simple nervous systems solve the odor localization and classification problem much more efficiently than their artificial counterparts: bees use odor to localize nests, ants use pheromone trails to organize foraging in swarms, lobsters use odor to locate food, the Escherichia bacteria use odors to locate nutrients, male moths use olfaction to locate female mates etc. The odor localization

Action-Planning and Execution from Multimodal Cues: An Integrated Cognitive Model for Artificial Autonomous Systems

Using multimodal sensors to perceive the environment and subsequently performing intelligent sensor/motor allocation is of crucial interest for building autonomous systems. Such a capability should allow autonomous entities to (re)allocate their resources for solving their most critical tasks depending on their current state, sensory input and knowledge about the world. Architectures of artificial real-world systems with internal representation of the world and such dynamic motor allocation capabilities are invaluable for systems with limited resources. Based upon recent advances in attention research and psychophysiology we propose a general purpose selective attention mechanism that supports the construction of a world model and subsequent intelligent motor control. We implement and test this architecture including its selective attention mechanism, to build a probabilistic world model. The constructed world-model is used to select actions by means of a Bayesian inference method. Our method is tested in a multi-robot task, both in simulation and in the real world, including a coordination mission involving aerial and ground vehicles.

re(PER)curso: an interactive mixed reality chronicle

re(PER)curso presents an interactive mixed reality narrative where two human performers – a percussionist and a dancer and a number of real-time synthetic actors including sonification, virtual cameras and an anthropomorphic avatar, explore the confluence of the physical and the virtual dimensions underlying existence and experience (Figure 1). The synthetic components of re(PER)curso are realized with computer generated graphics, automated moving light and stage control, video art, a synthetic music composition system called RoBoser [Manzolli and Verschure 2005], and an avatar embedded in a 3D graphic environment. The integration of all elements is realized through the multi-modal mixed reality system the eXperience Induction Machine (XIM) that is based on an earlier large scale public exhibition called Ada [Eng 2003]. XIM is controlled through a neuromorphic system that defines all the rules of interaction and performance dynamics and as a result the complete performance is synthesized in real-time and evolves without human intervention beyond that of the two human actors on the stage. re(PER)curso is an experiment in interactive narrative and explores the potential of virtual reality and augmented feedback technologies as tools for artistic expression. It expresses a general research strategy where the limits of advanced technologies are explored through their application in art. re(PER)curso is operated as an autonomous interactive installation that is augmented by 2 human performers. It is supported by a number of input devices that track and analyze the ongoing performance through cameras and microphones; controllers such as the synthetic composition engine RoBoser and output systems that include the large-scale real-time computer graphics, moving virtual and real cameras, and moving lights. Stage information obtained by the tracking systems is also projected onto the virtual world where it modulates the avatar’s behavior allowing it to adjust body position, posture and gaze to the physical world and to adjust properties of the virtual cameras.