Suhas E. Chelian

A bio-inspired system for spatio-temporal recognition in static and video imagery

This paper presents a bio-inspired method for spatio-temporal recognition in static and video imagery. It builds upon and extends our previous work on a bio-inspired Visual Attention and object Recognition System (VARS). The VARS approach locates and recognizes objects in a single frame. This work presents two extensions of VARS. The first extension is a Scene Recognition Engine (SCE) that learns to recognize spatial relationships between objects that compose a particular scene category in static imagery. This could be used for recognizing the category of a scene, e.g., office vs. kitchen scene. The second extension is the Event Recognition Engine (ERE) that recognizes spatio-temporal sequences or events in sequences. This extension uses a working memory model to recognize events and behaviors in video imagery by maintaining and recognizing ordered spatio-temporal sequences. The working memory model is based on an ARTSTORE1 neural network that combines an ART-based neural network with a cascade of sustained temporal order recurrent (STORE)1 neural networks. A series of Default ARTMAP classifiers ascribes event labels to these sequences. Our preliminary studies have shown that this extension is robust to variations in an objects motion profile. We evaluated the performance of the SCE and ERE on real datasets. The SCE module was tested on a visual scene classification task using the LabelMe2 dataset. The ERE was tested on real world video footage of vehicles and pedestrians in a street scene. Our system is able to recognize the events in this footage involving vehicles and pedestrians.

Bio-inspired visual attention and object recognition

This paper describes a bio-inspired Visual Attention and Object Recognition System (VARS) that can (1) learn representations of objects that are invariant to scale, position and orientation; and (2) recognize and locate these objects in static and video imagery. The system uses modularized bio-inspired algorithms/techniques that can be applied towards finding salient objects in a scene, recognizing those objects, and prompting the user for additional information to facilitate interactive learning. These algorithms are based on models of human visual attention, search, recognition and learning. The implementation is highly modular, and the modules can be used as a complete system or independently. The underlying technologies were carefully researched in order to ensure they were robust, fast, and could be integrated into an interactive system. We evaluated our systems capabilities on the Caltech-101 and COIL-100 datasets, which are commonly used in machine vision, as well as on simulated scenes. Preliminary results are quite promising in that our system is able to process these datasets with good accuracy and low computational times.

Model of the interactions between neuromodulators and prefrontal cortex during a resource allocation task

Neuromodulators such as dopamine (DA), serotonin (5-HT), and acetylcholine (ACh) are crucial to the representations of reward, cost, and attention respectively. Recent experiments suggest that the reward and cost of actions are also partially represented in orbitofrontal and anterior cingulate cortices in that order. Previous models of action selection with neuromodulatory systems have not extensively considered prefrontal contributions to action selection. Here, we extend these models of action selection to include prefrontal structures in a resource allocation task. The model adapts to its environment, modulating its aggressiveness based on its successes. Selective lesions demonstrate how neuromodulatory and prefrontal areas drive learning and performance of strategy selection.

Bio-inspired method and system for actionable intelligence

This paper describes a bio-inspired VISion based actionable INTelligence system (VISINT) that provides automated capabilities to (1) understand objects, patterns, events and behaviors in vision data; (2) translate this understanding into timely recognition of novel and anomalous entities; and (3) discover underlying hierarchies and relationships between disparate labels entered by multiple users to provide a consistent data representation. VISINT is both a system and a novel collection of novel bio-inspired algorithms/modules. These modules can be used independently for various aspects of the actionable intelligence problem or sequenced together for an end-to-end actionable intelligence system. The algorithms can be useful in many other applications such as scene understanding, behavioral analysis, automatic surveillance systems, etc. The bio-inspired algorithms are a novel combination of hierarchical spatial and temporal networks based on the Adaptive Resonance Theory (ART). The novel aspects of this work are that it is an end-to-end system for actionable intelligence that combines existing and novel implementations of various modules in innovative ways to develop a system concept for actionable intelligence. Although there are other algorithms/implementations of several of the modules in VISINT, they suffer from various limitations and often system integration is not considered. The overall VISINT system can be viewed an incremental learning system where no offline training is required and data from multiple sources and times can be seamlessly integrated. The user is in the loop, but due to the semi-supervised nature of the underlying algorithms, only significant variations of entities, not all false alarms, are shown to the user. It does not forget the past even with new learning. While VISINT is designed as a vision-based system, it could also work with other kinds of sensor data that can recognize and locate individual objects in the scene. Beyond that stage of object recognition and localization, all aspects of VISINT are applicable to other kinds of sensor data.

The neural basis of decision-making during sensemaking: Implications for human-system interaction

We have created a high-fidelity model of 9 regions of the brain involved in making sense of complex and uncertain situations. Sense making is a proactive form of situation awareness requiring sifting through information of various types to form hypotheses about evolving situations. The MINDS model (Mirroring Intelligence in a Neural Description of Sensemaking) reveals the neural principles and cognitive tradeoffs that explain weaknesses in human reasoning and decision-making.

Learning to Prognostically Forage in a Neural Network Model of the Interactions between Neuromodulators and Prefrontal Cortex

Abstract Neuromodulatory systems and prefrontal cortex are involved in a number of decision-making contexts. In this work, we adapt a recent neural network model that simulates interactions between neuromodulatory and prefrontal areas to the problem of prognostic foraging—that is choosing information to update or form a hypothesis. In the context of a simulated geospatial intelligence task, the model assesses a number of decision variables and strategies to choose actions that maximize information utility to more accurately predict the actions of an adversary. The model is also capable of modeling biases in decision making such as deviations from the optimal solution of maximizing information gain. Comparisons to other approaches and problem domains in information foraging are also discussed.

A spiking thalamus model for form and motion processing of images

The thalamus, far from being a simple relay, supports several functions including attention and awareness. Recent spiking models of the thalamus tend to focus on abstract thalamocortical features such as rhythms and synchrony. Here a new spiking retino-thalamic model is presented that reproduces several aspects in visual processing including distinct form and motion processing pathways. Using test and natural image sequences, differences between parvocellular and magnocellular relay neurons are studied. In line with several experimental results, parvocellular neurons are found to be more sensitive to changes in color (necessary for form processing) than temporal frequency (necessary for motion processing) and conversely for magnocellular neurons. This model can in turn be used as input into subsequent cortical models or as a tool to aid in experimentation. Future extensions could include modeling brainstem or cortical influence on thalamic processing, as well as the control of virtual agents.

Reinforcement learning and instance-based learning approaches to modeling human decision making in a prognostic foraging task

Procedural memory and episodic memory are known to be distinct and both underlie the performance of many tasks. Reinforcement learning (RL) and instance-based learning (IBL) represent common approaches to modeling procedural and episodic memory in that order. In this work, we present a neural model utilizing RL dynamics and an ACT-R model utilizing IBL productions to the task of modeling human decision making in a prognostic foraging task. The task performed was derived from a geospatial intelligence domain wherein agents must choose among information sources to more accurately predict the actions of an adversary. Results from both models are compared to human data and suggest that information gain is an important component in modeling decision-making behavior using either memory system; with respect to the episodic memory approach, the procedural memory approach has a small but significant advantage in fitting human data. Finally, we discuss the interactions of multi-memory systems in complex decision-making tasks.

DISCOV (DImensionless Shunting COlor Vision): A neural model for spatial data analysis

The DISCOV (DImensionless Shunting COlor Vision) system models a cascade of primate color vision neurons: retinal ganglion, thalamic single opponent, and cortical double opponent. A unified model derived from psychophysical axioms produces transparent network dynamics and principled parameter settings. DISCOV fits an array of physiological data for each cell type, and makes testable experimental predictions. Binary DISCOV augments an earlier version of the model to achieve stable computations for spatial data analysis. The model is described in terms of RGB images, but inputs may consist of any number of spatially defined components. System dynamics are derived using algebraic computations, and robust parameter ranges that meet experimental data are fully specified. Assuming default values, the only free parameter for the user to specify is the spatial scale. Multi-scale analysis accommodates items of various sizes and perspective. Image inputs are first processed by complement coding, which produces an ON channel stream and an OFF channel stream for each component. Subsequent computations are on-center/off-surround, with the OFF channel replacing the off-center/on-surround fields of other models. Together with an orientation filter, DISCOV provides feature input vectors for an integrated recognition system. The development of DISCOV models is being carried out in the context of a large-scale research program that is integrating cognitive and neural systems derived from analyses of vision and recognition to produce both biological models and technological applications.