Anthony S. Maida

Using TD learning to simulate working memory performance in a model of the prefrontal cortex and basal ganglia

Delayed-response tasks (DRTs) have been used to assess working memory (WM) processes in human and nonhuman animals. Experiments have shown that the basal ganglia (BG) and dorsolateral prefrontal cortex (DLPFC) subserve DRT performance. Here, we report the results of simulation studies of a systems-level model of DRT performance. The model was trained using the temporal difference (TD) algorithm and uses an actor-critic architecture. The matrisomes of the BG represent the actor and the striosomes represent the critic. Unlike existing models, we hypothesize that the BG subserve the selection of both motor- and cognitive-related information in these tasks. We also assume that the learning of both processes is based on reward presentation. A novel feature of the model is the incorporation of delay-active neurons in the matrisomes, in addition to DLPFC. Another novel feature of the model is the subdivision of the matrisomal neurons into segregated winner-take-all (WTA) networks consisting of delay- versus transiently-active units. Our simulation model proposes a new neural mechanism to account for the occurrence of perseverative responses in WM tasks in striatal-, as well as in prefrontal damaged subjects. Simulation results also show that the model both accounts for the phenomenon of time shifting of dopamine phasic signals and the effects of partial reinforcement and reward magnitude on WM performance at both behavioral and neural levels. Our simulation results also found that the TD algorithm can subserve learning in delayed-reversal tasks.

Neural maps for mobile robot navigation

Neural maps have been recently proposed as an alternative method for mobile robot path planning. However, these proposals are mostly theoretical and primarily concerned with biological plausibility. This paper addresses the applicability of neural maps to mobile robot navigation with focus on efficient implementations It is suggested that neural maps offer a promising alternative compared to the traditional distance transform and harmonic function methods. Applications of neural maps are presented for both global and local navigation. Experimental results (both simulated and real-world on a Nomad 200 mobile robot) demonstrate the validity of the approach. Our work reveals that a key issue for success of the method is the organization of the map that needs to be optimized for the situation at hand.

Maintaining mental models of agents who have existential misconceptions

Abstract This work describes methods for using and maintaining models of agents who have existential misconceptions. Existential misconceptions are situations where agents are in disagreement about the existence of objects in some domain of discourse. This paper characterizes existential misconceptions, discusses related literature, and describes how to construct a model of an agent who has an existential misconception. This paper is concerned with two kinds of existential misconception. These are compression-based and dispersion-based. We provide algorithms to test for these existential misconceptions, to describe them, and to correct one kind of existential misconception (dispersion-based). We prove correctness theorems for these algorithms and have implemented them in COMMON LISP.

Acquisition of visual features through probabilistic spike-timing-dependent plasticity

This paper explores modifications to a feedforward five-layer spiking convolutional network (SCN) of the ventral visual stream [Masquelier, T., Thorpe, S., Unsupervised learning of visual features through spike timing dependent plasticity. PLoS Computational Biology, 3(2), 247-257]. The original model showed that a spike-timing-dependent plasticity (STDP) learning algorithm embedded in an appropriately selected SCN could perform unsupervised feature discovery. The discovered features where interpretable and could effectively be used to perform rapid binary decisions in a classifier. In order to study the robustness of the previous results, the present research examines the effects of modifying some of the components of the original model. For improved biological realism, we replace the original non-leaky integrate-and-fire neurons with Izhikevich-like neurons. We also replace the original STDP rule with a novel rule that has a probabilistic interpretation. The probabilistic STDP slightly but significantly improves the performance for both types of model neurons. Use of the Izhikevich-like neuron was not found to improve performance although performance was still comparable to the IF neuron. This shows that the model is robust enough to handle more biologically realistic neurons. We also conclude that the underlying reasons for stable performance in the model are preserved despite the overt changes to the explicit components of the model.

Using parallel GPU architecture for simulation of planar I/F networks

Our work describes the simulation of a planar network of spiking I/F neurons on graphics processing hardware. The described approach adds to the fast-growing field of general-purpose computation on GPUs (GPGPU). We provide an in-depth explanation of the steps involved in implementing the network using programmable shading hardware. We replicated simulation results by Hopfield et al. [1] and Maida et al. [2] and give qualitative and quantitative measures of our implementation.

CajunBot: Architecture and algorithms

CajunBot, an autonomous ground vehicle and a finalist in the 2005 DARPA Grand Challenge, is built on the chassis of MAX IV, a six-wheeled ATV. Transformation of the ATV to an AGV (Autonomous Ground Vehicle) required adding drive-by-wire control, LIDAR sensors, an INS, and a computing system. Significant innovations in the core computational algorithms include an obstacle detection algorithm that takes advantage of shocks and bumps to improve visibility; a path planning algorithm that takes into account the vehicle’s maneuverability limits to generate paths that are navigable at high speed; efficient data structures and algorithms that require just a single Intel Pentium 4 HT 3.2 Ghz machine to handle all computations and a middleware layer that transparently distributes the computation to multiple machines, if desired. In addition, CajunBot also features support technologies such as a simulator, playback of logged data and live visualization on off-board computers to aid in development, testing, and debugging.

Selecting a humanly understandable knowledge representation for reasoning about knowledge

Three formalisms for representing knowledge about knowledge are briefly examined from the point of view of allowing a computer program to communicate its knowledge to a human. The first two formalisms are philosophically motivated and the last is psychologically motivated. Although all three formalisms are adequate for the purposes of valid inference in this problem domain, it is argued that the psychologically motivated formalism is the most useful of the three for the purposes of man-machine communication. The first two formalisms express more distinctions than a human would, when reasoning about the same problem, whereas the last formalism express the right number of distinctions.

A spiking network that learns to extract spike signatures from speech signals

Spiking neural networks (SNNs) with adaptive synapses reflect core properties of biological neural networks. Speech recognition, as an application involving audio coding and dynamic learning, provides a good test problem to study SNN functionality. We present a simple, novel, and efficient nonrecurrent SNN that learns to convert a speech signal into a spike train signature. The signature is distinguishable from signatures for other speech signals representing different words, thereby enabling digit recognition and discrimination in devices that use only spiking neurons. The method uses a small, nonrecurrent SNN consisting of Izhikevich neurons equipped with spike timing dependent plasticity (STDP) and biologically realistic synapses. This approach introduces an efficient and fast network without error-feedback training, although it does require supervised training. The new simulation results produce discriminative spike train patterns for spoken digits in which highly correlated spike trains belong to the same category and low correlated patterns belong to different categories. The proposed SNN is evaluated using a spoken digit recognition task where a subset of the Aurora speech dataset is used. The experimental results show that the network performs well in terms of accuracy rate and complexity.

Using temporal binding for hierarchical recruitment of conjunctive concepts over delayed lines

The temporal correlation hypothesis proposes using distributed synchrony for the binding of different stimulus features. However, synchronized spikes must travel over cortical circuits that have varying-length pathways, leading to mismatched arrival times. This raises the question of how initial stimulus-dependent synchrony might be preserved at a destination binding site. Earlier, we proposed constraints on tolerance and segregation parameters for a phase-coding approach, within cortical circuits, to address this question [C. Gunay, A.S. Maida, Temporal binding as an inducer for connectionist recruitment learning over delayed lines, Neural Networks 16 (5-6) (2003) 593-600]. The purpose of the present paper is twofold. First, we conduct simulation studies that explore the effectiveness of the proposed constraints. Second, we place the studies in a broader context of synchrony-driven recruitment learning [L. Shastri, V. Ajjanagadde, From simple associations to systematic reasoning: a connectionist representation of rules, variables, and dynamic bindings using temporal synchrony, Behav. Brain Sci. 16 (3) (1993) 417-451; L.G. Valiant, Circuits of the Mind, Oxford University Press, Oxford, 1994] which brings together von der Malsburgs temporal binding [C. von der Malsburg, The correlation theory of brain function, in: E. Domany, J.L. van Hemmen, K. Schulten (Ed.), Models of Neural Networks, vol. 2, Physics of Neural Networks, Chapter 2, Springer, New York, 1994, pp. 95-120, (Originally appeared as a Technical Report at the Max-Planck Institute for Biophysical Chemistry, Gottingen, 1981)] and Feldmans recruitment learning [J.A. Feldman, Dynamic connections in neural networks, Biol. Cybern. 46 (1982) 27-39]. A network based on Valiants neuroidal architecture is used to implement synchrony-driven recruitment learning. Complementing similar approaches, we use a continuous-time learning procedure allowing computation with spiking neurons. The viability of the proposed binding scheme is investigated by conducting simulation studies which examine binding errors. In the simulation, binding errors cause the formation of illusory conjunctions among features belonging to separate objects. Our results indicate that when tolerance and segregation parameters obey our proposed constraints, the sets of correct bindings are dominant over sets of spurious bindings in reasonable operating conditions. We also improve the stability of the recruitment method in deep hierarchies for use in limited size structures suitable for computer simulations. e also improve the stability of the recruitment method in deep hierarchies for use in limited size structures suitable for computer simulations.

Subgoal-based local navigation and obstacle avoidance using a grid-distance field

The local path-planning and obstacle-avoidance module used in the CajunBot, six-wheeled, all-terrain, autonomous land rover is described. The module is designed for rapid subgoal extraction in service of a global navigation system that follows GPS-supplied waypoints. The core algorithm is built around a grid-based, linear-activation field (a type of artificial potential field). The local path planner has three novel features: the artificial potential field delivers local waypoints, or navigation subgoals, rather than a gradient; the planner aggressively avoids obstacles; and, the algorithm makes use of a repulsive expansion region to compensate for imperfect manoeuvrability.(A)