Gennady Livitz

Review and unification of learning framework in Cog Ex Machina platform for memristive neuromorphic hardware

Realizing adaptive brain functions subserving perception, cognition, and motor behavior on biological temporal and spatial scales remains out of reach for even the fastest computers. Newly introduced memristive hardware approaches open the opportunity to implement dense, low-power synaptic memories of up to 1015 bits per square centimeter. Memristors have the unique property of “remembering” the past history of their stimulation in their resistive state and do not require power to maintain their memory, making them ideal candidates to implement large arrays of plastic synapses supporting learning in neural models. Over the past decades, many learning rules have been proposed in the literature to explain how neural activity shapes synaptic connections to support adaptive behavior. To ensure an optimal implementation of a large variety of learning rules in hardware, some general and easily parameterized form of learning rule must be designed. This general form learning equation would allow instantiation of multiple learning rules through different parameterizations, without rewiring the hardware. This paper characterizes a subset of local learning rules amenable to implementation in memristive hardware. The analyzed rules belong to four broad classes: Hebb rule derivatives with various methods for gating learning and decay, Threshold rule variations including the covariance and BCM families, Input reconstruction-based learning rules, and Explicit temporal trace-based rules.

The Animat: New Frontiers in Whole Brain Modeling

The researchers at Boston University (BU)s Neuromorphics Laboratory, part of the National Science Foundation (NSF)-sponsored Center of Excellence for Learning in Education, Science, and Technology (CELEST), are working in collaboration with the engineers and scientists at Hewlett-Packard (HP) to implement neural models of intelligent processes for the next generation of dense, low-power, computer hardware that will use memristive technology to bring data closer to the processor where computation occurs. The HP and BU teams are jointly designing an optimal infrastructure, simulation, and software platform to build an artificial brain. The resulting Cog Ex Machina (Cog) software platform has been successfully used to implement a large-scale, multicomponent brain system that is able to simulate some key rat behavioral results in a virtual environment and has been applied to control robotic platforms as they learn to interact with their environment.

Review of stability properties of neural plasticity rules for implementation on memristive neuromorphic hardware

In the foreseeable future, synergistic advances in high-density memristive memory, scalable and massively parallel hardware, and neural network research will enable modelers to design large-scale, adaptive neural systems to support complex behaviors in virtual and robotic agents. A large variety of learning rules have been proposed in the literature to explain how neural activity shapes synaptic connections to support adaptive behavior. A generalized parametrizable form for many of these rules is proposed in a satellite paper in this volume [1]. Implementation of these rules in hardware raises a concern about the stability of memories created by these rules when the learning proceeds continuously and affects the performance in a network controlling freely-behaving agents. This paper can serve as a reference document as it summarizes in a concise way using a uniform notation the stability properties of the rules that are covered by the general form in [1].

Persuading Computers to Act More Like Brains

Convergent advances in neural modeling, neuroinformatics, neuromorphic engineering, materials science, and computer science will soon enable the development and manufacture of novel computer architectures, including those based on memristive technologies that seek to emulate biological brain structures. A new computational platform, Cog Ex Machina, is a flexible modeling tool that enables a variety of biological-scale neuromorphic algorithms to be implemented on heterogeneous processors, including both conventional and neuromorphic hardware. Cog Ex Machina is specifically designed to leverage the upcoming introduction of dense memristive memories close to computing cores. The MoNETA (Modular Neural Exploring Traveling Agent) model is comprised of such algorithms to generate complex behaviors based on functionalities that include perception, motivation, decision-making, and navigation. MoNETA is being developed with Cog Ex Machina to exploit new hardware devices and their capabilities as well as to demonstrate intelligent, autonomous behaviors in both virtual animats and robots. These innovations in hardware, software, and brain modeling will not only advance our understanding of how to build adaptive, simulated, or robotic agents, but will also create innovative technological applications with major impacts on general-purpose and high-performance computing.

MoNETA: massive parallel application of biological models navigating through virtual Morris water maze and beyond

The primary goal of a Modular Neural Exploring Traveling Agent (MoNETA) project is to create an autonomous agent capable of object recognition and localization, navigation, and planning in virtual and real environments. Major components of the system perform sensory object recognition, motivation and rewards processing, goal selection, allocentric representation of the world, spatial planning, and motor execution. MoNETA is based on the real time, massively parallel, Cog Ex Machina environment co-developed by Hewlett-Packard Laboratories and the Neuromorphics Lab at Boston University. The agent is tested in virtual environments replicating neurophysiological and psychological experiments with real rats. The currently used environment replicates the Morris water maze [1]. The motivational system represents the internal state of the agent that can be adjusted by sensory inputs. In the Morris water maze, only one drive can be satisfied (a desire to get out of the water) that persists as long as the animat is swimming and sharply decreases as soon as it is fully positioned on the platform. Another drive – curiosity – is constantly active and is never satisfied. It forces the agent to explore unfamiliar parts of the environment. Familiarity with environmental locations provides inhibition to the curiosity drive in a selective manner, so that recently explored locations are less appealing than either unexplored locations or locations that were explored long time ago. The main output of the motivational system is a goal selection map. It is based on competition between goals set by the curiosity system and goals learned by the animat. The goal selection map uses a winner-take-all selection of the most prominent input signal as a winning goal. Because curiosity-driven goals receive weaker inputs than well-learned reward locations, they can only win if there are no prominent inputs corresponding to the learned goals with an active motivational drive. The spatial planning system is built around a previously developed neural algorithm for goal-directed navigation [2]. The original model provided the desired destination and left it up to the virtual environment to move the animat in this location. In MoNETA the model was extended by a chain of neural populations that convert the allocentric desired destination into an allocentric desired direction and further into a rotational velocity motor command. A second extension of the model deals with the mapping of the environment. The original algorithm included goal and obstacle information into path planning, but this information was provided in the form of allocentric maps where the locations of both the goals and obstacles were received directly from the environment. MoNETA uses these maps, but also creates them from egocentric sensory information through a process of active exploration. Although the current version only uses somatosensory information, visual input will be added in later stages. The system converts egocentric representations to allocentric ones and then learns the mapping of obstacles and goals in the environment. It uses a learning rule that is local to dendrites and does not require any postsynaptic activity. The complete implementation of MoNETA consists of 75,301 neurons and 1,362,705 synapses.

A co-robotic assistant capable of object selection and search via a brain machine interface

Co-robotic assistants, or Cobots can improve the quality of life for individuals with locked-in syndrome (LIS) by allowing augmented control over their surroundings. Implemented in collaboration between the Boston University Neuromorphics Lab and Neural Prosthesis Lab, this work provides a proof of concept of an autonomous robot coupled with a non-invasive brain machine interface (BMI). The system uses Steady State Visually Evoked Potential (SSVEP) for target selection and a massively parallel neural network that models functionality of the primate “where” and “what” visual pathways. The simulated visual processes perform object recognition on images streamed from the Cobot equipped with a pan-and-tilt camera. In this paper, we describe a subcomponent of the system designed to allow the neural network to learn the identity of objects, the user to select a target object and the Cobot to perform autonomous visual investigation.