Björn Levin

SWIM — A Simulation Environment for Realistic Neural Network Modeling

When simulating networks of several hundred interconnected neurons using realistic neuron models, tools are needed for handling the specification of all parameters involved. We have designed a specification language for this purpose. A simulation environment based on this language has been implemented, including tools for running the actual simulations on either a Unix workstation, a CRAY supercomputer, or a Connection Machine (CM, a massively parallel supercomputer). On a CM with 8192 processors the program is typically capable of handling some tens of thousands of neurons and millions of synapses. The SWIM simulation environment has been used to simulate experimentally verified networks (CPG in lamprey spinal cord) as well as for explorative experiments relating to attractor models of cortical associative memory.

BIOSIM—A PROGRAM FOR BIOLOGICALLY REALISTIC NEURAL NETWORK SIMULATIONS ON THE CONNECTION MACHINE

This paper describes a program for large scale biologically realistic neural network simulations on the Connection Machine, CM2. The neuron model used in the simulations is a compartmentalized abstraction of the neuron. The model includes Na+, K+, Ca2+, and calcium dependent K+ channels. Synaptic interaction includes conventional receptors and voltage gated NMDA receptors. On a CM2 with 8,192 processors the program is typically capable of handling some tens of thousands of compartments and more than ten times that number of synapses.

Simulation Support and ATM Performance Prediction

The estimation of quality of service through the simulation of traffic passing an ATM network requires considerable computational resources due to the rare event nature of the phenomena in question. By training a neural network to mimic the output of the simulator, i.e. to capture the functional relationship between different configurations and loads and their corresponding quality of service, estimates can be produced in under a millisecond as opposed to requiring around half a minute using only the simulation. This speed-up in turn enables interactive applications with smooth and instantaneous feedback and the treatment of considerably bigger ATM networks than before. The system can also be run “backwards”, i.e. given a desired quality of service the system can determine the acceptable loads and configurations.

Emulating process simulators with learning systems

We explore the possibility of replacing a process simulator with a learning system. This is motivated in the presented test case setting by a need to speed up a simulator that is to be used in conjunction with an optimisation algorithm to find near optimal process parameters. Here we will discuss the potential problems and difficulties in this application, how to solve them and present the results from a paper mill test case.

Document Retrieval and Protein Sequence Matching using a Neural Network

During the development of algorithms for the generation of higher order complex units in neural networks we have come to be interested in free text document retrieval and protein sequence matching. Document retrieval, as it is percieved here, consists of returning a list of the documents in the library that are the most relevant according to a description of a subject, sorted according to descending probability of relevance. Obviously, the key is having a reasonable measure of similarity between the individual documents and a given concept, a measure here provided by the neural network. [1] contains an overview of similar work.

BIOLOGICALLY REALISTIC AND ARTIFICIAL NEURAL NETWORK SIMULATORS ON THE CONNECTION MACHINE

We describe two neural network (NN) simulators implemented on the Connection Machine (CM). The first program is aimed at biologically realistic simulations and the second at recurrent artificial NNs. Both programs are currently used as simulation engines in research within the SANS group as well as in other groups. The program for biologically realistic NN simulations on the CM is called BIOSIM. The aim is to simulate NNs in which the neurons are modeled with a high degree of biological realism. The cell model used is a compartmentalized abstraction of the neuron. It includes sodium, potassium, calcium, and calcium dependent potassium channels. Synaptic interaction includes conventional chemical synapses as well as voltage gated NMDA synapses. On a CM with 8K processors the program is typically capable of handling some tens of thousands of compartments and more than ten times as many synapses. The artificial NN simulator implements the SANS model, a recurrent NN model closely related to the Hopfield model. The aim has been to effectively support large network simulations, in the order of 8–16K units, on an 8K CM. To make the simulator optimal for different applications, it supports both fully and sparsely connected networks. The implementation for sparsely connected NNs uses a compacted weight matrix. Both implementations are optimized for sparse activity.

TOOLS FOR BIOLOGICALLY REALISTIC SIMULATIONS OF NEURAL NETWORKS

When simulating thousands of neurons, using realistic neuron models and with the neurons not necessarily identical, tools for handling the specification of all parameters involved are needed. We have designed a specification language for this purpose. A simulator based on this language has been implemented including tools for running the actual simulations on either a regular Unix workstation or on the Connection Machine (a massively parallel supercomputer). This system has been used to simulate experimentally verified networks (CPG in Lamprey spinal cord) as well as for explorative experiments relating to pattern processing in artificial neural networks.

Implementation of the SANS algorithm on the CM2

This report describes a number of implementations on the massively parallel SIMD machine called the Connection Machine of the artificial neural network algorithm developed within the SANS project. The algorithm uses a recurrent network with the connection strengths defined according to Bayesian probability estimates. The purpose of these implementations has been to study how the algorithm can be adapted to run on parallel hardware and what can be gained from using such hardware.