Christoph von der Malsburg

On the Construction of Artificial Brains

This book presents a first generation of artificial brains, using vision as sample application. An object recognition system is built, using neurons and synapses as exclusive building elements. The system contains a feature pyramid with 8 orientations and 5 resolution levels for 1000 objects and networks for binding of features into objects. This vision system can recognize objects robustly in the presence of changes in illumination, deformation, distance and pose (as long as object components remain visible). The neuro-synaptic network owes its functional power to the introduction of rapidly modifiable dynamic synapses. These give a network greater pattern recognition capabilities than are achievable with fixed connections. The spatio-temporal correlation structure of patterns is captured by a single synaptic differential equation in a universal way. The correlation can appear as synchronous neural firing, which signals the presence of a feature in a robust way, or binds features into objects. Although in this book we can present only a first generation artificial brain and believe many more generations will have to follow to reach the full power of the human brain, we nevertheless see a new era of computation on the horizon. There were times when computers, with their precision, reliability and blinding speed, were considered to be as superior to the wet matter of our brain as a jet plane is to a sparrow. These times seem to be over, given the fact that digital systems inspired by formal logic and controlled algorithmically - todays computers - are hitting a complexity crisis. A paradigm change is in the air: from the externally organised to the self-organised computer, of which the results described in this book may give an inkling.

Architecture and Chip Design of the Feature Recognizer

The system concept of our vision cube is based on the 3D integration technology the realisation of which is described in chapter 16. The basic concept behind the application of this technology is illustrated in Figure 14.1.

Architecture of First Generation Vision Cube

We now want to determine the number of different modules which are required to build an artificial vision system which achieves the resolution of the retina, possesses 8 orientations per resolution plane and 5 resolution planes and recognises distant-invariantly up to 1000 objects each featuring 10 zones. The neural architecture of this vision system is described in chapters 6-9.

Architecture and Chip Design of the Feature Detector

This chapter describes the system architecture and chip design of the feature detector. The design aims at a widely flexible configuration of the connection topology to realise arbitrary feature detectors, at a faithful reproduction of the simulated individual or group behaviour of neurons and synapses, and at low power dissipation as well as the integration of image sensor pixel cells in order to directly supply the feature detectors with input signals.

Preliminary Considerations on the Microelectronic Implementation

The simulation and calculation of the activity diagrams in Figures 8.2 a-n required about 10 days and nights. A statistical testing or significant augmentation of the architecture by means of simulation on a PC is therefore not a passable route in the long run. An essential advancement is offered by these two paths:

3D Stacking Technology

The preceding chapters presented fundamentals of pulsed neural networks for image processing. The outcome was a modular architecture containing dedicated layers for each subtask. Finally, it was shown how dedicated VLSI circuits supporting these subtasks can be implemented. The integration of particular components (CMOS imager, feature cascade, object recogniser) into a system poses further questions, however. Particularly, the problem of interconnection of neurons and neural nets was only discussed in a rudimentary way. This chapter is devoted to associated questions and presents the first results and solutions.

Elementary Circuits for Neurons, Synapses, and Photosensors

The membrane potential a i of the IAF model (section 11.3) is realised as a voltage produced by the currents accumulating on the capacitor formed by the membrane of a neuron. A threshold switch with hysteresis determines the state variable with two values x i ∈ {0,1} of neuron i by comparing the accumulator potential with a reference voltage. At the time of firing, x i takes on the value 1 during the time span t d , which is achieved by discharging the capacitor.

Theory of Nets with Constant or Dynamic Synapses

The experimental observation of the presence and duration of partitions and pulse patterns, respectively, is one alternative to achieve the characterisation of a stationary frequency of distribution. For larger networks, however, the simulation time exceeds our human patience, hence a theoretic coverage is necessary. As the constant synapses form a special case, the development of a generic theory of nets with dynamic synapses becomes necessary. To this end we will introduce the idea of the signal energy and show that the signal equations as well as the entropy are derivable from it. This way the state trajectories of a net can be obtained by theory, for example as the dependency of the entropy on the input pulses. The theory developed here holds for arbitrary input signals (> 0), but pulses of amplitude x and duration td only. The reason for the latter restriction is the much more complicated calculations that would be required, and for the purposes of this book, it would not add anything regarding pulse signals in the treatment of image recognition tasks.

Nets for Feature Recognition

Let us consider an image with 1024 ×768 pixels and let it be covered by a grid of 69 ×85 Gabor wavelet detectors, such that the pulse rate of N7a and N7b, resp., is obtained at each node of the grid. Let each detector possess 128 sensor pairs. Moreover, 6 complex, ie. 12 real Gabor wavelet detectors are applied. At each grid node, the vector of the pulse response of the 12 output neurons of the respective detectors is recorded. A further grid with 18 ×22 nodes and 12 detector levels just covers a face, which is chosen for reference (Figure 7.1 mid), and is moved over the fine-grain larger net.

Information Processing in Nets with Constant Synapses

In this book, we attempt to construct an artificial vision system similar to the human brain which can perform complete image recognition tasks, ie. those which comprise feature detection, binding of features to objects, and object recognition. Similarity we see implemented by the usage of neurons and synapses as simply as possible, plus the restriction to only these: rejecting any adjunct of algorithms additional to the ones defining neurons and synapses.