Charles F. Neugebauer

The CCD neural processor: a neural network integrated circuit with 65536 programmable analog synapses

The design, fabrication, and preliminary testing of an integrated circuit implementing neural network (NN) models with 256 on-chip, fully interconnected neurons and programmable analog synapses are reported. The integrated circuit was built using a charge-coupled-device-(CCD)-based architecture. A study of the current efforts to develop NN hardware reveals that the conventional electronic approach suffers from two major problems: (1) a tradeoff between the complexity of the synapse and the number of synapses per chip; and (b) the I/O (input/output) problem, namely, the slow communication between the chip and the surrounding environment. This approach circumvents the problems by using CCD arrays and/or a spatial light modulator as a short-term memory for the device. The preliminary results presented serve to validate the assumptions on which the CCD approach is based and to reassess the potential of this approach. The CCD architecture is based on two main assumptions: (a) the revolving charge packets in the CCD rings can complete several full cycles without substantial decay, (thus the required refresh of the matrix from an external memory will not significantly degrade the overall operation speed) and (b) the multiplication process, namely, the nondestructive sensing of the W/sub ij/ packets revolving in the CCD rings and their accumulation (provided the respective V/sub j/ is on) can be accomplished accurately and quickly. It is now clear that both these assumptions are valid. >

A CCD based neural network integrated circuit with 64K analog programmable synapses

A report is presented on the design, fabrication, and testing of a neural network integrated circuit with 65536 analog programmable synapses (256 fully interconnected neurons). The integrated circuit utilizes charge-coupled devices (CCDs) based on a generic architecture that the authors proposed (1987). Preliminary testing of the CCD neural processor indicates that the operating speed is 0.5×109 analog interconnect updates/s. Loading of the synaptic interaction matrix can be accomplished either electrically or optically within 0.5 ms or 1 ms, respectively

Analysis and verification of an analog VLSI incremental outer-product learning system

An architecture is described for the microelectronic implementation of arbitrary outer-product learning rules in analog floating-gate CMOS matrix-vector multiplier networks. The weights are stored permanently on floating gates and are updated under uniform UV illumination with a general incremental analog four-quadrant outer-product learning scheme, performed locally on-chip by a single transistor per matrix element on average. From the mechanism of floating gate relaxation under UV radiation, the authors derive the learning parameters and their dependence on the illumination level and circuit parameters. It is shown that the weight increments consists of two parts: one term contains the outer product of two externally applied learning vectors; the other part represents a uniform weight decay, with time constant originating from the floating gate relaxation. The authors address the implementation of supervised and unsupervised learning algorithms with emphasis on the delta rule. Experimental results from a simple implementation of the delta rule on an 8x7 linear network are included.

A parallel analog CCD/CMOS neural network IC

A mixed analog/digital neural network chip is presented that uses standard 2- mu m CCD/CMOS fabrication. The device incorporates a matrix of charge injection device elements which hold a matrix of charge encoding the synapse strengths. A vector-matrix multiplier with simple charge-domain multiply-accumulate units has been implemented. The device computes the product of a binary vector and an analog matrix of charge, providing an analog output vector. At a clock rate of 1 MHz, the fabricated device computes 64 million binary/analog multiply accumulates per second in a very small area. The matrix refresh time is small compared to the refresh period.<<ETX>>

Pattern matching and parallel processing with CCD technology

A fully parallel charge coupled device (CCD) memory chip of N address lines is presented. It detects, in just one clock cycle, a perfect matching between input pattern and any of the stored patterns. It detects in fewer than N cycles the best matching in case a perfect one does not exist. The charge packets, representing binary words, are generated by external pulses that are applied to the chip through data input lines and then are compared to the data applied to the address lines. The chip architecture is described. This chip is suitable for applications in pattern recognition, Kanerva memories, data decoders, and other systems that require peak detection or Hamming distance calculation. Typical results expected in these kinds of implementations based on similar devices are reported.<<ETX>>

An adaptive CMOS matrix-vector multiplier for large scale analog hardware neural network applications

The authors present an analog four-quadrant matrix-vector multiplier of low circuit complexity in floating gate CMOS technology, capable of on-chip weight adaptation following an arbitrary incremental outer-product local learning scheme, and with permanent storage of the weights after learning is performed. The complete adaptive circuit employs, on average, as few as two transistors per matrix element (C.F. Neugenbauer et al., 1990), allowing a very compact VLSI circuit layout (less than 30 mu m*30 mu m per synapse in standard 2 mu m CMOS technology) suitable for the use in fully interconnected neural network hardware of densities above 256 neurons per cm/sup 2/. With proper biasing techniques, an input linearity region for the multiplier ranging 800 mV at modest current levels are demonstrated. Four-quadrant outer-product weight adaptation, performed locally on-chip by floating gate voltage increments under ultraviolet illumination, has been achieved with floating gate adaptation up to 10 mV/s.<<ETX>>

Large Scale Optoelectronic Integration of Asynchronous Analog Neural Networks

Asimple circuit architecture in standard CMOS technology for the optoelectronic implementation of analog continuous-time neural networks (NN) is presented. The circuit enables the implementation of recurrent NN models with analog synapses and neurons, with continuous dynamics. The basic cell consists of a synapse coupled with a distributed neuron, where synaptic linear superposition and neuron nonlinear thresholding are combined together, using only five MOS transistors and one phototransistor per synapse. The synaptic interaction matrix is imaged continuously on the chip from a spatial light modulator, thus allowing fast reprogramming of the connections. The performance of the proposed system is illustrated by some measurements of synapse and neuron characteristics on a 16 neuron (256 synapse) prototype fabricated in MOSIS CMOS technology. The expected performance and limitations of a scaled up system are discussed.

Spatial Light Modulators As Parallel Memories For Optoelectronic Neural Networks

A generic architecture for realizing neural networks is presented in which the synaptic interaction matrix is loaded in parallel into an electronic integrated circuit from a SLM. Three types of the electronic processors are described using CCD, CID and CMOS technologies respectively. The pros and cons of currently existing SLMs for this architecture are pointed out.