Jane Bromley

An analog neural network processor with programmable topology

The architecture, implementation, and applications of a special-purpose neural network processor are described. The chip performs over 2000 multiplications and additions simultaneously. Its data path is particularly suitable for the convolutional topologies that are typical in classification networks, but can also be configured for fully connected or feedback topologies. Resources can be multiplexed to permit implementation of networks with several hundreds of thousands of connections on a single chip. Computations are performed with 6 b accuracy for the weights and 3 b for the neuron states. Analog processing is used internally for reduced power dissipation and higher density, but all input/output is digital to simplify system integration. The practicality of the chip is demonstrated with an implementation of a neural network for optical character recognition. This network contains over 130000 connections and was evaluated in 1 ms. >

Application of the ANNA neural network chip to high-speed character recognition

A neural network with 136000 connections for recognition of handwritten digits has been implemented using a mixed analog/digital neural network chip. The neural network chip is capable of processing 1000 characters/s. The recognition system has essentially the same rate (5%) as a simulation of the network with 32-b floating-point precision.

Reading handwritten digits: a ZIP code recognition system

A neural network algorithm-based system that reads handwritten ZIP codes appearing on real US mail is described. The system uses a recognition-based segmenter, that is a hybrid of connected-components analysis (CCA), vertical cuts, and a neural network recognizer. Connected components that are single digits are handled by CCA. CCs that are combined or dissected digits are handled by the vertical-cut segmenter. The four main stages of processing are preprocessing, in which noise is removed and the digits are deslanted, CCA segmentation and recognition, vertical-cut-point estimation and segmentation, and directly lookup. The system was trained and tested on approximately 10000 images, five- and nine-digit ZIP code fields taken from real mail.<<ETX>>

Improving rejection performance on handwritten digits by training with “rubbish”

Very good performance for the classification of handwritten digits has been achieved using feedforward backpropagation networks (LeCun et al. 1990; Martin and Pittman 1990). These initial networks were trained and tested on clean, well-segmented images. In the real world, however, images are rarely perfect, which causes problems. For example, at one time one of our best performing digit classifiers interpreted a horizontal bar as a 2; in this example the most useful response would be to reject the image as unclassifiable. The aim of the work reported here was to train a network to reject the type of unclassifiable images (“rubbish”) typically produced by an automatic segmenter for strings of digits (e.g., zip codes), while maintaining its performance level at classifying digits, by adding images of rubbish to the training set.

Hardware requirements for neural network pattern classifiers: a case study and implementation

A special-purpose chip, optimized for computational needs of neural networks and performing over 2000 multiplications and additions simultaneously, is described. Its data path is particularly suitable for the convolutional architectures typical in pattern classification networks but can also be configured for fully connected or feedback topologies. A development system permits rapid prototyping of new applications and analysis of the impact of the specialized hardware on system performance. The power and flexibility of the processor are demonstrated with a neural network for handwritten character recognition containing over 133000 connections.<<ETX>>

An analog neural network processor and its application to high-speed character recognition

A high-speed programmable neural network chip and its application to character recognition are described. A network with over 130000 connections has been implemented on a single chip and operates at a rate of over 1000 classifications per second. The chip performs up to 2000 multiplications and additions simultaneously. Its datapath is suitable for the convolutional architectures that are typical in pattern classification networks, but can also be configured for fully connected or feedback topologies. Computations were performed with 6 bits accuracy for the weights and 3 bits for the states. The chip uses analog processing internally for higher density and reduced power dissipation, but all input/output is digital to simplify system integration.<<ETX>>

Systems, Networks, and Policy

Systems theory is fundamental to understanding the dynamics of the complex social systems of concern to policy makers. A system is defined as: (1) an assembly of components, connected together in an organised way; (2) the components are affected by being in the system and the behaviour of the systems is changed if they leave it; (3) the organised assembly of components does something; and (4) the assembly has been identified as being of particular interest. Feedback is central to system behaviour at all levels, and can be responsible for systems behaving in complex and unpredictable ways. Systems can be represented by networks and there is a growing literature that shows how the behaviour of individuals is highly dependent on their social networks. This includes copying or following the advice of others when making decisions. Network theory gives insights into social phenomena such as the spread of information and the way people form social groups which then constrain their behaviour. It is emerging as a powerful way of examining the dynamics of social systems. Most systems relevant to policy have many levels, from the individual to local and national and international organisations and institutions. In many social systems the micro, meso and macrolevel dynamics are coupled, meaning that they cannot be studied or modified in isolation. Systems and network science allow computer simulations to be used to investigate possible system behaviour. This science can be made available to policy makers through policy informatics which involves computer-based simulation, data, visualisation, and interactive interfaces. The future of science-based policy making is seen to be through Global Systems Science which combines complex systems science and policy informatics to inform policy makers and facilitate citizen engagement. In this context, systems theory and network science are fundamental for modelling far-from-equilibrium systems for policy purposes.

Penacée: A Neural Net System for Recognizing On-Line Handwriting

We report on progress in handwriting recognition and signature verification. Our system, which uses pen-trajectory information, is suitable for use in pen-based computers. It has a multimodular architecture whose central trainable module is a time-delay neural network. Results comparing our system and a commercial recognizer are presented. Our best recognizer makes three times less errors on hand-printed word recognition than the commercial one.

Neural network applications in character recognition and document analysis

Character Recognition has served as one of the principal proving grounds for neural-net methods and has emerged as one of the most successful applications of this technology. This chapter outlines optical character recognition document analysis systems developed at AT&T Bell Labs that combine the strengths of machine-learning algorithms with high-speed, fine-grained parallel hardware. From our point of view, the most significant aspect of this work has been the efficient integration of diverse methods into end-to-end systems. In this paper we use the task of locating and reading ZIP codes on US mail pieces as an illustration of the character recognition / document analysis process. We will also describe other applications of the technology, including interpretation of faxed forms and bit-mapped text to ASCII conversion.

A neural network approach to handprint character recognition

The authors outline OCR (optical character recognition) technology developed at AT&T Bell Laboratories, including a recognition network that learns feature extraction kernels and a custom VLSI chip that is designed for neural-net image processing. It is concluded that both high speed and high accuracy can be obtained using neural-net methods for character recognition. Networks can be designed that learn their own feature extraction kernels. Special-purpose neural-net chips combined with digital signal processors can quickly evaluate character-recognition neural nets. This high speed is particularly useful for recognition-based segmentation of character strings.<<ETX>>