David E. van den Bout

Maximum entropy and learning theory

We derive the learning theory recently reported by Tishby, Levin, and Solla (TLS) directly from the principle of maximum entropy instead of statistical mechanics. The theory generally applies to any problem of modeling data. We analyze an elementary example for which we find the predictions consistent with intuition and conventional statistical results and we numerically examine the more realistic problem of training a competitive net to learn a one-dimensional probability density from samples. The TLS theory is useful for predicting average training behavior.

Rapid Prototyping for Neural Networks

A massively parallel, all-digital, stochastic architecture — TInMANN — is described which performs competitive and Kohonen types of learning at rates as high as 145,000 training examples per second. TInMANN is composed of very simple neurons and is very amenable to VLSI implementation, but rapid advances in IC technology and neural network theory reduce the rewards of such an endeavor. As an alternative, we discuss the rapid-prototyping of a bit-serial version of TInMANN using commercially available logic cell arrays and RAMs.

Automatic circuit partitioning in the Anyboard rapid prototyping system

Abstract The Anyboard rapid prototyping system is described. The Anyboard circuit partitioner is discussed and the results of experiments are presented that characterize its ability to find good partitions. Under some conditions it was found that an algorithm that is generally regarded as poor in fact produces good results in less time than an algorithm that is generally regarded as more powerful.

Scalable VLSI implementations for neural networks

This paper discusses research on scalable VLSI implementations of feed-forward and recurrent neural networks. These two families of networks are useful in a wide variety of important applications—classification tasks for feed-forward nets and optimization problems for recurrent nets—but their differences affect the way they should be built. We find that analog computation with digitally programmable weights works best for feed-forward networks, while stochastic processing takes advantage of the integrative nature of recurrent networks. We have shown early prototypes of these networks which compute at rates of 1–2 billion connections per second. These general-purpose neural building blocks can be coupled with an overall data transmission framework that is electronically reconfigured in a local manner to produce arbitrarily large, fault-tolerant networks.