Armin Alaghi

Survey of Stochastic Computing

Stochastic computing (SC) was proposed in the 1960s as a low-cost alternative to conventional binary computing. It is unique in that it represents and processes information in the form of digitized probabilities. SC employs very low-complexity arithmetic units which was a primary design concern in the past. Despite this advantage and also its inherent error tolerance, SC was seen as impractical because of very long computation times and relatively low accuracy. However, current technology trends tend to increase uncertainty in circuit behavior and imply a need to better understand, and perhaps exploit, probability in computation. This article surveys SC from a modern perspective where the small size, error resilience, and probabilistic features of SC may compete successfully with conventional methodologies in certain applications. First, we survey the literature and review the key concepts of stochastic number representation and circuit structure. We then describe the design of SC-based circuits and evaluate their advantages and disadvantages. Finally, we give examples of the potential applications of SC and discuss some practical problems that are yet to be solved.

Stochastic circuits for real-time image-processing applications

Real-time image-processing applications impose severe design constraints in terms of area and power. Examples of interest include retinal implants for vision restoration and on-the-fly feature extraction. This work addresses the design of image-processing circuits using stochastic computing techniques. We show how stochastic circuits can be integrated at the pixel level with image sensors, thus supporting efficient real-time (pre)processing of images. We present the design of several representative circuits, which demonstrate that stochastic designs can be significantly smaller, faster, more power-efficient, and more noise-tolerant than conventional ones. Furthermore, the stochastic designs naturally produce images with progressive quality improvement.

Exploiting correlation in stochastic circuit design

Stochastic computing (SC) is a re-emerging computing paradigm which enables ultra-low power and massive parallelism in important applications like real-time image processing. It is characterized by its use of pseudo-random numbers implemented by 0-1 sequences called stochastic numbers (SNs) and interpreted as probabilities. Accuracy is usually assumed to depend on the interacting SNs being highly independent or uncorrelated in a loosely specified way. This paper introduces a new and rigorous SC correlation (SCC) measure for SNs, and shows that, contrary to intuition, correlation can be exploited as a resource in SC design. We propose a general framework for analyzing and designing combinational circuits with correlated inputs, and demonstrate that such circuits can be significantly more efficient and more accurate than traditional SC circuits. We also provide a method of analyzing stochastic sequential circuits, which tend to have inherently correlated state variables and have proven very hard to analyze.

A spectral transform approach to stochastic circuits

Stochastic computing (SC) processes data in the form of long pseudo-random bit-streams denoting probabilities. Its key advantages are simple computational elements and high soft-error tolerance. Recent technology developments have revealed important new SC applications such as image processing and LDPC decoding. Despite its long history, SC still lacks a comprehensive design methodology; existing methods tend to be ad hoc and limited to a few arithmetic functions. We demonstrate a fundamental relation between stochastic circuits and spectral transforms. Based on this, we propose a transform approach to the analysis and synthesis of SC circuits. We illustrate the approach for a variety of basic combinational SC design problems, and show that the area cost associated with stochastic number generation can be significantly reduced.

Fast and accurate computation using stochastic circuits

Stochastic computing (SC) is a low-cost design technique that has great promise in applications such as image processing. SC enables arithmetic operations to be performed on stochastic bit-streams using ultra-small and low-power circuitry. However, accurate computations tend to require long run-times due to the random fluctuations inherent in stochastic numbers (SNs). We present novel techniques for SN generation that lead to better accuracy/run-time trade-offs. First, we analyze a property called progressive precision (PP) which allows computational accuracy to grow systematically with run-time. Second, borrowing from Monte Carlo methods, we show that SC performance can be greatly improved by replacing the usual pseudo-random number sources by low-discrepancy (LD) sequences that are predictably progressive. Finally, we evaluate the use of LD stochastic numbers in SC, and show they can produce significantly faster and more accurate results than existing stochastic designs.

STRAUSS: Spectral Transform Use in Stochastic Circuit Synthesis

Stochastic computing (SC) is an approximate computing technique that processes data in the form of long pseudorandom bit-streams which can be interpreted as probabilities. Its key advantages are low-complexity hardware and high-error tolerance. SC has recently been finding application in several important areas, including image processing, artificial neural networks, and low-density parity check decoding. Despite a long history, SC still lacks a comprehensive design methodology, so existing designs tend to be either ad hoc or based on specialized design methods. In this paper, we demonstrate a fundamental relation between stochastic circuits and spectral transforms. Based on this, we propose a general, transform-based approach to the analysis and synthesis of SC circuits. We implemented this approach in a program spectral transform use in stochastic circuit synthesis (STRAUSS), which also includes a method of optimizing stochastic number-generation circuitry. Finally, we show that the area cost of the circuits generated by STRAUSS is significantly smaller than that of previous work.

Behavior of stochastic circuits under severe error conditions

Abstract Stochastic computing is an old and unconventional computing technique that is finding promising new applications in image processing and the handling of complex error-correcting codes. Stochastic circuits offer an alternative to conventional digital circuits because of their extremely small size and inherent noise tolerance. They are also well-suited to meeting the requirements of emerging nanoscale technologies where non-deterministic behavior due to manufacturing defects and soft errors cannot be ignored. Error analysis of stochastic circuits, however, has received little attention and remains a largely open problem, especially when multiple errors affecting both the data sources and the stochastic circuits can occur in the course of a computation. This paper attempts to analyze stochastic circuits under various error conditions, and to compare their behavior to that of conventional circuits under similar error conditions. We use probabilistic transfer matrices for this analysis, complemented by circuit simulation. Our results indicate that stochastic circuits provide significantly better error tolerance under severe error conditions.

Tomographic Testing and Validation of Probabilistic Circuits

Some emerging technologies for building computers depend on components and signals whose behavior, under normal or fault conditions, is probabilistic. Examples include stochastic and quantum computing circuits, and conventional nano electronic circuits subject to design, manufacturing or environmental errors. Problems common to these technologies are testing and validation, which require determining whether observed non-deterministic behavior is within acceptable limits. Traditional solution methods rely on the determinism of operations performed by the circuit under test, and are not applicable to probabilistic circuits, where signals are often described by probability distributions. We introduce a generic methodology for testing probabilistic circuits by approximating signal probability distributions using tomograms, which aggregate the outcomes of multiple, repeated test measurements. While the name comes from quantum computation, tomography is applicable to both quantum and non-quantum probabilistic circuits, as we demonstrate. Our methodology makes use of fault or error models that allow handling of large and complex circuits. We report the first experimental results on the tomographic testing of quantum and stochastic circuits.

On the Functions Realized by Stochastic Computing Circuits

Stochastic computing (SC) employs conventional logic circuits to implement analog-style arithmetic functions acting on digital bit-streams. It exploits the advantages of analog computation -powerful basic operations, high operating speed, and error tolerance- in important applications such as sensory image processing and neuromorphic systems. At the same time, SC exhibits the analog drawbacks of low precision and complex underlying behavior. Although studied since the 1960s, many of SCs fundamental properties are not well known or well understood. This paper presents, in a uniform manner and notation, what is known about the relations between the logical and stochastic behavior of stochastic circuits. It also considers how correlation among input bit-streams and the presence of memory elements influences stochastic behavior. Some related research challenges posed by SC are also discussed.

Energy-efficient hybrid stochastic-binary neural networks for near-sensor computing

Recent advances in neural networks (NNs) exhibit unprecedented success at transforming large, unstructured data streams into compact higher-level semantic information for tasks such as handwriting recognition, image classification, and speech recognition. Ideally, systems would employ near-sensor computation to execute these tasks at sensor endpoints to maximize data reduction and minimize data movement. However, near-sensor computing presents its own set of challenges such as operating power constraints, energy budgets, and communication bandwidth capacities. In this paper, we propose a stochastic-binary hybrid design which splits the computation between the stochastic and binary domains for near-sensor NN applications. In addition, our design uses a new stochastic adder and multiplier that are significantly more accurate than existing adders and multipliers. We also show that retraining the binary portion of the NN computation can compensate for precision losses introduced by shorter stochastic bit-streams, allowing faster run times at minimal accuracy losses. Our evaluation shows that our hybrid stochastic-binary design can achieve 9.8x energy efficiency savings, and application-level accuracies within 0.05% compared to conventional all-binary designs.