Sayeh Sharify

Bit-pragmatic deep neural network computing

Deep Neural Networks expose a high degree of parallelism, making them amenable to highly data parallel architectures. However, data-parallel architectures often accept inefficiency in individual computations for the sake of overall efficiency. We show that on average, activation values of convolutional layers during inference in modern Deep Convolutional Neural Networks (CNNs) contain 92% zero bits. Processing these zero bits entails ineffectual computations that could be skipped. We propose Pragmatic (PRA), a massively data-parallel architecture that eliminates most of the ineffectual computations on-the-fly, improving performance and energy efficiency compared to state-of-the-art high-performance accelerators [5]. The idea behind PRA is deceptively simple: use serial-parallel shift-and-add multiplication while skipping the zero bits of the serial input. However, a straightforward implementation based on shift-and-add multiplication yields unacceptable area, power and memory access overheads compared to a conventional bit-parallel design. PRA incorporates a set of design decisions to yield a practical, area and energy efficient design. Measurements demonstrate that for convolutional layers, PRA is 4.31

Loom: exploiting weight and activation precisions to accelerate convolutional neural networks

\times

Value-Based Deep Learning Hardware Accelerators

faster than DaDianNao [5] (DaDN) using a 16-bit fixed-point representation. While PRA requires 1.68

Exploiting Typical Values to Accelerate Deep Learning

\times

Dynamic Stripes: Exploiting the Dynamic Precision Requirements of Activation Values in Neural Networks.

more area than DaDN, the performance gains yield a 1.70

Tartan: Accelerating Fully-Connected and Convolutional Layers in Deep Learning Networks by Exploiting Numerical Precision Variability

\times

Cnvlutin2: Ineffectual-Activation-and-Weight-Free Deep Neural Network Computing.

increase in energy efficiency in a 65nm technology. With 8-bit quantized activations, PRA is 2.25

Bit-Tactical: Exploiting Ineffectual Computations in Convolutional Neural Networks: Which, Why, and How.

\times

DPRed: Making Typical Activation Values Matter In Deep Learning Computing.

faster and 1.31

Laconic Deep Learning Computing.

\times