Pritish Narayanan

Experimental Demonstration and Tolerancing of a Large-Scale Neural Network (165 000 Synapses) Using Phase-Change Memory as the Synaptic Weight Element

Using 2 phase-change memory (PCM) devices per synapse, a 3-layer perceptron network with 164,885 synapses is trained on a subset (5000 examples) of the MNIST database of handwritten digits using a backpropagation variant suitable for NVM+selector crossbar arrays, obtaining a training (generalization) accuracy of 82.2% (82.9%). Using a neural network (NN) simulator matched to the experimental demonstrator, extensive tolerancing is performed with respect to NVM variability, yield, and the stochasticity, linearity and asymmetry of NVM-conductance response.

Access devices for 3D crosspoint memorya)

The emergence of new nonvolatile memory (NVM) technologies—such as phase change memory, resistive, and spin-torque-transfer magnetic RAM—has been motivated by exciting applications such as storage class memory, embedded nonvolatile memory, enhanced solid-state disks, and neuromorphic computing. Many of these applications call for such NVM devices to be packed densely in vast “crosspoint” arrays offering many gigabytes if not terabytes of solid-state storage. In such arrays, access to any small subset of the array for accurate reading or low-power writing requires a strong nonlinearity in the IV characteristics, so that the currents passing through the selecteddevices greatly exceed the residual leakage through the nonselecteddevices. This nonlinearity can either be included explicitly, by adding a discrete access device at each crosspoint, or implicitly with an NVM device which also exhibits a highly nonlinear IV characteristic. This article reviews progress made toward implementing such access device functionality, focusing on the need to stack such crosspoint arrays vertically above the surface of a silicon wafer for increased effective areal density. The authors start with a brief overview of circuit-level considerations for crosspoint memory arrays, and discuss the role of the access device in minimizing leakage through the many nonselected cells, while delivering the right voltages and currents to the selected cell. The authors then summarize the criteria that an access device must fulfill in order to enable crosspoint memory. The authors review current research on various discrete access device options, ranging from conventional silicon-based semiconductor devices, to oxide semiconductors, threshold switch devices, oxide tunnel barriers, and devices based on mixed-ionic-electronic-conduction. Finally, the authors discuss various approaches for self-selected nonvolatile memories based on Resistive RAM.

Fault-Tolerant Nanoscale Processors on Semiconductor Nanowire Grids

Nanoscale processor designs pose new challenges not encountered in the world of conventional CMOS designs and manufacturing. Nanoscale devices based on crossed semiconductor nanowires (NWs) have promising characteristics in addition to providing great density advantage over conventional CMOS devices. This density advantage could, however, be easily lost when assembled into nanoscale systems and especially after techniques dealing with high defect rates and manufacturing related layout/doping constraints are incorporated. Most conventional defect/fault-tolerance techniques are not suitable in nanoscale designs because they are designed for very small defect rates and assume arbitrary layouts for required circuits. Reconfigurable approaches face fundamental challenges including a complex interface between the micro and nano components required for programming. In this paper, we present our work on adding fault-tolerance to all components of a processor implemented on a 2-D semiconductor NW fabric called nanoscale application specific integrated circuits (NASICs). We combine and explore structural redundancy, built-in nanoscale error correcting circuitry, and system-level redundancy techniques and adapt the techniques to the NASIC fabric. Faulty signals caused by defects and other error sources are masked on-the-fly at various levels of granularity. Faults can be masked at up to 15% rates, while maintaining a 7 density advantage compared to an equivalent CMOS processor at projected 18-nm technology. Detailed analysis of yield, density, and area tradeoffs is provided for different error sources and fault distributions.

Experimental demonstration and tolerancing of a large-scale neural network (165,000 synapses), using phase-change memory as the synaptic weight element

Using two phase-change memory devices per synapse, a three-layer perceptron network with 164 885 synapses is trained on a subset (5000 examples) of the MNIST database of handwritten digits using a backpropagation variant suitable for nonvolatile memory (NVM) + selector crossbar arrays, obtaining a training (generalization) accuracy of 82.2% (82.9%). Using a neural network simulator matched to the experimental demonstrator, extensive tolerancing is performed with respect to NVM variability, yield, and the stochasticity, linearity, and asymmetry of the NVM-conductance response. We show that a bidirectional NVM with a symmetric, linear conductance response of high dynamic range is capable of delivering the same high classification accuracies on this problem as a conventional, software-based implementation of this same network.

Neuromorphic Computing Using Non-Volatile Memory

Abstract Dense crossbar arrays of non-volatile memory (NVM) devices represent one possible path for implementing massively-parallel and highly energy-efficient neuromorphic computing systems. We first review recent advances in the application of NVM devices to three computing paradigms: spiking neural networks (SNNs), deep neural networks (DNNs), and ‘Memcomputing’. In SNNs, NVM synaptic connections are updated by a local learning rule such as spike-timing-dependent-plasticity, a computational approach directly inspired by biology. For DNNs, NVM arrays can represent matrices of synaptic weights, implementing the matrix–vector multiplication needed for algorithms such as backpropagation in an analog yet massively-parallel fashion. This approach could provide significant improvements in power and speed compared to GPU-based DNN training, for applications of commercial significance. We then survey recent research in which different types of NVM devices – including phase change memory, conductive-bridging RAM, filamentary and non-filamentary RRAM, and other NVMs – have been proposed, either as a synapse or as a neuron, for use within a neuromorphic computing application. The relevant virtues and limitations of these devices are assessed, in terms of properties such as conductance dynamic range, (non)linearity and (a)symmetry of conductance response, retention, endurance, required switching power, and device variability. Graphical Abstract

CMOS Control Enabled Single-Type FET NASIC

A new hybrid CMOS-nanoscale circuit style has been developed that uses only one type of Field Effect Transistor (FET) in the logic portions of a design. This is enabled by CMOS providing control signals that coordinate the operation of the logic implemented in the nanoscale. In this paper, the new circuit style is explored, examples from a microprocessor design are shown, performance, manufacturing and density implications discussed. The system is based on the existing CMOS-nano hybrid fabric architecture NASIC, but the new circuit style reduces the requirements on devices and manufacturing from previous NASIC designs, significantly improves performance without any deterioration in circuit density.

Heterogeneous Two-Level Logic and Its Density and Fault Tolerance Implications in Nanoscale Fabrics

Most proposed nanoscale computing architectures are based on a certain type of two-level logic family, e.g., AND-OR, NOR-NOR, NAND-NAND, etc. In this paper, a new fabric architecture that combines different logic families in the same nanofabric is proposed for higher density and better defect tolerance. To achieve this, we apply very minor modifications on the way of controlling nanogrids, while the basic manufacturing requirements remain the same. The fabric that is based on the new heterogeneous two-level logic yields higher density for the applications mapped to it. We find that it also improves the efficiency of fault tolerance techniques as it significantly simplifies the designs. In addition, we found that it enables voting at nanoscale that can improve fault tolerance further. A nanoscale processor is implemented for evaluation purposes. We found that compared with an implementation on a Nanoscale Application-Specific IC (NASIC) fabric with one type of two-level logic, the density of this processor improves by up to 52% by using the heterogeneous logic. Furthermore, the yield is improved by 15% at 2% defective transistors and by 147% at 5% defect rates. Detailed analysis on density and yield is provided. The approach is applicable in grid-based fabrics in general, e.g., it can be used in both NASIC and hybrid semiconductor/nanowire/molecular (CMOL) designs.

Large-scale neural networks implemented with non-volatile memory as the synaptic weight element: Comparative performance analysis (accuracy, speed, and power)

We review our work towards achieving competitive performance (classification accuracies) for on-chip machine learning (ML) of large-scale artificial neural networks (ANN) using Non-Volatile Memory (NVM)-based synapses, despite the inherent random and deterministic imperfections of such devices. We then show that such systems could potentially offer faster (up to 25×) and lower-power (from 120-2850×) ML training than GPU-based hardware.

NASICs: A nanoscale fabric for nanoscale microprocessors

The rapid progress of manufacturing nanoscale devices is pushing researchers to explore appropriate nanoscale computing architectures for high density beyond the physical limitations of conventional lithography. However, manufacturing and layout constraints, as well as high defect/fault rates expected in nanoscale fabrics, could make most device density lost when integrated into computing systems. Therefore, a nanoscale architecture that can deal with those constraints and tolerate defects/faults at expected rates, while still retaining the density advantage, is highly desirable. In this paper, we describe a novel nanoscale architecture based on semiconductor nanowires: NASICs (nanoscale application specific ICs). NASIC is a tile-based fabric built on 2-D nanowire grids and NW FETs. WISP-0 (wire streaming processor) is a processor design built on NASIC fabric where NASIC design principles and optimizations are applied. Built-in fault tolerance techniques are applied on NASICs designs to tolerate defects/faults on-the-fly. Evaluations show that compared with the equivalent CMOS design with 18 nm process (the most advanced technology expected in 2018), WISP-0 with combined built-in redundancy could be still 2~3X denser. Its yield would be 98% if the defect rate of transistors is 5%, and 77% for 10% defective transistors.

Nanoscale Application Specific Integrated Circuits

This fabric update summarizes recent advances for the Nanoscale Application Specific Integrated Circuits (NASICs) nanoscale computing fabric. We provide a brief overview of NASICs, and discuss recent work at all fabric levels. We present advances in device design and optimization including omega gated and junctionless nanowire field effect transistors, methodologies for validation of functionality and parameter variation evaluation, new circuit-level sequencing schemes and performance optimization techniques. We also discuss techniques for defect and parameter variation resilience, ongoing fabrication directions including prototyping and scalable assembly efforts, and directions for the future.