Bipin Rajendran

Reducing read latency of phase change memory via early read and Turbo Read

Phase Change Memory (PCM) is an emerging memory technology that can enable scalable high-density main memory systems. Unfortunately, PCM has higher read latency than DRAM, resulting in lower system performance. This paper investigates architectural techniques to improve the read latency of PCM. We observe that there is a wide distribution in cell resistance in both the SET state and the RESET state, and that the read latency of PCM is designed conservatively to handle the worst case cell. If PCM sensing can be tuned to exploit the variability in cell resistance, then we can get reduced read latency. We propose two schemes to enable better-than-worst-case read latency for PCM systems. Our first proposal, Early Read, reads the data earlier than the specified time period. Our key observation that Early Read causes only unidirectional errors (SET being read as RESET) allows us to efficiently detect data errors using Berger codes. In the uncommon case that Early Read causes data error(s), we simply retry the read operation with original latency. Our evaluations show that Early Read can reduce the read latency by 25% while incurring a storage overhead of only 10 bits per 64 byte line. Our second proposal, Turbo Read, reduces the sensing time for read operations by pumping higher current, at the expense of accidentally switching the PCM cell with small probability during the read operation. We analyze Error Correction Codes (ECC) and Probabilistic Row Scrubbing (PRS) for maintaining data integrity under Turbo Read. We show that a combination of Early Read and Turbo Read can reduce the PCM read latency by 30%, improve the system performance by 21%, and reduce the Energy Delay Product (EDP) by 28%, while requiring minimal changes to the memory system.

Neuromorphic Computing Based on Emerging Memory Technologies

In this paper, we review some of the novel emerging memory technologies and how they can enable energy-efficient implementation of large neuromorphic computing systems. We will highlight some of the key aspects of biological computation that are being mimicked in these novel nanoscale devices, and discuss various strategies employed to implement them efficiently. Though large scale learning systems have not been implemented using these devices yet, we will discuss the ideal specifications and metrics to be satisfied by these devices based on theoretical estimations and simulations. We also outline the emerging trends and challenges in the path towards successful implementations of large learning systems that could be ubiquitously deployed for a wide variety of cognitive computing tasks.

NormAD - Normalized Approximate Descent based supervised learning rule for spiking neurons

NormAD is a novel supervised learning algorithm to train spiking neurons to produce a desired spike train in response to a given input. It is shown that NormAD provides faster convergence than state-of-the-art supervised learning algorithms for spiking neurons, often the gain in the rate of convergence being more than a factor of 10. The algorithm leverages the fact that a leaky integrate-and-fire neuron can be described as a non-linear spatio-temporal filter, allowing us to treat supervised learning as a mathematically tractable optimization problem with a cost function in terms of the membrane potential rather than the spike arrival time. A variant of stochastic gradient descent along with normalization has been used to derive the synaptic weight update rule. NormAD uses leaky integration of the input to determine the synaptic weight change. Since leaky integration is fundamental to all integrate-and-fire models of spiking neurons, we claim universal applicability of the learning rule to other models such as adaptive exponential integrate-and-fire model of neurons by demonstrating equally good performance in training with our algorithm.

On Pairing of Bipolar RRAM Memory With NPN Selector Based on Set/Reset Array Power Considerations

In this paper, we present a methodology of choosing an NPN selector (1S) for a given memory element (1M) to form a cross point (consisting of the memory element in series with the selector-1S1M) based on the overall array power efficiency requirements. This methodology is based on extensive TCAD simulations that show excellent match with our experimentally demonstrated n+/p/n+ epitaxial Si punch-through diode as selector (NPN selector) for symmetric bipolar resistive RAM. Using a TCAD validated circuit model of the NPN selector, we derive an equivalent circuit model for the cross point. For an exemplary selector design, our model suggests that the power P_xp dissipated in the cross point during set operation obeys the relationship P_xp ∝ V_set^0.5 I_set^1.25, even though the power dissipated in the memory element P_memory is V_setI_set. This shows that lowering the set current I_set of the memory element leads to a larger reduction in array power than lowering the set voltage V_set.

An ultra-compact and low power neuron based on SOI platform

Analog and digital circuit designs have been proposed to mimic the biological neuron in CMOS compatible learning circuits for “brain” like computing. However, the adaptation of such conventional circuit based strategies requires many devices, large areas and hence power consumption. We propose a neuronal device based on the well-investigated impact-ionization based NPN selector on an SOI platform. The neuronal device has a small footprint (225·F2) and low active power (11.5nW/spike) and provides ~10,000x speed-up over biological timescales. In comparison to analog neuron, ultra-high density (>60x improvement) and low power operation (>5x improvement) are demonstrated.

Pulsed laser annealing: A scalable and practical technology for monolithic 3D IC

Classical dimensional scaling faces challenges from growing on-chip interconnect time delays, and escalating lithography costs and layout limitations. In this paper, we present practical integration schemes for developing cost-efficient 3D ICs in a monolithic fashion, which employ fully depleted transistor channels and laser annealing to achieve sharper junction definition.

Neuromorphic computing with multi-memristive synapses

Neuromorphic computing has emerged as a promising avenue towards building the next generation of intelligent computing systems. It has been proposed that memristive devices, which exhibit history-dependent conductivity modulation, could efficiently represent the synaptic weights in artificial neural networks. However, precise modulation of the device conductance over a wide dynamic range, necessary to maintain high network accuracy, is proving to be challenging. To address this, we present a multi-memristive synaptic architecture with an efficient global counter-based arbitration scheme. We focus on phase change memory devices, develop a comprehensive model and demonstrate via simulations the effectiveness of the concept for both spiking and non-spiking neural networks. Moreover, we present experimental results involving over a million phase change memory devices for unsupervised learning of temporal correlations using a spiking neural network. The work presents a significant step towards the realization of large-scale and energy-efficient neuromorphic computing systems.Memristive technology is a promising avenue towards realizing efficient non-von Neumann neuromorphic hardware. Boybat et al. proposes a multi-memristive synaptic architecture with a counter-based global arbitration scheme to address challenges associated with the non-ideal memristive device behavior.

C. elegans chemotaxis inspired neuromorphic circuit for contour tracking and obstacle avoidance

We demonstrate a spiking neural network for navigation motivated by the chemotaxis circuit of Caenorhabditis elegans. Our network uses information regarding temporal gradients in intensity of local variables such as chemical concentration, temperature, radiation, etc., to make navigational decisions for contour tracking and obstacle avoidance. The gradient information is determined by mimicking the underlying mechanisms of the ASE neurons of C. elegans. Simulations show that our software-worm is able to identify the set-point with 92% efficiency, 68.5% higher than an optimal memoryless Lévy foraging strategy and 33% higher than an equivalent non-spiking neural network configuration. The software-worm is able to track the set-point with an average deviation of 1% from the set-point, and this performance degrades merely by 1.8% in the presence of intense salt and pepper noise in the local tracking variable. We also develop a VLSI implementation for the main gradient detector neurons, which could be integrated with standard comparator circuitry to develop robust circuits for navigation and contour tracking. We demonstrate noise-resilience of our network to environmental, architectural and circuit noise.

Biomimetic 4F 2 synapse with intrinsic timescale for pulse based STDP by I-NPN selection device

We have proposed a 4F2 synapse using I-NPN based selector with high on-off ratio as well as intrinsic timescales which are utilized to implement time correlation in a synapse for high density neuromorphic circuits. STDP based learning is demonstrated using simple pulses while is true biomimetic.

Spiking neural networks for handwritten digit recognition—Supervised learning and network optimization

We demonstrate supervised learning in Spiking Neural Networks (SNNs) for the problem of handwritten digit recognition using the spike triggered Normalized Approximate Descent (NormAD) algorithm. Our network that employs neurons operating at sparse biological spike rates below 300Hz achieves a classification accuracy of 98.17% on the MNIST test database with four times fewer parameters compared to the state-of-the-art. We present several insights from extensive numerical experiments regarding optimization of learning parameters and network configuration to improve its accuracy. We also describe a number of strategies to optimize the SNN for implementation in memory and energy constrained hardware, including approximations in computing the neuronal dynamics and reduced precision in storing the synaptic weights. Experiments reveal that even with 3-bit synaptic weights, the classification accuracy of the designed SNN does not degrade beyond 1% as compared to the floating-point baseline. Further, the proposed SNN, which is trained based on the precise spike timing information outperforms an equivalent non-spiking artificial neural network (ANN) trained using back propagation, especially at low bit precision. Thus, our study shows the potential for realizing efficient neuromorphic systems that use spike based information encoding and learning for real-world applications.