Jialing Li

Making neural encoding robust and energy efficient: an advanced analog temporal encoder for brain-inspired computing systems

Neural encoder is one of the key components in neuromorphic computing systems, whereby sensory information is transformed into spike coded trains. The design of temporal encoder has attracted a widespread attention in the field of neuromorphic computing in the past few years. The information in the temporal encoding scheme with inter-spike intervals can arise from correlations between spike times, which could not be incorporated in the traditional rate encoding scheme. In this paper, we propose a robust and energy efficient analog implementation of the spiking temporal encoder. We pattern the neural activities across multiple timescales and encode the sensory information using time dependent temporal scales. The concept of iteration structure is introduced to construct a neural encoder that greatly increases the information process ability of the proposed temporal encoder. Integrated with iteration technique and operational-amplifier-free design, the error rate of the output temporal codes is reduced to an extremely low level. A lower sampling rate accompanied by additional verification spikes is introduced in the schemes, which significantly reduces the power consumption of the encoding system. To the best of our knowledge, our proposed neuron circuit is the first analog hardware implementation of the neural encoder that could present the sensory data using inter-spike interval temporal encoding scheme. The simulation and measurement results show the proposed temporal encoder exhibits not only energy efficiency but also high accuracy.

Novel Spike based Reservoir Node Design with High Performance Spike Delay Loop

With the emerging cutting-edge semiconductor nanotechnologies, reservoir computing has shifted the focus from software implementation towards hardware and optical implementations over the last few decades. Nowadays, in the field of reservoir computing, pure analog implementation with the employment of CMOS nanotechnology has attracted a worldwide attention. In this paper, we introduce a spike-based analog reservoir node with compact delay performing at extremely high accuracy. The detailed work on this novel spike processor working in near chaotic condition is presented and analyzed. Furthermore, we announce a novel class of spike-based delay loop which enables the pure spike implementation of reservoir computing. The simulation results demonstrated that our delay loop possesses exceptionally high accuracy of 99.24% and gain-hold property without adopting any kind of amplifiers. Our proposed reservoir node has shown the potential of transforming spike signals to a high-dimensional state space.

Energy Efficient Spiking Temporal Encoder Design for Neuromorphic Computing Systems

Neuromorphic computing hardware has undergone a rapid development and progress in the past few years. One of the key components in neuromorphic computing systems is the neural encoder which transforms sensory information into spike trains. In this paper, both rate encoding and temporal encoding schemes are discussed. Two novel temporal encoding schemes, parallel and iteration, are presented. The power consumption of the encoder has been significantly reduced by combing the iteration encoding and low sampling rate in advanced complementary metal-oxide semiconductor (CMOS) nano-technology. Both the simulation and measurement results show the accuracy and efficiency of the proposed encoding circuits. The proposed iteration encoder has immediate applicability as a general purpose input encoder for a reservoir computing system.

Enabling Sustainable Cyber Physical Security Systems through Neuromorphic Computing

As the novel paradigm in the field of machine learning, reservoir computing possesses exceptional performance, e.g., energy efficiency, in tasks in which the traditional von Neumann computing systems cannot incorporate. This makes reservoir computing an ideal candidate to enable the sustainable development of cyber-physical systems (CPS). In the realm of CPS, the tight interaction among physical objects places security threats under the spotlight of attention. For such systems, especially the power grid network, false data injection could potentially lead to catastrophic consequences such as blackouts in large geographical areas. In this paper, we will introduce a reservoir computing architecture, the delayed feedback system, and apply the reservoir computing architecture for anomaly detection. To be specific, detailed design of the three imperative components in the delayed feedback system will be discussed and the corresponding energy efficiency performance will be analyzed. The application of the reservoir computing architecture to anomaly detection in a smart grid network will be introduced.

Analog hardware implementation of spike-based delayed feedback reservoir computing system

The rate of enhancement is starting to saturate and slow down which indicates the end of Moores prediction due to the fundamental performance limits of the chips. The need of breaking through the barrier has directed researchers into several directions, for instance, novel computing architecture. Reservoir computing, a novel concept in the field of machine learning, has emerged over the past few years. Combined the memory and spatio-temporal processing of recurrent neural networks, reservoir computing possesses the capability of processing temporal information. In this paper, we present an analog hardware implementation of delayed feedback reservoir computing system. We build a new class of computationally efficient spike timing-dependent encoders and delay-based reservoirs within reservoir networks. This approach allows us to avoid using power-consuming analog-to-digital converters (ADCs) and operational amplifiers (Op-AMPs), resulting in significant savings in power requirements and design area.

Interspike-Interval-Based Analog Spike-Time-Dependent Encoder for Neuromorphic Processors

Von Neumann bottleneck, which refers to the limited throughput between the CPU and memory, has already become a major factor hindering the technical advances of computing systems. In recent years, neuromorphic systems have started to gain the increasing attentions as compact and energy-efficient computing platforms. As one of the most crucial components in the neuromorphic computing systems, neural encoder transforms the stimulus (input signals) into spike trains. In this paper, we adapt the temporal encoding scheme of interspike intervals (ISIs) and present an analog temporal neural encoder with its verification and recovery schemes. The proposed neural encoder allows efficient mapping of signal amplitude information into a spike-time sequence that represents the input data and offers perfect recovery for band-limited stimuli. With the novel iterative structure, the number of spikes increases exponentially with the number of neurons. From the measurements obtained from the fabricated neural encoder chip, our temporal encoder with ISI encoding is proved to be robust and error tolerant.

Enabling An New Era of Brain-inspired Computing: Energy-efficient Spiking Neural Network with Ring Topology

The reservoir computing, an emerging computing paradigm, has proven its benefit to multifarious applications. In this work, we successfully designed and fabricated an analog delayed feedback reservoir (DFR) chip. Measurement results demonstrate its rich dynamic behaviors and high energy efficiency. System performance, as well as the robustness, are evaluated. The application of video frame recognition is investigated using a hybrid neural network, which employs the multilayer perceptron (MLP) training model as the readout layer of our designed DFR system, and yields 98% classification accuracy. Compared to results of using the MLP training only, our hybrid training model exhibits much higher recognition rate and accuracy.

Analog Spike-Timing-Dependent Resistive Crossbar Design for Brain Inspired Computing

With the rapid advancement in the complementary metal-oxide-semiconductor (CMOS) technology, the performance of neuromorphic computing systems that employed the emerging devices has entered a new era of high accuracy and energy efficient operations. As one of the most fundamental components in computing systems, encoding scheme characterizes the relationship between the stimulus and the individual or ensemble neuronal responses. Although rate encoding is easier to implement and with high error tolerance, temporal encoding offers high data density and energy efficiency. In this paper, an inter-spike interval (ISI)-based resistive crossbar neuromorphic design, built with the standard CMOS technology, is proposed. Our proposed decoder exhibits not only computation accuracy but also robustness. Another worth-mentioning achievement is the data compressor in our work, which is based on the spike temporal encoding scheme. In this paper, design and performance analysis for proposed ISI-based resistive crossbar are elaborated; as well as the application of our proposed design. 24 patterns are adopted to encode the sensory information which are successfully represented by seven inter-spike intervals leading to significantly high compression rate. To evaluate the performance, a test bench of video frames consisting one person rotating her head from 0° to 75° with an increment of 15° has been employed. The results showed that the ISI code has better performance in both recognition rate and converging speed.

Energy efficient analog spiking temporal encoder with verification and recovery scheme for neuromorphic computing systems

Making a computing system that mimic biological neural behavior in mammalian brain has attracted worldwide attention and endeavor. Neuromorphic computing systems, employing very-large-scale integration circuits to implement onto hardware, incorporates learning. Neural encoder, as one of the crucial component in neuromorphic computing systems, encodes the input information into spikes. By taking the temporal response structure into consideration, temporal encoding with interspike intervals exhibits the capability of containing more information and encoding information using the time correlation between spikes. In this paper, a neural encoder with iterative structure, adapting interspike interval encoding scheme, is proposed. Considered the tradeoff between power consumption and die area, we employed an analog implementation of the spiking neuron. By doing so, power-consuming analog-to-digital converters (ADCs) and operational amplifiers (Op-AMPs) are not needed, resulting in a tremendous saving on power consumption and die area. Due to the iterative processing, the growth of the spike amounts with respect to the neuron number is exponential, which significantly reduces power consumption.

Three dimensional memristor-based neuromorphic computing system and its application to cloud robotics

Abstract Neuromorphic computing based on three-dimensional inetgraed circuits (3D-NCs) offers a novel hardware implementation of neuromorphic computing, and provides high device density, massively parallel signal processing capability, low power consumption, and direct analog signal processing capability. In this paper, by replacing conventional CPUs based on Von Neumann architecture with 3D-NCs, a novel neuromorphic computing based cloud robotics (NC-robotics) system is proposed, which is constructed by 1) cloud server center using 3D-NCs as computing units, 2) neuromorphic robotics based on neural network control technology. Besides the benefits of normal Cloud Robotics platform, this NC- Robotics system has more advantages on massive parallel-computing, analog signals processing, and lower power consumption. In order to implement this NC––Robotics system, a novel 3D-NCs architecture combining vertical RRAM structure is investigated and its concise equivalent circuit model is created, evaluated, and analyzed through SPICE simulations.