Robert A. Nawrocki

A Mini Review of Neuromorphic Architectures and Implementations

Neuromorphic architectures are hardware systems that aim to use the principles of neural function for their basis of operation. Their goal is to harness biologically inspired concepts such as weighted connections, activation thresholds, short-and long-term potentiation, and inhibition to solve problems in distributed computation. Compared with todays methods of emulating neural function in software on conventional von Neumann hardware, neuromorphic systems provide the promise of inherently low power and fault-tolerant operation directly implemented into hardware, for application in distributed and embedded computing tasks, where the vast scaling of todays architectures does not provide a long-term solution. This mini review is intended for a general engineering audience not currently familiar with this exciting research area. It provides descriptions of some of the recent advances, including supercomputer and single-device implementations, approaches based on spiking and nonspiking neurons, machine learning hardware accelerators, and those utilizing memristive devices. Hardware implementations utilizing both conventional electronic materials and organic electronic materials are reviewed.

A neuromorphic architecture from single transistor neurons with organic bistable devices for weights

Artificial Intelligence (AI) has made tremendous progress since it was first postulated in the 1950s. However, AI systems are primarily emulated on serial machine hardware that result in high power consumption, especially when compared to their biological counterparts. Recent interest in neuromorphic architectures aims to more directly emulate biological information processing to achieve substantially lower power consumption for appropriate information processing tasks. We propose a novel way of realizing a neuromorphic architecture, termed Synthetic Neural Network (SNN), that is modeled after conventional artificial neural networks and incorporates organic bistable devices as circuit elements that resemble the basic operation of a binary synapse. Via computer simulation we demonstrate how a single synthetic neuron, created with only a single transistor, a single-bistable-device-per-input, and two resistors, exhibits a behavior of an artificial neuron and approximates the sigmoidal activation function. We also show that, by increasing the number of bistable devices per input, a single neuron can be trained to behave like a Boolean logic AND or OR gate. To validate the efficacy of our design, we show two simulations where SNN is used as a pattern classifier of complicated, non-linear relationships based on real-world problems. In the first example, our SNN is shown to perform the trained task of directional propulsion due to water hammer effect with an average error of about 7.2%. The second task, a robotic wall following, resulted in SNN error of approximately 9.6%. Our simulations and analysis are based on the performance of organic electronic elements created in our laboratory.

Artificial neural network performance degradation under network damage: Stuck-at faults

Biological neural networks are spectacularly more energy efficient than currently available man-made, transistor-based information processing units. Additionally, biological systems do not suffer catastrophic failures when subjected to physical damage, but experience proportional performance degradation. Hardware neural networks promise great advantages in information processing tasks that are inherently parallel or are deployed in an environment where the processing unit might be susceptible to physical damage. This paper, intended for hardware neural network applications, presents analysis of performance degradation of various architectures of artificial neural networks when subjected to ‘stuck-at-0’ and ‘stuck-at-1’ faults. This study aims to determine if a fixed number of neurons should be kept in a single or multiple hidden layers. Faults are administered to input and hidden layer(s) and analysis of unoptimized and optimized, feedforward and recurrent networks, trained with uncorrelated and correlated data sets is conducted. A comparison of networks with single, dual, triple, and quadruple hidden layers is quantified. The main finding is that ‘stuck-at-0’ faults administered to input layer result in least performance degradation in networks with multiple hidden layers. However, for ‘stuck-at-0’ faults occurring to cells in hidden layer(s), the architecture that sustains the least damage is that of a single hidden layer. When ‘stuck-at-1’ errors are applied to either input or hidden layers, the network(s) that offer the most resilience are those with multiple hidden layers. The study suggests that hardware neural network architecture should be chosen based on the most likely type of damage that the system may be subjected to, namely damage to sensors or the neural network itself.

Structured Computational Polymers for a soft robot: Actuation and cognition

Structured Computational Polymers (SCP) is a concept of layered class of active material that can sense its environment and, due to its cognitive capabilities, react “intelligently” to those changes. In such a material, we envision semiconducting polymer based sensing, actuation, and information processing for on-board decision making to be combined into one active material. This paper describes incremental steps taken towards developing such a multifunctional active material, concentrating on distributed forms of actuation and cognition, with an intermediate goal of utilizing SCP as a “skin” of a soft robot - a robot, made of flexible materials, which is not bounded by its rigid structure and can adjust to its changing environment - with its sensing, cognition, and actuation embedded in the shape. We demonstrate, via experiment and rudimentary simulation, the feasibility of utilizing water hammer as a form of directed, distributed actuation. We also show that distributed form of cognition can be realized via a novel concept termed Synthetic Neural Network (SNN), which is a type of organic neuromorphic architecture modeled after Artificial Neural Network. SNN, based on a single-transistor-single-memristor-per-input for an individual neuron, can approximate the sigmoidal activation function with an accuracy of about 3%. A simulation of the SNN is shown to accurately predict the directionality of water hammer propulsion with an accuracy of 7.2 percent.

Morphing Bus: A New Paradigm in Peripheral Interconnect Bus

Modern digital peripheral interconnect buses are typically described as one of the two types, serial or parallel, based on whether or not the physical data channel(s) is (are) shared across the bits of a coherent data word. Conventional serial and parallel buses operate in a time-multiplexed mode, allowing access to only one peripheral device at a time. PCIexpress expanded this simple bifurcation by combining serial data channels with simultaneous access to multiple peripheral devices (or multiple regions of the same peripheral). Moving to that nontime-multiplexed mode opened a new dimension by which bus architectures could be classified. Under this taxonomy of serial versus parallel and time-multiplexed versus nontime multiplexed, our Morphing Bus presents a new paradigm that fills the parallel, nontime multiplexed interconnect niche. Furthermore, the Morphing Bus eliminates the notion of an intermediate data format and thereby obviates the need for bus interface circuitry. Instead of a common data format to which all the sensors and actuators are translated, the Morphing Bus transforms-or morphs-its signal lines to meet the needs of the connected sensors or actuators. For digital sensors and actuators, this morphing is achieved through a field-programmable gate array on the processor side of the bus. As programmable devices begin to incorporate analog signal paths, the Morphing Bus paves the way for integrating analog and digital signals into a single bus paradigm. The efficacy of the Morphing Bus is verified via implementation on a miniature robot and the bandwidth of the bus implementation is experimentally characterized. Its unique physical I/O stack in the form of a helical structure provides efficient cooling and power distribution for compact embedded systems in addition to the novel interconnect paradigm.

Towards an all-polymer robot for search and rescue

This paper discusses two components suitable for construction of an all-polymer robot, namely a Synthetic Neural Network and water hammer based actuation. A new data processing element, termed Synthetic Neural Network, or SNN, based on a concept of a polymer-based bistable memory device and a conventional transistor made from polymers, is proposed. A phenomenon known as the water hammer effect is described for the purposes of propulsion of the serpentine robot constructed from polymer tubing. Arresting the flow of water in the tube causes it to lurch forward. A relationship between the shape of the hose and the direction of propulsion is investigated with the goal of using the SNN to learn to control the forward progress of the robot based on polymer bend sensors.

Enhancement of Closed-Loop Gain of Organic Amplifiers Using Double-Gate Structures

We demonstrate a significant improvement in the gain of closed-loop organic amplifiers by systematically controlling the threshold voltage (VTH) with double-gate structures. The use of an ultrathin parylene dielectric results in an operating voltage as low as 4 V with a supply voltage (VDD) of 2 V. Double-gate structures were introduced for controlling VTH of the transistor and the switching voltage of the inverter. The maximum dc gain of the inverter was recorded as 29.3 dB at a VDD value of 2 V. In addition, the switching voltage of the circuits can be controlled from 0 V to VDD without a significant performance degradation. The effects of the VTH control on the closed-loop amplifier were systematically investigated and a significant gain improvement of up to 18.8 dB was demonstrated in the closed-loop gain by adjusting the switching voltage to half of VDD.

Monitoring Performance Degradation of Cerebellar Functions Using Computational Neuroscience Methods: Implications on Neurological Diseases

Neurodegeneration is a major cause of human disease. Within the cerebellum, neuronal degeneration and/or dysfunction has been associated with many diseases, including several forms of cerebellar ataxia, since normal cerebellar function is paramount for proper motor coordination, balance, and motor learning. The cerebellum represents a well-established neural circuit. Determining the effects of neuronal loss is of great importance for understanding the fundamental workings of the cerebellum and disease-associated dysfunctions. This paper presents computational modeling of cerebellar function in relation to neurodegeneration either affecting a specific cerebellar cell type, such as granule cells or Purkinje cells, or more generally affecting cerebellar cells and the implications on effects in relation to performance degradation throughout the progression of cell death. The results of the models show that the overall number of cells, as a percentage of the total cell number in the model, of a particular type and, primarily, their proximity to the circuit output, and not the neuronal convergence due to the relative number of cells of a particular type, is the main indicator of the gravity of the functional deficit caused by the degradation of that cell type. Specifically, the greater the percentage loss of neurons of a specific type and the closer proximity of those cells to the deep cerebellar neurons, the greater the deficit caused by the neuronal cell loss. These findings contribute to the understanding of the functional consequences of neurodegeneration and the functional importance of specific connectivity within a neuronal circuit.

Polymer and Nanoparticle-Composite Bistable Devices: Physics of Operation and Initial Applications

Polymer and nanoparticle-composite bistable devices, which fall under the category of Organic Bistable Devices (OBDs), provide non-volatile, two-state ON/OFF behavior for potential memristor or other applications. These materials consist of insulating, semiconducting, or conducting polymers such as poly(methylmethacrylate) (PMMA), poly(vinylcarbazole) (PVK), or poly(3,4-ethylenedioxythiophene):poly(4-styrenesulfonate) (PEDOT:PSS), respectively. Composites can be made by blending these with inorganic nanoparticles made from materials such as silver, gold, or zinc oxide. Such devices offer several potential advantages compared with their inorganic counterparts. These include reduced cost from solution-based methods of deposition, ease of direct-write printing over large areas, high throughput processing, and light-weight and flexible mechanical properties. Here we review the materials, designs, physics, and operation of these devices. We conclude the chapter with a discussion of possible applications, including recent advances in neuromorphic engineering specifically geared toward use in robotics.

Wireless electrical power to sub-millimeter robots

A sub-millimeter scale coil is investigated as an alternative means to power electronics for small-scale robots. The AC voltage is induced by time-varying magnetic field. FEM analysis of employing magnetic field concentrators to increase the field density is carried out, concluding with their ineffectiveness to offset the occupied space. The choice of conductive versus non-conductive photoresist is investigated. The coil fabrication process is based upon three-dimensional, two-photon-absorption photolithography. Additional steps include metal sputtering, microlaser patterning and wire-bonding. The steps detailing the entire design process are described. With the coil occupying a volume of 0.45 pico m3, the maximum AC voltage of approximately 84 nV, with power density of about 1.96 mW per meter cube were measured. The study concludes with proposing ways to increase the induced voltage to a useable voltage of 2 V.