Tomasz Talaska

Realization of the Conscience Mechanism in CMOS Implementation of Winner-Takes-All Self-Organizing Neural Networks

This paper presents a complementary metal-oxide-semiconductor (CMOS) implementation of a conscience mechanism used to improve the effectiveness of learning in the winner-takes-all (WTA) artificial neural networks (ANNs) realized at the transistor level. This mechanism makes it possible to eliminate the effect of the so-called ¿dead neurons,¿ which do not take part in the learning phase competition. These neurons usually have a detrimental effect on the network performance, increasing the quantization error. The proposed mechanism comes as part of the analog implementation of the WTA neural networks (NNs) designed for applications to ultralow power portable diagnostic devices for online analysis of ECG biomedical signals. The study presents Matlab simulations of the networks model, discusses postlayout circuit level simulations and includes results of measurement completed for the physical realization of the circuit.

Current-Mode Analog Adaptive Mechanism for Ultra-Low-Power Neural Networks

Neural networks (NNs) implemented at the transistor level are powerful adaptive systems. They can perform hundreds of operations in parallel but at the expense of a large number of building blocks. In the case of analog realization, an extremely low chip area and low power dissipation can be achieved. To accomplish this, the building blocks should be simple. This brief presents a new current-mode low-complexity flexible adaptive mechanism (ADM) with a strongly reduced leakage in analog memory. Input signals ranging from 0.5 to 20 μA are held for 10-50 ms, with the leakage rate from 0.2%/ms to 0.04%/ms, respectively, depending on temperature. A small storage capacitor of 200 fF enables a short write time ( <; 100 ns). A single ADM cell occupies 1400 μm2 when realized in the Taiwan Semiconductor Manufacturing Company Ltd. CMOS 0.18-μm technology. The potential application of this NN is envisioned in a mobile platform based on a wireless sensor network to be used for online analysis of electrocardiography signals.

Analog Programmable Distance Calculation Circuit for Winner Takes All Neural Network Realized in the CMOS Technology

This paper presents a programmable analog current-mode circuit used to calculate the distance between two vectors of currents, following two distance measures. The Euclidean (L2) distance is commonly used. However, in many situations, it can be replaced with the Manhattan (L1) one, which is computationally less intensive, whose realization comes with less power dissipation and lower hardware complexity. The presented circuit can be easily reprogrammed to operate with one of these distances. The circuit is one of the components of an analog winner takes all neural network (NN) implemented in the complementary metal-oxide-semiconductor 0.18-μm technology. The learning process of the realized NN has been successfully verified by the laboratory tests of the fabricated chip. The proposed distance calculation circuit (DCC) features a simple structure, which makes it suitable for networks with a relatively large number of neurons realized in hardware and operating in parallel. For example, the network with three inputs occupies a relatively small area of 3900 μm2. When operating in the L2 mode, the circuit dissipates 85 μW of power from the 1.5 V voltage supply, at maximum data rate of 10 MHz. In the L1 mode, an average dissipated power is reduced to 55 μW from 1.2 V voltage supply, while data rate is 12 MHz in this case. The given data rates are provided for the worst case scenario, where input currents differ by 1%-2% only. In this case, the settling time of the comparators used in the DCC is quite long. However, that kind of situation is very rare in the overall learning process.

Analog-Counter-Based Conscience Mechanism in Kohonen's Neural Network Implemented in CMOS 0.18 μm Technology

In this study, we present a hardware implementation of the conscience mechanism in Kohonen self-organizing maps. The proposed realization of the conscience mechanism is important to the functioning of the neural network as it eliminates so-called dead (inactive) neurons. As a result the network learning, the level quantization error can be reduced. The conscience mechanism and the Winner Take All (WTA) block have been implemented in 0.18 μm CMOS process. The implementation of the conscience mechanism itself occupies 1200 μm 2 and its maximum power consumption is 9.5 μW. The WTA block together with the conscience mechanism occupies 0.024 mm2 and dissipates 55 μ W

Implementation Of The Conscience Mechanism For Kohonen's Neural Network In Cmos 0.18 /spl mu/m Technology

Hardware implementation of the conscience mechanism in Kohonens neural networks is presented in this work. The conscience mechanism is an important component of the neural network as it eliminates so called dead neurons leading to larger network efficiency and smaller quantization error. The conscience mechanism itself and winner take all (WTA) block have been implemented in 0.18 mum CMOS process. The design integrated circuit (IC) is a continuation of the earlier effort to produce elements required to implement competitive learning mechanisms in neural networks. The conscience mechanism circuit occupies 270 mum and dissipates maximum power of 22 muW at 2V power supply. The WTA block occupies 0.02 mm and dissipates 50 muW

A novel recursive algorithm used to model hardware programmable neighborhood mechanism of self-organizing neural networks

In this paper we propose a novel recursive algorithm that models the neighborhood mechanism, which is commonly used in self-organizing neural networks (NNs). The neighborhood can be viewed as a map of connections between particular neurons in the NN. Its relevance relies on a strong reduction of the number of neurons that remain inactive during the learning process. Thus it substantially reduces the quantization error that occurs during the learning process. This mechanism is usually difficult to implement, especially if the NN is realized as a specialized chip or in Field Programmable Gate Arrays (FPGAs). The main challenge in this case is how to realize a proper, collision-free, multi-path data flow of activations signals, especially if the neighborhood range is large. The proposed recursive algorithm allows for a very efficient realization of such mechanism. One of major advantages is that different learning algorithms and topologies of the NN are easily realized in one simple function. An additional feature is that the proposed solution accurately models hardware implementations of the neighborhood mechanism.

Experimental results of CMOS-implemented conscience mechanism applied for WTA networks

This paper presents experimental results demonstrating a positive effect of using conscience mechanism in on chip learning Kohonen networks. Application of this mechanism allows to eliminate so called ldquodead neuronsrdquo, which do not take part in the competition during the learning phase, thus increasing the quantization error of the network. Both Matlab simulations and measurement results are shown.

Analog sorting circuit for the application in self-organizing neural networks based on neural gas learning algorithm

The paper presents a new, mixed analog-digital, circuit for analog sorting signals. In comparison to other circuits of this type the proposed solution offers large versatility. The main objective is its application in Neural Gas (NG) learning algorithm used to train unsupervised neural networks (NNs). However, the circuit can also be used in nonlinear processing of analog signals. It is capable of performing simultaneously several typical nonlinear operations that include Min, Max and Median filtering. The circuit offers high accuracy, however the difference between signals that can be distinguished depends on the steepness of a reference ramp signal. For example, the circuit it able to distinguish signals that differ by 10 nA if the assumed time is larger than 1 μs. Since a typical number of neurons in the NN exceeds 100-200, the circuit has been designed to sort so many input signals. The sorting operation provides us values of particular output signals, as well as the information which inputs signals deliver particular output signals. This second feature is used in case of the application of the circuit in NN. The system was implemented in the TSMC 180nm CMOS technology and verified in the HSpice environment. For 8 inputs varying in between 1 to 10 μA the circuit dissipates an average power of 250 μW.

Low power nonlinear Min/Max filters implemented in the CMOS technology

A novel, binary-tree, asynchronous, nonlinear Min/Max filter is presented in the paper. In the proposed circuit an input signal (current in this case) is first sampled in the circular delay line, controlled by a multiphase clock (8 phases in this case). In the next stage particular samples are converted to 1-bit signals with delays proportional to the values of these samples. In the following step the delay times are compared in digital binary-tree structure. The circuit has been simulated in the TSMC CMOS 0.18 μm process. It offers a precision of 99.5% at data rate of 2.5 MSamples/s and energy consumption of 0.3-1 pJ per input.

A serial distance calculation circuit for the application in artificial neural networks and pattern recognition

The paper presents a novel circuit for the calculation of Manhattan distance between two vectors of signals, suitable for various machine learning algorithms realized at the transistor level. In Self-Organizing Artificial Neural Networks, for example, one of the basic operations is the calculation of a distance between input learning patterns and vectors of neuron weights. In pattern recognition two patterns are being compared in a similar way. The proposed digital circuit is a serial solution, whose calculation scheme relies on the accumulation of absolute values of differences between corresponding components of both vectors. This allows to stop the calculations, if necessary, at any component of the compared vectors. The circuit is has been realized in the CMOS 130 nm technology and verified by means of postlayout simulations.