Sudhir Rao

Estimating the information potential with the fast gauss transform

In this paper, we propose a fast and accurate approximation to the information potential of Information Theoretic Learning (ITL) using the Fast Gauss Transform (FGT). We exemplify here the case of the Minimum Error Entropy criterion to train adaptive systems. The FGT reduces the complexity of the estimation from O(N2) to O(pkN) wherep is the order of the Hermite approximation and k the number of clusters utilized in FGT. Further, we show that FGT converges to the actual entropy value rapidly with increasing order p unlike the Stochastic Information Gradient, the present O(pN) approximation to reduce the computational complexity in ITL. We test the performance of these FGT methods on System Identification with encouraging results.

A Normalized Minimum Error Entropy Stochastic Algorithm

We propose in this paper the normalized minimum error entropy (NMEE). Following the same rational that lead to the normalized LMS, the weight update adjustment for minimum error entropy (MEE) is constrained by the principle of minimum disturbance. Unexpectedly, we obtained an algorithm that not only is insensitive to the power of the input, but is also faster than the MEE for the same misadjustment, and also that is less sensitive to the kernel size. We explain these results analytically, and through system identification simulations

Spectral Clustering of Synchronous Spike Trains

In this paper a clustering algorithm that learns the groups of synchronized spike trains directly from data is proposed. Clustering of spike trains based on the presence of synchronous neural activity is of high relevance in neurophys-iological studies. In this context such activity is thought to be associated with functional structures in the brain. In addition, clustering has the potential to analyze large volumes of data. The algorithm couples a distance between two spike trains recently proposed in the literature with spectral clustering. Finally, the algorithm is illustrated in sets of computer generated spike trains and analyzed for the dependence on its parameters and accuracy with respect to features of interest.

A Novel Weighted LBG Algorithm for Neural Spike Compression

In this paper, we present a weighted Linde-Buzo-Gray algorithm (WLBG) as a powerful and efficient technique for compressing neural spike data. We compare this technique with the recently proposed self-organizing map with dynamic learning (SOM-DL) and the traditional SOM. A significant achievement of WLBG over SOM-DL is a 15 dB increase in the SNR of the spike data apart from having a compression ratio of 150 : 1. Being simple and extremely fast, this algorithm allows real-time implementation on DSP chips opening new opportunities in BMI applications.

Information Theoretic Vector Quantization with Fixed Point Updates

In this paper, we revisit information theoretic vector quantization (ITVQ) algorithm introduced in (T. Lehn-Schioler et al., 2005) and make it practical. We derive a fixed point update rule to minimize the Cauchy-Schwartz(CS) pdf divergence between the set of codewords and the actual data. In doing so, we overcome two severe deficiencies of the previous gradient based method namely, the number of parameters to be optimized and slow convergence rate, thus making this algorithm more efficient and useful as a compression algorithm.

Spike Sorting Using non Parametric Clustering VIA Cauchy Schwartz PDF Divergence

We propose a new method of clustering neural spike waveforms for spike sorting. After detecting the spikes using a threshold detector, we use principal component analysis (PCA) to get the first few PCA components of the data. Clustering on these PCA components is achieved by maximizing the Cauchy Schwartz PDF divergence measure which uses the Parzen window method to non parametrically estimate the PDF of the clusters. Comparison with other clustering techniques in spike sorting like k-means and Gaussian mixture elucidates the superiority of our method in terms of classification results and computational complexity

Self-Organizing ITL Principles for Unsupervised Learning

Chapter 1 presented a synopsis of information theory to understand its foundations and how it affected the field of communication systems. In a nutshell, mutual information characterizes the fundamental compromise of maximum rate for error-free information transmission (the channel capacity theorem) as well as the minimal information that needs to be sent for a given distortion (the rate distortion theorem). In essence given the statistical knowledge of the data and these theorems the optimal communication system emerges, or self-organizes from the data.

Wireless transmission of neuronal recordings using a portable real-time discrimination/compression algorithm

A design challenge of portable wireless neural recording systems is the tradeoff between bandwidth and power consumption. This paper investigates the compression of neuronal recordings in real-time using a novel discriminating Linde-Buzo-Gray algorithm (DLBG) that preserves spike shapes while filtering background noise. The technique is implemented in a low power digital signal processor (DSP) which is capable of wirelessly transmitting raw neuronal recordings. Depending on the signal to noise ratio of the recording, the compression ratio can be tailored to the data to maximally preserve power and bandwidth. The approach was tested in real and synthetic data and achieved compression ratios between 184:1 and 10:1.

Clustering with ITL Principles

Learning and adaptation deal with the quantification and exploitation of the input source “structure” as pointed out perhaps for the first time by Watanabe [330]. Although structure is a vague and difficult concept to quantify, structure fills the space with identifiable patterns that may be distinguishable macroscopically by the shape of the probability density function. Therefore, entropy and the concept of dissimilarity naturally form the foundations for unsupervised learning because they are descriptors of PDFs.

A Novel Switching Scheme Between Adaptive Information Algorithms

Switching approaches can improve the performance of adaptive schemes, however a data driven criterion to accomplish the task is unclear. In this paper, we propose a new optimization criterion for switching which is estimated directly from data. We apply the method to the recently introduced MEE and MEE-SAS algorithms. Using this novel switching scheme, we develop a single algorithm which effectively combines the strengths of MEE and MEE-SAS without sacrificing the simplicity and stability properties of MEE. We explain these results analytically, and through simulations.