Il Park

An Information Theoretic Approach of Designing Sparse Kernel Adaptive Filters

This paper discusses an information theoretic approach of designing sparse kernel adaptive filters. To determine useful data to be learned and remove redundant ones, a subjective information measure called surprise is introduced. Surprise captures the amount of information a datum contains which is transferable to a learning system. Based on this concept, we propose a systematic sparsification scheme, which can drastically reduce the time and space complexity without harming the performance of kernel adaptive filters. Nonlinear regression, short term chaotic time-series prediction, and long term time-series forecasting examples are presented.

Extended Kernel Recursive Least Squares Algorithm

This paper presents a kernelized version of the extended recursive least squares (EX-KRLS) algorithm which implements for the first time a general linear state model in reproducing kernel Hilbert spaces (RKHS), or equivalently a general nonlinear state model in the input space. The center piece of this development is a reformulation of the well known extended recursive least squares (EX-RLS) algorithm in RKHS which only requires inner product operations between input vectors, thus enabling the application of the kernel property (commonly known as the kernel trick). The first part of the paper presents a set of theorems that shows the generality of the approach. The EX-KRLS is preferable to 1) a standard kernel recursive least squares (KRLS) in applications that require tracking the state-vector of general linear state-space models in the kernel space, or 2) an EX-RLS when the application requires a nonlinear observation and state models. The second part of the paper compares the EX-KRLS in nonlinear Rayleigh multipath channel tracking and in Lorenz system modeling problem. We show that the proposed algorithm is able to outperform the standard KRLS and EX-RLS in both simulations.

A reproducing kernel hilbert space framework for spike train signal processing

This letter presents a general framework based on reproducing kernel Hilbert spaces (RKHS) to mathematically describe and manipulate spike trains. The main idea is the definition of inner products to allow spike train signal processing from basic principles while incorporating their statistical description as point processes. Moreover, because many inner products can be formulated, a particular definition can be crafted to best fit an application. These ideas are illustrated by the definition of a number of spike train inner products. To further elicit the advantages of the RKHS framework, a family of these inner products, the cross-intensity (CI) kernels, is analyzed in detail. This inner product family encapsulates the statistical description from the conditional intensity functions of spike trains. The problem of their estimation is also addressed. The simplest of the spike train kernels in this family provide an interesting perspective to others work, as will be demonstrated in terms of spike train distance measures. Finally, as an application example, the RKHS framework is used to derive a clustering algorithm for spike trains from simple principles.

Neuronal functional connectivity dynamics in cortex: An MSC-based analysis

The activation of neural ensembles in the cortex is correlated with behavioral states and a change in neuronal functional connectivity patterns is expected. In this paper, we investigate this dynamic nature of functional connectivity in the cortex. Because of the time scale of behavior, a robust method with limited sample size is desirable. In light of this, we utilize mean square contingency (MSC) to measure the pairwise neural dependency to quantify the cortical functional connectivity. Simulation results show that MSC is more robust than cross correlation when the sample size is small. In monkey neural data test, our approach is more effective in detecting the dynamics of functional connectivity associated with the transitions between rest and movement states.

A comparison of binless spike train measures

Several binless spike train measures which avoid the limitations of binning have been recently been proposed in the literature. This paper presents a systematic comparison of these measures in three simulated paradigms designed to address specific situations of interest in spike train analysis where the relevant feature may be in the form of firing rate, firing rate modulations, and/or synchrony. The measures are first disseminated and extended for ease of comparison. It also discusses how the measures can be used to measure dissimilarity in spike trains firing rate despite their explicit formulation for synchrony.

A Reproducing Kernel Hilbert Space Framework for Information-Theoretic Learning

This paper provides a functional analysis perspective of information-theoretic learning (ITL) by defining bottom-up a reproducing kernel Hilbert space (RKHS) uniquely determined by the symmetric nonnegative definite kernel function known as the cross-information potential (CIP). The CIP as an integral of the product of two probability density functions characterizes similarity between two stochastic functions. We prove the existence of a one-to-one congruence mapping between the ITL RKHS and the Hilbert space spanned by square integrable probability density functions. Therefore, all the statistical descriptors in the original information-theoretic learning formulation can be rewritten as algebraic computations on deterministic functional vectors in the ITL RKHS, instead of limiting the functional view to the estimators as is commonly done in kernel methods. A connection between the ITL RKHS and kernel approaches interested in quantifying the statistics of the projected data is also established.

An adaptive decoder from spike trains to micro-stimulation using kernel least-mean-squares (KLMS)

This paper proposes a nonlinear adaptive decoder for somatosensory micro-stimulation based on the kernel least mean square (KLMS) algorithm applied directly on the space of spike trains. Instead of using a binned representation of spike trains, we transform the vector of spike times into a function in reproducing kernel Hilbert space (RKHS), where the inner product of two spike time vectors is defined by a nonlinear cross intensity kernel. This representation encapsulates the statistical description of the point process that generates the spike trains, and bypasses the curse of dimensionality-resolution of the binned spike representations. We compare our method with two other methods based on binned data: GLM and KLMS, in reconstructing biphasic micro-stimulation. The results indicate that the KLMS based on RKHS for spike train is able to detect the timing, the shape and the amplitude of the biphasic stimulation with the best accuracy.

Strictly positive-definite spike train kernels for point-process divergences

Exploratory tools that are sensitive to arbitrary statistical variations in spike train observations open up the possibility of novel neuroscientific discoveries. Developing such tools, however, is difficult due to the lack of Euclidean structure of the spike train space, and an experimenter usually prefers simpler tools that capture only limited statistical features of the spike train, such as mean spike count or mean firing rate. We explore strictly positive-definite kernels on the space of spike trains to offer both a structural representation of this space and a platform for developing statistical measures that explore features beyond count or rate. We apply these kernels to construct measures of divergence between two point processes and use them for hypothesis testing, that is, to observe if two sets of spike trains originate from the same underlying probability law. Although there exist positive-definite spike train kernels in the literature, we establish that these kernels are not strictly definite and thus do not induce measures of divergence. We discuss the properties of both of these existing nonstrict kernels and the novel strict kernels in terms of their computational complexity, choice of free parameters, and performance on both synthetic and real data through kernel principal component analysis and hypothesis testing.

A Unified Framework for Quadratic Measures of Independence

This paper proposes a unified framework for several available measures of independence by generalizing the concept of information theoretic learning (ITL). The key component of ITL is the use of inner product between two density functions as a measure of similarity between two random variables. We show that by generalizing the inner product using a symmetric strictly positive-definite kernel and by choosing appropriate kernels, it is possible to reproduce a number of popular measures of independence. This unified framework also allows the design of new strictly positive-definite kernels and corresponding measures of independence. Following this framework we explore a new measure of independence and apply it in the context of linear independent component analysis (ICA). An attractive property of the proposed method is that it does not involve any free parameter and we demonstrate that it performs equally well compared to the existing methods for ICA.

Correntropy based Granger causality

We propose a novel nonlinear extension to Granger causality. It is derived from a nonlinear mapping of a stochastic process using the recently introduced generalized correlation measure called correntropy. The method is demonstrated by detecting the direction of coupling in a chaotic system where the original Granger causality failed.