András Lrincz

Separation theorem for independent subspace analysis and its consequences

Independent component analysis (ICA) - the theory of mixed, independent, non-Gaussian sources - has a central role in signal processing, computer vision and pattern recognition. One of the most fundamental conjectures of this research field is that independent subspace analysis (ISA) - the extension of the ICA problem, where groups of sources are independent - can be solved by traditional ICA followed by grouping the ICA components. The conjecture, called ISA separation principle, (i) has been rigorously proven for some distribution types recently, (ii) forms the basis of the state-of-the-art ISA solvers, (iii) enables one to estimate the unknown number and the dimensions of the sources efficiently, and (iv) can be extended to generalizations of the ISA task, such as different linear-, controlled-, post nonlinear-, complex valued-, partially observed problems, as well as to problems dealing with nonparametric source dynamics. Here, we shall review the advances on this field.

Neural Kalman filter

Anticipating future events is a crucial function of the central nervous system and can be modelled by Kalman filter-like mechanisms, which are optimal for predicting linear dynamical systems. Connectionist representation of such mechanisms with Hebbian learning rules has not yet been derived. We show that the recursive prediction error method offers a solution that can be mapped onto the entorhinal-hippocampal loop in a biologically plausible way. Model predictions are provided.

Letters: Spike-based cross-entropy method for reconstruction

Most neural optimization algorithms use either gradient tuning methods or complicated recurrent dynamics that may lead to suboptimal solutions or require huge number of iterations. Here we propose a framework based on the cross-entropy method (CEM). CEM is an efficient global optimization technique, but it requires batch access to many samples. We transcribed CEM to an online form and embedded it into a reconstruction network that finds optimal representations in a robust way as demonstrated by computer simulations. We argue that this framework allows for neural implementation and suggests a novel computational role for spikes in real neuronal systems.

Letters: Neurally plausible, non-combinatorial iterative independent process analysis

It has been shown recently that the identification of mixed hidden independent auto-regressive processes (independent process analysis, IPA), under certain conditions, can be free from combinatorial explosion. The key is that IPA can be reduced (i) to independent subspace analysis and then, via a novel decomposition technique called Separation Theorem, (ii) to independent component analysis. Here, we introduce an iterative scheme and its neural network representation that takes advantage of the reduction method and can accomplish the IPA task. Computer simulation illustrates the working of the algorithm.

Independent component analysis forms place cells in realistic robot simulations

It has been argued that the processing of sensory information in the entorhinal-hippocampal loop involves independent component analysis (ICA) on temporally concatenated inputs. Here, we demonstrate that ICA in a realistic robot simulation on a U-shaped track forms place fields similar to those found in rat experiments in vivo.

Simple conditions for forming triangular grids

We have used simple learning rules to study how firing maps containing triangular grids-as found in in vivo experiments-can be developed by Hebbian means in realistic robotic simulations. We started from typical non-local postrhinal neuronal responses. We found that anti-Hebbian weight pruning can develop triangular grids under certain conditions. Experimental evidences and the present study suggest that within this model, whitening is a bottom-up process, whereas weight pruning and possibly the non-linear extension of whitening to bottom-up information maximization are guided by top-down influences that reorganize entorhinal responses. We connect our model to the computational model of the entorhinal-hippocampal region of Lorincz and Buzsaki. In the joined model, the hippocampus is the origin of response reorganization. The joined model may provide insights for memory reorganization guided by hippocampal supervision.

Distributed high dimensional information theoretical image registration via random projections

Information theoretical measures, such as entropy, mutual information, and various divergences, exhibit robust characteristics in image registration applications. However, the estimation of these quantities is computationally intensive in high dimensions. On the other hand, consistent estimation from pairwise distances of the sample points is possible, which suits random projection (RP) based low dimensional embeddings. We adapt the RP technique to this task by means of a simple ensemble method. To the best of our knowledge, this is the first distributed, RP based information theoretical image registration approach. The efficiency of the method is demonstrated through numerical examples.