András Lőrincz

Cross-Entropy optimization for independent process analysis

We treat the problem of searching for hidden multi-dimensional independent auto-regressive processes. First, we transform the problem to Independent Subspace Analysis (ISA). Our main contribution concerns ISA. We show that under certain conditions, ISA is equivalent to a combinatorial optimization problem. For the solution of this optimization we apply the cross-entropy method. Numerical simulations indicate that the cross-entropy method can provide considerable improvements over other state-of-the-art methods.

Reinforcement learning with echo state networks

Function approximators are often used in reinforcement learning tasks with large or continuous state spaces. Artificial neural networks, among them recurrent neural networks are popular function approximators, especially in tasks where some kind of of memory is needed, like in real-world partially observable scenarios. However, convergence guarantees for such methods are rarely available. Here, we propose a method using a class of novel RNNs, the echo state networks. Proof of convergence to a bounded region is provided for k-order Markov decision processes. Runs on POMDPs were performed to test and illustrate the working of the architecture.

Fast adapting value estimation-based hybrid architecture for searching the world-wide web

The slogan that information is powerhas undergone a slight change. Today, information updatingis in the focus of interest. The largest source of information is the world-wide web. Fast search methods are in need for this enormous source. In this paper a hybrid architecture that combines soft support vector classification and reinforcement learning for value estimation is introduced for the evaluation of a link (a document) and its neighboring links (or documents), called the context of a document. The method is motivated by (i) large differences between such contexts on the web, (ii) the facilitation of goal oriented search using context classifiers, and (iii) attractive fast adaptation properties, that could counteract diversity of web environments. We demonstrate that value estimation-based fast adaptation offers considerable improvement over other known search methods.

Critical echo state networks

We are interested in the optimization of the recurrent connection structure of Echo State Networks (ESNs), because their topology can strongly influence performance. We study ESN predictive capacity by numerical simulations on Mackey-Glass time series, and find that a particular small subset of ESNs is much better than ordinary ESNs provided that the topology of the recurrent feedback connections satisfies certain conditions. We argue that the small subset separates two large sets of ESNs and this separation can be characterized in terms of phase transitions. With regard to the criticality of this phase transition, we introduce the notion of Critical Echo State Networks (CESN). We discuss why CESNs perform better than other ESNs.

Independent subspace analysis on innovations

Independent subspace analysis (ISA) that deals with multi-dimensional independent sources, is a generalization of independent component analysis (ICA). However, all known ISA algorithms may become ineffective when the sources possess temporal structure. The innovation process instead of the original mixtures has been proposed to solve ICA problems with temporal dependencies. Here we show that this strategy can be applied to ISA as well. We demonstrate the idea on a mixture of 3D processes and also on a mixture of facial pictures used as two-dimensional deterministic sources. ISA on innovations was able to find the original subspaces, while plain ISA was not.

Kalman filter control embedded into the reinforcement learning framework

There is a growing interest in using Kalman filter models in brain modeling. The question arises whether Kalman filter models can be used on-line not only for estimation but for control. The usual method of optimal control of Kalman filter makes use of off-line backward recursion, which is not satisfactory for this purpose. Here, it is shown that a slight modification of the linear-quadratic-gaussian Kalman filter model allows the on-line estimation of optimal control by using reinforcement learning and overcomes this difficulty. Moreover, the emerging learning rule for value estimation exhibits a Hebbian form, which is weighted by the error of the value estimation.

Optimistic initialization and greediness lead to polynomial time learning in factored MDPs

In this paper we propose an algorithm for polynomial-time reinforcement learning in factored Markov decision processes (FMDPs). The factored optimistic initial model (FOIM) algorithm, maintains an empirical model of the FMDP in a conventional way, and always follows a greedy policy with respect to its model. The only trick of the algorithm is that the model is initialized optimistically. We prove that with suitable initialization (i) FOIM converges to the fixed point of approximate value iteration (AVI); (ii) the number of steps when the agent makes non-near-optimal decisions (with respect to the solution of AVI) is polynomial in all relevant quantities; (iii) the per-step costs of the algorithm are also polynomial. To our best knowledge, FOIM is the first algorithm with these properties.

Spatio-temporal Event Classification Using Time-Series Kernel Based Structured Sparsity

In many behavioral domains, such as facial expression and gesture, sparse structure is prevalent. This sparsity would be well suited for event detection but for one problem. Features typically are confounded by alignment error in space and time. As a consequence, high-dimensional representations such as SIFT and Gabor features have been favored despite their much greater computational cost and potential loss of information. We propose a Kernel Structured Sparsity (KSS) method that can handle both the temporal alignment problem and the structured sparse reconstruction within a common framework, and it can rely on simple features. We characterize spatio-temporal events as time-series of motion patterns and by utilizing time-series kernels we apply standard structured-sparse coding techniques to tackle this important problem. We evaluated the KSS method using both gesture and facial expression datasets that include spontaneous behavior and differ in degree of difficulty and type of ground truth coding. KSS outperformed both sparse and non-sparse methods that utilize complex image features and their temporal extensions. In the case of early facial event classification KSS had 10% higher accuracy as measured by F1 score over kernel SVM methods.

A framework for anonymous but accountable self-organizing communities

Abstract In this paper we propose a novel architecture and approach to provide accountability for Web communities that require a high-level of privacy. A two-layered privacy protection architecture is proposed, that supports (i) registration of participants and enforcement of community rules, called internal accountability , and (ii) rule-based interaction with real world organizations, called external accountability . Our security protocols build upon community-based trust and limit the exposure of private data on trusted third parties. The two-layered architecture protects the mappings between real users and their virtual identities, and among the virtual users, while guaranteeing internal and external accountability. We target Web communities that are dynamic and self-organizing, i.e. roles and contributions of participants may change over time. The proposed concepts and protocols are implemented in our SyllabNet project that supports anonymous course evaluations by university students.

Non-combinatorial estimation of independent autoregressive sources

Abstract Identification of mixed independent subspaces is thought to suffer from combinatorial explosion of two kinds: the minimization of mutual information between the estimated subspaces and the search for the optimal number and dimensions of the subspaces. Here we show that independent autoregressive process analysis, under certain conditions, can avoid this problem using a two-phase estimation process. We illustrate the solution by computer demonstration.