András Lörincz

Physiological patterns in the hippocampo-entorhinal cortex system

The anatomical connectivity and intrinsic properties of entorhinal cortical neurons give rise to ordered patterns of ensemble activity. How entorhinal ensembles form, interact, and accomplish emergent processes such as memory formation is not well‐understood. We lack sufficient understanding of how neuronal ensembles in general can function transiently and distinctively from other neuronal ensembles. Ensemble interactions are bound, foremost, by anatomical connectivity and temporal constraints on neuronal discharge. We present an Overview of the structure of neuronal interactions within the entorhinal cortex and the rest of the hippocampal formation. We wish to highlight two principle features of entorhinal‐hippocampal interactions. First, large numbers of entorhinal neurons are organized into at least two distinct high‐frequency population patterns: gamma (40–100 Hz) frequency volleys and ripple (140–200 Hz) frequency volleys. These patterns occur coincident with other well‐defined electrophysiological patterns. Gamma frequency volleys are modulated by the theta cycle. Ripple frequency volleys occur on each sharp wave event. Second, these patterns occur dominantly in specific layers of the entorhinal cortex. Theta/gamma frequency volleys are the principle pattern observed in layers I–III, in the neurons that receive cortical inputs and project to the hippocampus. Ripple frequency volleys are the principle population pattern observed in layers V–VI, in the neurons that receive hippocampal output and project primarily to the neocortex. Further, we will highlight how these ensemble patterns organize interactions within distributed forebrain structures and support memory formation. Hippocampus 10:457–465, 2000

Learning tetris using the noisy cross-entropy method

The cross-entropy method is an efficient and general optimization algorithm. However, its applicability in reinforcement learning (RL) seems to be limited because it often converges to suboptimal policies. We apply noise for preventing early convergence of the cross-entropy method, using Tetris, a computer game, for demonstration. The resulting policy outperforms previous RL algorithms by almost two orders of magnitude.

Two-Phase Computational Model Training Long-Term Memories in the Entorhinal-Hippocampal Region

Abstract: The computational model described here is driven by the hypothesis that a major function of the entorhinal cortex (EC)‐hippocampal system is to alter synaptic connections in the neocortex. It is based on the following postulates: (1) The EC compares the difference between neocortical representations (primary input) and feedback information conveyed by the hippocampus (the “reconstructed input”). The difference between the primary input and the reconstructed input (termed “error”) initiates plastic changes in the hippocampal networks (error compensation). (2) Comparison of the primary input and reconstructed input requires that these representations are available simultaneously in the EC network. We suggest that compensation of time delays is achieved by predictive structures, such as the CA3 recurrent network and EC‐CA1 connections. (3) Alteration of intrahippocampal connections gives rise to a new hippocampal output. The hippocampus generates separated (independent) outputs, which, in turn, train long‐term memory traces in the EC (independent components, IC). The ICs of the long‐term memory trace are generated in a two‐step manner, the operations of which we attribute to the activities of the CA3 (whitening) and CA1 (separation) fields. (4) The different hippocampal fields can perform both nonlinear and linear operations, albeit at different times (theta and sharp phases). We suggest that long‐term memory is represented in a distributed and hierarchical reconstruction network, which is under the supervision of the hippocampal output. Several of these model predictions can be tested experimentally.

Inferior Temporal Neurons Show Greater Sensitivity to Nonaccidental than to Metric Shape Differences

It has long been known that macaque inferior temporal (IT) neurons tend to fire more strongly to some shapes than to others, and that different IT neurons can show markedly different shape preferences. Beyond the discovery that these preferences can be elicited by features of moderate complexity, no general principle of (nonface) object recognition had emerged by which this enormous variation in selectivity could be understood. Psychophysical, as well as computational work, suggests that one such principle is the difference between viewpoint-invariant, nonaccidental (NAP) and view-dependent, metric shape properties (MPs). We measured the responses of single IT neurons to objects differing in either a NAP (namely, a change in a geon) or an MP of a single part, shown at two orientations in depth. The cells were more sensitive to changes in NAPs than in MPs, even though the image variation (as assessed by wavelet-like measures) produced by the former were smaller than the latter. The magnitude of the response modulation from the rotation itself was, on average, similar to that produced by the NAP differences, although the image changes from the rotation were much greater than that produced by NAP differences. Multidimensional scaling of the neural responses indicated a NAP/MP dimension, independent of an orientation dimension. The present results thus demonstrate that a significant portion of the neural code of IT cells represents differences in NAPs rather than MPs. This code may enable immediate recognition of novel objects at new views.

Learning to play using low-complexity rule-based policies: illustrations through Ms. Pac-Man

In this article we propose a method that can deal with certain combinatorial reinforcement learning tasks. We demonstrate the approach in the popular Ms. Pac-Man game. We define a set of high-level observation and action modules, from which rule-based policies are constructed automatically. In these policies, actions are temporally extended, and may work concurrently. The policy of the agent is encoded by a compact decision list. The components of the list are selected from a large pool of rules, which can be either handcrafted or generated automatically. A suitable selection of rules is learnt by the cross-entropy method, a recent global optimization algorithm that fits our framework smoothly. Cross-entropy-optimized policies perform better than our hand-crafted policy, and reach the score of average human players. We argue that learning is successful mainly because (i) policies may apply concurrent actions and thus the policy space is sufficiently rich, (ii) the search is biased towards low-complexity policies and therefore, solutions with a compact description can be found quickly if they exist.

Online group-structured dictionary learning

We develop a dictionary learning method which is (i) online, (ii) enables overlapping group structures with (iii) non-convex sparsity-inducing regularization and (iv) handles the partially observable case. Structured sparsity and the related group norms have recently gained widespread attention in group-sparsity regularized problems in the case when the dictionary is assumed to be known and fixed. However, when the dictionary also needs to be learned, the problem is much more difficult. Only a few methods have been proposed to solve this problem, and they can handle two of these four desirable properties at most. To the best of our knowledge, our proposed method is the first one that possesses all of these properties. We investigate several interesting special cases of our framework, such as the online, structured, sparse non-negative matrix factorization, and demonstrate the efficiency of our algorithm with several numerical experiments.

Independent subspace analysis using geodesic spanning trees

A novel algorithm for performing Independent Subspace Analysis, the estimation of hidden independent subspaces is introduced. This task is a generalization of Independent Component Analysis. The algorithm works by estimating the multi-dimensional differential entropy. The estimation utilizes minimal geodesic spanning trees matched to the sample points. Numerical studies include (i) illustrative examples, (ii) a generalization of the cocktail-party problem to songs played by bands, and (iii) an example on mixed independent subspaces, where subspaces have dependent sources, which are pairwise independent.

Design and characterization of a windowless resonant photoacoustic chamber equipped with resonance locking circuitry

A novel design of a windowless resonant photoacoustic chamber with open acoustic filters and an electronic resonance locking circuitry is presented. The acoustic behavior of the cell and preliminary measurements on a certified gas mixture with a CO2 laser demonstrate the feasibility for trace gas monitoring.

Novel anti-cholesterol monoclonal immunoglobulin G antibodies as probes and potential modulators of membrane raft-dependent immune functions.

Natural autoantibodies against cholesterol are present in the sera of all healthy individuals; their function, production, and regulation, however, are still unclear. Here, we managed to produce two monoclonal anti-cholesterol antibodies (ACHAs) by immunizing mice with cholesterol-rich liposomes. The new ACHAs were specific to cholesterol and to some structurally closely related 3β-hydroxyl sterols, and they reacted with human lipoproteins VLDL, LDL, and HDL. They bound, usually with low avidity, to live human or murine lymphocyte and monocyte-macrophage cell lines, which was enhanced substantially by a moderate papain digestion of the cell surface, removing some protruding extracellular protein domains. Cell-bound ACHAs strongly colocalized with markers of cholesterol-rich lipid rafts and caveolae at the cell surface and intracellularly with markers of the endoplasmic reticulum and Golgi complex. These data suggest that these IgG ACHAs may serve as probes of clustered cholesterol (e.g., different lipid rafts) in live cells and thus may also have immunomodulatory potential.

ε-mdps: learning in varying environments

In this paper e-MDP-models are introduced and convergence theorems are proven using the generalized MDP framework of Szepesvari and Littman. Using this model family, we show that Q-learning is capable of finding near-optimal policies in varying environments. The potential of this new family of MDP models is illustrated via a reinforcement learning algorithm called event-learning which separates the optimization of decision making from the controller. We show that event-learning augmented by a particular controller, which gives rise to an e-MDP, enables near optimal performance even if considerable and sudden changes may occur in the environment. Illustrations are provided on the two-segment pendulum problem.