Bailu Si

Grid alignment in entorhinal cortex

The spatial responses of many of the cells recorded in all layers of rodent medial entorhinal cortex (mEC) show mutually aligned grid patterns. Recent experimental findings have shown that grids can often be better described as elliptical rather than purely circular and that, beyond the mutual alignment of their grid axes, ellipses tend to also orient their long axis along preferred directions. Are grid alignment and ellipse orientation aspects of the same phenomenon? Does the grid alignment result from single-unit mechanisms or does it require network interactions? We address these issues by refining a single-unit adaptation model of grid formation, to describe specifically the spontaneous emergence of conjunctive grid-by-head-direction cells in layers III, V, and VI of mEC. We find that tight alignment can be produced by recurrent collateral interactions, but this requires head-direction (HD) modulation. Through a competitive learning process driven by spatial inputs, grid fields then form already aligned, and with randomly distributed spatial phases. In addition, we find that the self-organization process is influenced by any anisotropy in the behavior of the simulated rat. The common grid alignment often orients along preferred running directions (RDs), as induced in a square environment. When speed anisotropy is present in exploration behavior, the shape of individual grids is distorted toward an ellipsoid arrangement. Speed anisotropy orients the long ellipse axis along the fast direction. Speed anisotropy on its own also tends to align grids, even without collaterals, but the alignment is seen to be loose. Finally, the alignment of spatial grid fields in multiple environments shows that the network expresses the same set of grid fields across environments, modulo a coherent rotation and translation. Thus, an efficient metric encoding of space may emerge through spontaneous pattern formation at the single-unit level, but it is coherent, hence context-invariant, if aided by collateral interactions.

Self-organization of multiple spatial and context memories in the hippocampus.

One obstacle to understanding the exact processes unfolding inside the hippocampus is that it is still difficult to clearly define what the hippocampus actually does, at the system level. Associated for a long time with the formation of episodic and semantic memories, and with their temporary storage, the hippocampus is also regarded as a structure involved in spatial navigation. These two independent perspectives on the hippocampus are not necessarily exclusive: proposals have been put forward to make them fit into the same conceptual frame. We review both approaches and argue that three critical developments need consideration: (a) recordings of neuronal activity in rodents, revealing beautiful spatial codes expressed in entorhinal cortex, upstream of the hippocampus; (b) comparative behavioral results suggesting, in an evolutionary perspective, qualitative similarity of function across homologous structures with a distinct internal organization; (c) quantitative measures of information, shifting the focus from who does what to how much each neuronal population expresses each code. These developments take the hippocampus away from philosophical discussions of all-or-none cause-effect relations, and into the quantitative mainstream of modern neural science.

Robot Exploration by Subjectively Maximizing Objective Information Gain

Localization, mapping and action selection are three main aspects in robot exploration. This paper proposes an autonomous exploration method for robot localization and mapping in unknown environments. First an ideal global-probabilistic measure, which we call objective objective function, is introduced to evaluate the objective exploration performance. By minimizing a local approximation of this measure (which we call subjective objective function) the robot learns the internal models, and achieves a consistent correlation between the internal representation and the reality. Furthermore, an action policy search method is used to learn the optimal action selection strategy by maximizing the information gain obtained in exploration. Simulation results demonstrate that the proposed framework provides an integrated solution for localization and mapping task in unstructured environment

Continuous attractor network model for conjunctive position-by-velocity tuning of grid cells.

The spatial responses of many of the cells recorded in layer II of rodent medial entorhinal cortex (MEC) show a triangular grid pattern, which appears to provide an accurate population code for animal spatial position. In layer III, V and VI of the rat MEC, grid cells are also selective to head-direction and are modulated by the speed of the animal. Several putative mechanisms of grid-like maps were proposed, including attractor network dynamics, interactions with theta oscillations or single-unit mechanisms such as firing rate adaptation. In this paper, we present a new attractor network model that accounts for the conjunctive position-by-velocity selectivity of grid cells. Our network model is able to perform robust path integration even when the recurrent connections are subject to random perturbations.

A Model for the Differentiation Between Grid and Conjunctive Units in Medial Entorhinal Cortex

The multiple layers of medial entorhinal cortex (mEC) contain cells that differ in selectivity, connectivity, and cellular properties. Grid cells in layer II and in the deeper layers express triangular grid patterns in the environment. The firing rate of the conjunctive cells found in layer III and below, on the other hand, show grid‐by‐head direction tuning. In this study, we model the differentiation between grid and conjunctive cells in a network with self‐organized connections. Arranged into distinct “layers”, the model grid units and conjunctive units develop, with a similar time course, grid fields resulting from firing rate adaptation and competitive learning. Grid alignment in both layers is delayed with respect to the formation of triangular grids. A common grid orientation among conjunctive units is produced, in the model, by head‐direction modulated collateral interactions, while the grids of grid units inherit the same orientation through connections from conjunctive units. Grid units as well as conjunctive units share a similar spacing but show a random distribution of spatial phases. Grid units however carry more spatial information than conjunctive units, thus providing better inputs for the hippocampus to form spatial memories.

Grid cells on the ball

What sort of grid cells do we expect to see in rodents who have spent their developmental period inside a large spherical cage? Or, in a different experimental paradigm, toddling on a revolving ball, with virtual reality simulating a coherently revolving surround? We consider a simple model of grid firing map formation, based on firing rate adaptation, that we have earlier analyzed when playing out on a flat environment. The model predicts that whether experienced on the outside or inside, a spherical environment induces one of a succession of grid maps realized as combinations of spherical harmonics, depending on the relation of the radius to the preferred grid spacing, itself related to the parameters of firing rate adaptation. Numerical simulations concur with analytical predictions.

Grid maps for spaceflight, anyone? They are for free!

We show that, given extensive exploration of a three-dimensional volume, grid units can form with the approximate periodicity of a face-centered cubic crystal, as the spontaneous product of a self-organizing process at the single unit level, driven solely by firing rate adaptation.

Local Autoencoding for Parameter Estimation in a Hidden Potts-Markov Random Field

A local-autoencoding (LAE) method is proposed for the parameter estimation in a Hidden Potts-Markov random field model. Due to sampling cost, Markov chain Monte Carlo methods are rarely used in real-time applications. Like other heuristic methods, LAE is based on a conditional independence assumption. It adapts, however, the parameters in a block-by-block style with a simple Hebbian learning rule. Experiments with given label fields show that the LAE is able to converge in far less time than required for a scan. It is also possible to derive an estimate for LAE based on a Cramer-Rao bound that is similar to the classical maximum pseudolikelihood method. As a general algorithm, LAE can be used to estimate the parameters in anisotropic label fields. Furthermore, LAE is not limited to the classical Potts model and can be applied to other types of Potts models by simple label field transformations and straightforward learning rule extensions. Experimental results on image segmentations demonstrate the efficiency and generality of the LAE algorithm.

Gain-based Exploration: From Multi-armed Bandits to Partially Observable Environments

We introduce gain-based policies for exploration in active learning problems. For exploration in multi-armed bandits with the knowledge of reward variances, an ideal gain-maximization exploration policy is described in a unified framework which also includes error-based and counter-based exploration. For realistic situations without prior knowledge of reward variances, we establish an upper bound on the gain function, resulting in a realistic gain- maximization exploration policy which achieves the optimal exploration asymptotically. Finally, we extend the gain- maximization exploration scheme towards partially observable environments. Approximating the environment by a set of local bandits, the agent actively selects its actions by maximizing discounted gain in learning local bandits. The resulting gain-based exploration not only outperforms random exploration, but also produces curiosity-driven behavior which is observed in natural agents.

Label field initialization for MRF-based sonar image segmentation by selective autoencoding

The optimal solution of a Markov random field (MRF) can be solved by constructing a Markov chain that eventually goes to a balance state. However, in most situations, only an suboptimal solution can be obtained, because it is hard to choose the ideal initial state and the updating strategy. While the updating strategy has been extensively investigated, the initialization issue has been fully neglected. Though k-means-clustering has been used exclusively in initializing the label field, it suffers from the lack of account of the local constraints, which is the most essential part of the MRF model. A structural method based on selective autoencoding (SAE) is proposed for the label field initialization of MRF model in the task of sonar image segmentation. SAE is similar to the AutoEncoder, with the largest difference on the activation function, where a piece-wise sigmoid activation function with two different slop parameters is used to selectively encode image patches that resemble shadow ares or other areas. The synapse matrixes of SAE network act as information filters, preserve specific area adaptively and selectively, generating a label field that is much closer to the balance state. Experiments on sonar image segmentation demonstrate the efficiency of the SAE algorithm.