Dileep George

Towards a Mathematical Theory of Cortical Micro-circuits

The theoretical setting of hierarchical Bayesian inference is gaining acceptance as a framework for understanding cortical computation. In this paper, we describe how Bayesian belief propagation in a spatio-temporal hierarchical model, called Hierarchical Temporal Memory (HTM), can lead to a mathematical model for cortical circuits. An HTM node is abstracted using a coincidence detector and a mixture of Markov chains. Bayesian belief propagation equations for such an HTM node define a set of functional constraints for a neuronal implementation. Anatomical data provide a contrasting set of organizational constraints. The combination of these two constraints suggests a theoretically derived interpretation for many anatomical and physiological features and predicts several others. We describe the pattern recognition capabilities of HTM networks and demonstrate the application of the derived circuits for modeling the subjective contour effect. We also discuss how the theory and the circuit can be extended to explain cortical features that are not explained by the current model and describe testable predictions that can be derived from the model.

A hierarchical Bayesian model of invariant pattern recognition in the visual cortex

We describe a hierarchical model of invariant visual pattern recognition in the visual cortex. In this model, the knowledge of how patterns change when objects move is learned and encapsulated in terms of high probability sequences at each level of the hierarchy. Configuration of object parts is captured by the patterns of coincident high probability sequences. This knowledge is then encoded in a highly efficient Bayesian network structure. The learning algorithm uses a temporal stability criterion to discover object concepts and movement patterns. We show that the architecture and algorithms are biologically plausible. The large scale architecture of the system matches the large scale organization of the cortex and the micro-circuits derived from the local computations match the anatomical data on cortical circuits. The system exhibits invariance across a wide variety of transformations and is robust in the presence of noise. Moreover, the model also offers alternative explanations for various known cortical phenomena.

Sequence memory for prediction, inference and behaviour.

In this paper, we propose a mechanism which the neocortex may use to store sequences of patterns. Storing and recalling sequences are necessary for making predictions, recognizing time-based patterns and generating behaviour. Since these tasks are major functions of the neocortex, the ability to store and recall time-based sequences is probably a key attribute of many, if not all, cortical areas. Previously, we have proposed that the neocortex can be modelled as a hierarchy of memory regions, each of which learns and recalls sequences. This paper proposes how each region of neocortex might learn the sequences necessary for this theory. The basis of the proposal is that all the cells in a cortical column share bottom-up receptive field properties, but individual cells in a column learn to represent unique incidences of the bottom-up receptive field property within different sequences. We discuss the proposal, the biological constraints that led to it and some results modelling it.

Computing with inter-spike interval codes in networks of integrate and fire neurons

Information encoding in spikes and computations performed by spiking neurons are two sides of the same coin and should be consistent with each other. This study uses this consistency requirement to derive some new results for inter-spike interval (ISI) coding in networks of integrate and fire (IF) neurons. Our analysis shows that such a model can carry out useful computations and that it does also account for variability in spike timing as observed in cortical neurons. Our general result is that IF type neurons, though highly non-linear, perform a simple linear weighted sum operation of ISI coded quantities. Further, we derive bounds on the variation of ISIs that occur in the model although the neurons are deterministic. We also derive useful estimates of the maximum processing speed in a hierarchical network.

Discovering Ecosystem Models from Time-Series Data

Ecosystem models are used to interpret and predict the interactions of species and their environment. In this paper, we address the task of inducing ecosystem models from background knowledge and time-series data, and we review IPM, an algorithm that addresses this problem. We demonstrate the system’s ability to construct ecosystem models on two different Earth science data sets. We also compare its behavior with that produced by a more conventional autoregression method. In closing, we discuss related work on model induction and suggest directions for further research on this topic.

Cortical Microcircuits from a Generative Vision Model.

Understanding the information processing roles of cortical circuits is an outstanding problem in neuroscience and artificial intelligence. The theoretical setting of Bayesian inference has been suggested as a framework for understanding cortical computation. Based on a recently published generative model for visual inference (George et al., 2017), we derive a family of anatomically instantiated and functional cortical circuit models. In contrast to simplistic models of Bayesian inference, the underlying generative model’s representational choices are validated with real-world tasks that required efficient inference and strong generalization. The cortical circuit model is derived by systematically comparing the computational requirements of this model with known anatomical constraints. The derived model suggests precise functional roles for the feedforward, feedback and lateral connections observed in different laminae and columns, and assigns a computational role for the path through the thalamus.

Explaining Visual Cortex Phenomena using Recursive Cortical Network

The connectivity and information pathways of visual cortex are well studied, as are observed physiological phenomena, yet a cohesive model for explaining visual cortex processes remains an open problem. For a comprehensive understanding, we need to build models of the visual cortex that are capable of robust real-world performance, while also being able to explain psychophysical and physiological observations. To this end, we demonstrate how the Recursive Cortical Network (George et al., 2017) can be used as a computational model to reproduce and explain subjective contours, neon color spreading, occlusion vs. deletion, and the border-ownership competition phenomena observed in the visual cortex.

Inducing Biological Models from Temporal Gene Expression Data

We applied Inductive Process Modeling (Langley et al., in press) to induce biological process models from background knowledge and temporal gene expression data relating to the regulation of bacterial photosynthesis. Labiosa et al. (2003) studied the regulation of all of the genes in the Cyanobacterium Synechocystis sp. 6803. They simulated the natural day/night light cycle in a continuous culture cyclostat, and extracted samples at 2AM, 8AM, 10AM, noon, 2PM, 6PM, and midnight. Whole-cell RNA from these samples was converted to cDNA and hybridized to DNA microarrays, thereby measuring the abundance of RNA transcripts for all the genes in the organism at the selected times. Many of the photosynthesis-related RNAs show low abundance at night and increase rapidly when the sun rises, but these also exhibit an ‘M-shaped’ pattern with a substantial decrease at noon.