John Hertz

Information flow and temporal coding in primate pattern vision.

We perform time-resolved calculations of the information transmitted about visual patterns by neurons in primary visual and inferior temporal cortices. All measurable information is carried in an effective time-varying firing rate, obtained by averaging the neuronal response with a resolution no finer than about 25 ms in primary visual cortex and around twice that in inferior temporal cortex. We found no better way for a neuron receiving these messages to decode them than simply to count spikes for this long. Most of the information tends to be concentrated in one or, more often, two brief packets, one at the very beginning of the response and the other typically 100 ms later. The first packet is the most informative part of the message, but the second one generally contains new information. A small but significant part of the total information in the message accumulates gradually over the entire course of the response. These findings impose strong constraints on the codes used by these neurons.

Ising model for neural data: Model quality and approximate methods for extracting functional connectivity

We study pairwise Ising models for describing the statistics of multineuron spike trains, using data from a simulated cortical network. We explore efficient ways of finding the optimal couplings in these models and examine their statistical properties. To do this, we extract the optimal couplings for subsets of size up to 200 neurons, essentially exactly, using Boltzmann learning. We then study the quality of several approximate methods for finding the couplings by comparing their results with those found from Boltzmann learning. Two of these methods--inversion of the Thouless-Anderson-Palmer equations and an approximation proposed by Sessak and Monasson--are remarkably accurate. Using these approximations for larger subsets of neurons, we find that extracting couplings using data from a subset smaller than the full network tends systematically to overestimate their magnitude. This effect is described qualitatively by infinite-range spin-glass theory for the normal phase. We also show that a globally correlated input to the neurons in the network leads to a small increase in the average coupling. However, the pair-to-pair variation in the couplings is much larger than this and reflects intrinsic properties of the network. Finally, we study the quality of these models by comparing their entropies with that of the data. We find that they perform well for small subsets of the neurons in the network, but the fit quality starts to deteriorate as the subset size grows, signaling the need to include higher-order correlations to describe the statistics of large networks.

Decoding cortical neuronal signals: Network models, information estimation and spatial tuning

We have studied the encoding of spatial pattern information by complex cells in the primary visual cortex of awake monkeys. Three models for the conditional probabilities of different stimuli, given the neuronal response, were fit and compared using cross-validation. For our data, a feed-forward neural network proved to be the best of these models.The information carried by a cell about a stimulus set can be calculated from the estimated conditional probabilities. We performed a spatial spectroscopy of the encoding, examining how the transmitted information varies with both the average coarseness of the stimulus set and the coarseness differences within it. We find that each neuron encodes information about many features at multiple scales. Our data do not appear to allow a characterization of these variations in terms of the detection of simple single features such as oriented bars.

How well can we estimate the information carried in neuronal responses from limited samples

It is difficult to extract the information carried by neuronal responses about a set of stimuli because limited data samples result in biased es timates. Recently two improved procedures have been developed to calculate information from experimental results: a binning-and-correcting procedure and a neural network procedure. We have used data produced from a model of the spatiotemporal receptive fields of parvocellular and magnocellular lateral geniculate neurons to study the performance of these methods as a function of the number of trials used. Both procedures yield accurate results for one-dimensional neuronal codes. They can also be used to produce a reasonable estimate of the extra information in a three-dimensional code, in this instance, within 0.05-0.1 bit of the asymptotically calculated valueabout 10 of the total transmitted information. We believe that this performance is much more accurate than previous procedures.

Mean field theory for nonequilibrium network reconstruction.

There has been recent progress on inferring the structure of interactions in complex networks when they are in stationary states satisfying detailed balance, but little has been done for nonequilibrium systems. Here we introduce an approach to this problem, considering, as an example, the question of recovering the interactions in an asymmetrically coupled, synchronously updated Sherrington-Kirkpatrick model. We derive an exact iterative inversion algorithm and develop efficient approximations based on dynamical mean-field and Thouless-Anderson-Palmer equations that express the interactions in terms of equal-time and one-time-step-delayed correlation functions.

Statistical physics of pairwise probability models

Statistical models for describing the probability distribution over the states of biological systems are commonly used for dimensional reduction. Among these models, pairwise models are very attractive in part because they can be fit using a reasonable amount of data: knowledge of the mean values and correlations between pairs of elements in the system is sufficient. Not surprisingly, then, using pairwise models for studying neural data has been the focus of many studies in recent years. In this paper, we describe how tools from statistical physics can be employed for studying and using pairwise models. We build on our previous work on the subject and study the relation between different methods for fitting these models and evaluating their quality. In particular, using data from simulated cortical networks we study how the quality of various approximate methods for inferring the parameters in a pairwise model depends on the time bin chosen for binning the data. We also study the effect of the size of the time bin on the model quality itself, again using simulated data. We show that using finer time bins increases the quality of the pairwise model. We offer new ways of deriving the expressions reported in our previous work for assessing the quality of pairwise models.

Odour recognition and segmentation by a model olfactory bulb and cortex.

We present a model of an olfactory system that performs odour segmentation. Based on the anatomy and physiology of natural olfactory systems, it consists of a pair of coupled modules, bulb and cortex. The bulb encodes the odour inputs as oscillating patterns. The cortex functions as an associative memory: when the input from the bulb matches a pattern stored in the connections between its units, the cortical units resonate in an oscillatory pattern characteristic of that odour. Further circuitry transforms this oscillatory signal to a slowly varying feedback to the bulb. This feedback implements olfactory segmentation by suppressing the bulbar response to the pre-existing odour, thereby allowing subsequent odours to be singled out for recognition.

Cross-correlations in high-conductance states of a model cortical network

Neuronal firing correlations are studied using simulations of a simple network model for a cortical column in a high-conductance state with dynamically balanced excitation and inhibition. Although correlations between individual pairs of neurons exhibit considerable heterogeneity, population averages show systematic behavior. When the network is in a stationary state, the average correlations are generically small: correlation coefficients are of order 1N, where N is the number of neurons in the network. However, when the input to the network varies strongly in time, much larger values are found. In this situation, the network is out of balance, and the synaptic conductance is low, at times when the strongest firing occurs. However, examination of the correlation functions of synaptic currents reveals that after these bursts, balance is restored within a few milliseconds by a rapid increase in inhibitory synaptic conductance. These findings suggest an extension of the notion of the balanced state to include balanced fluctuations of synaptic currents, with a characteristic timescale of a few milliseconds.

Glassy transition and aging in a model without disorder.

We study the off-equilibrium relaxational dynamics of the Amit-Roginsky

Response Variability in Balanced Cortical Networks

\phi^3