Eftychios A. Pnevmatikakis

A video Time Encoding Machine

Time encoding is a real-time asynchronous mechanism of mapping analog amplitude information into multidimensional time sequences. We investigate the exact representation of analog video streams with a Time Encoding Machine realized with a population of spiking neurons. We also provide an algorithm that perfectly recovers streaming video from the spike trains of the neural population. Finally, we analyze the quality of recovery of a space-time separable video stream encoded with a population of integrate-and-fire neurons and demonstrate that the quality of recovery increases as a function of the population size.

Faithful representation of stimuli with a population of integrate-and-fire neurons

We consider a formal model of stimulus encoding with a circuit consisting of a bank of filters and an ensemble of integrate-and-fire neurons. Such models arise in olfactory systems, vision, and hearing. We demonstrate that bandlimited stimuli can be faithfully represented with spike trains generated by the ensemble of neurons. We provide a stimulus reconstruction scheme based on the spike times of the ensemble of neurons and derive conditions for perfect recovery. The key result calls for the spike density of the neural population to be above the Nyquist rate. We also show that recovery is perfect if the number of neurons in the population is larger than a threshold value. Increasing the number of neurons to achieve a faithful representation of the sensory world is consistent with basic neurobiological thought. Finally we demonstrate that in general, the problem of faithful recovery of stimuli from the spike train of single neurons is ill posed. The stimulus can be recovered, however, from the information contained in the spike train of a population of neurons.

Video Time Encoding Machines

We investigate architectures for time encoding and time decoding of visual stimuli such as natural and synthetic video streams (movies, animation). The architecture for time encoding is akin to models of the early visual system. It consists of a bank of filters in cascade with single-input multi-output neural circuits. Neuron firing is based on either a threshold-and-fire or an integrate-and-fire spiking mechanism with feedback. We show that analog information is represented by the neural circuits as projections on a set of band-limited functions determined by the spike sequence. Under Nyquist-type and frame conditions, the encoded signal can be recovered from these projections with arbitrary precision. For the video time encoding machine architecture, we demonstrate that band-limited video streams of finite energy can be faithfully recovered from the spike trains and provide a stable algorithm for perfect recovery. The key condition for recovery calls for the number of neurons in the population to be above a threshold value.

Consistent recovery of sensory stimuli encoded with MIMO neural circuits

We consider the problem of reconstructing finite energy stimuli encoded with a population of spiking leaky integrate-and-fire neurons. The reconstructed signal satisfies a consistency condition: when passed through the same neuron, it triggers the same spike train as the original stimulus. The recovered stimulus has to also minimize a quadratic smoothness optimality criterion. We formulate the reconstruction as a spline interpolation problem for scalar as well as vector valued stimuli and show that the recovery has a unique solution. We provide explicit reconstruction algorithms for stimuli encoded with single as well as a population of integrate-and-fire neurons. We demonstrate how our reconstruction algorithms can be applied to stimuli encoded with ON-OFF neural circuits with feedback. Finally, we extend the formalism to multi-input multi-output neural circuits and demonstrate that vector-valued finite energy signals can be efficiently encoded by a neural population provided that its size is beyond a threshold value. Examples are given that demonstrate the potential applications of our methodology to systems neuroscience and neuromorphic engineering.

Primacy of Flexor Locomotor Pattern Revealed by Ancestral Reversion of Motor Neuron Identity

Spinal circuits can generate locomotor output in the absence of sensory or descending input, but the principles of locomotor circuit organization remain unclear. We sought insight into these principles by considering the elaboration of locomotor circuits across evolution. The identity of limb-innervating motor neurons was reverted to a state resembling that of motor neurons that direct undulatory swimming in primitive aquatic vertebrates, permitting assessment of the role of motor neuron identity in determining locomotor pattern. Two-photon imaging was coupled with spike inference to measure locomotor firing in hundreds of motor neurons in isolated mouse spinal cords. In wild-type preparations, we observed sequential recruitment of motor neurons innervating flexor muscles controlling progressively more distal joints. Strikingly, after reversion of motor neuron identity, virtually all firing patterns became distinctly flexor like. Our findings show that motor neuron identity directs locomotor circuit wiring and indicate the evolutionary primacy of flexor pattern generation.

Cerebellar granule cells acquire a widespread predictive feedback signal during motor learning

Cerebellar granule cells, which constitute half the brains neurons, supply Purkinje cells with contextual information necessary for motor learning, but how they encode this information is unknown. Here we show, using two-photon microscopy to track neural activity over multiple days of cerebellum-dependent eyeblink conditioning in mice, that granule cell populations acquire a dense representation of the anticipatory eyelid movement. Initially, granule cells responded to neutral visual and somatosensory stimuli as well as periorbital airpuffs used for training. As learning progressed, two-thirds of monitored granule cells acquired a conditional response whose timing matched or preceded the learned eyelid movements. Granule cell activity covaried trial by trial to form a redundant code. Many granule cells were also active during movements of nearby body structures. Thus, a predictive signal about the upcoming movement is widely available at the input stage of the cerebellar cortex, as required by forward models of cerebellar control.

Encoding Natural Scenes with Neural Circuits with Random Thresholds

We present a general framework for the reconstruction of natural video scenes encoded with a population of spiking neural circuits with random thresholds. The natural scenes are modeled as space-time functions that belong to a space of trigonometric polynomials. The visual encoding system consists of a bank of filters, modeling the visual receptive fields, in cascade with a population of neural circuits, modeling encoding in the early visual system. The neuron models considered include integrate-and-fire neurons and ON-OFF neuron pairs with threshold-and-fire spiking mechanisms. All thresholds are assumed to be random. We demonstrate that neural spiking is akin to taking noisy measurements on the stimulus both for time-varying and space-time-varying stimuli. We formulate the reconstruction problem as the minimization of a suitable cost functional in a finite-dimensional vector space and provide an explicit algorithm for stimulus recovery. We also present a general solution using the theory of smoothing splines in Reproducing Kernel Hilbert Spaces. We provide examples of both synthetic video as well as for natural scenes and demonstrate that the quality of the reconstruction degrades gracefully as the threshold variability of the neurons increases.

Spatiotemporal receptive fields of barrel cortex revealed by reverse correlation of synaptic input

Of all of the sensory areas, barrel cortex is among the best understood in terms of circuitry, yet least understood in terms of sensory function. We combined intracellular recording in rats with a multi-directional, multi-whisker stimulator system to estimate receptive fields by reverse correlation of stimuli to synaptic inputs. Spatiotemporal receptive fields were identified orders of magnitude faster than by conventional spike-based approaches, even for neurons with little spiking activity. Given a suitable stimulus representation, a linear model captured the stimulus-response relationship for all neurons with high accuracy. In contrast with conventional single-whisker stimuli, complex stimuli revealed markedly sharpened receptive fields, largely as a result of adaptation. This phenomenon allowed the surround to facilitate rather than to suppress responses to the principal whisker. Optimized stimuli enhanced firing in layers 4–6, but not in layers 2/3, which remained sparsely active. Surround facilitation through adaptation may be required for discriminating complex shapes and textures during natural sensing.

Bayesian spike inference from calcium imaging data

We present efficient Bayesian methods for extracting neuronal spiking information from calcium imaging data. The goal of our methods is to sample from the posterior distribution of spike trains and model parameters (baseline concentration, spike amplitude etc) given noisy calcium imaging data. We present discrete time algorithms where that the existence of a spike at each time bin using Gibbs methods, as well as continuous time algorithms that sample over the number of spikes and their locations at an arbitrary resolution using Metropolis-Hastings methods for point processes. We provide Rao-Blackwellized extensions that (i) marginalize over several model parameters and (ii) provide smooth estimates of the marginal spike posterior distribution in continuous time. Our methods serve as complements to standard point estimates and allow for quantification of uncertainty in estimating the underlying spike train and model parameters.

Reconstruction of sensory stimuli encoded with integrate-and-fire neurons with random thresholds

We present a general approach to the reconstruction of sensory stimuli encoded with leaky integrate-and-fire neurons with random thresholds. The stimuli are modeled as elements of a Reproducing Kernel Hilbert Space. The reconstruction is based on finding a stimulus that minimizes a regularized quadratic optimality criterion. We discuss in detail the reconstruction of sensory stimuli modeled as absolutely continuous functions as well as stimuli with absolutely continuous first-order derivatives. Reconstruction results are presented for stimuli encoded with single as well as a population of neurons. Examples are given that demonstrate the performance of the reconstruction algorithms as a function of threshold variability.