Garrett B. Stanley

A naturalistic open source movie for optical flow evaluation

Ground truth optical flow is difficult to measure in real scenes with natural motion. As a result, optical flow data sets are restricted in terms of size, complexity, and diversity, making optical flow algorithms difficult to train and test on realistic data. We introduce a new optical flow data set derived from the open source 3D animated short film Sintel. This data set has important features not present in the popular Middlebury flow evaluation: long sequences, large motions, specular reflections, motion blur, defocus blur, and atmospheric effects. Because the graphics data that generated the movie is open source, we are able to render scenes under conditions of varying complexity to evaluate where existing flow algorithms fail. We evaluate several recent optical flow algorithms and find that current highly-ranked methods on the Middlebury evaluation have difficulty with this more complex data set suggesting further research on optical flow estimation is needed. To validate the use of synthetic data, we compare the image- and flow-statistics of Sintel to those of real films and videos and show that they are similar. The data set, metrics, and evaluation website are publicly available.

Reconstruction of Natural Scenes from Ensemble Responses in the Lateral Geniculate Nucleus

A major challenge in studying sensory processing is to understand the meaning of the neural messages encoded in the spiking activity of neurons. From the recorded responses in a sensory circuit, what information can we extract about the outside world? Here we used a linear decoding technique to reconstruct spatiotemporal visual inputs from ensemble responses in the lateral geniculate nucleus (LGN) of the cat. From the activity of 177 cells, we have reconstructed natural scenes with recognizable moving objects. The quality of reconstruction depends on the number of cells. For each point in space, the quality of reconstruction begins to saturate at six to eight pairs of on and off cells, approaching the estimated coverage factor in the LGN of the cat. Thus, complex visual inputs can be reconstructed with a simple decoding algorithm, and these analyses provide a basis for understanding ensemble coding in the early visual pathway.

Encoding of Natural Scene Movies by Tonic and Burst Spikes in the Lateral Geniculate Nucleus

The role of the lateral geniculate nucleus (LGN) of the thalamus in visual encoding remains an open question. Here, we characterize the function of tonic and burst spikes in cat LGN X-cells in signaling features of natural stimuli. A significant increase in bursting was observed during natural stimulation (relative to white noise stimulation) and was linked to the strong correlation structure of the natural scene movies. Burst responses were triggered by specific stimulus events consisting of a prolonged inhibitory stimulus, followed by an excitatory stimulus, such as the movement of an object into the receptive field. LGN responses to natural scene movies were predicted using an integrate-and-fire (IF) framework and compared with experimentally observed responses. The standard IF model successfully predicted LGN responses to natural scene movies during tonic firing, indicating a linear relationship between stimulus and response. However, the IF model typically underpredicted the LGN response during periods of bursting, indicating a nonlinear amplification of the stimulus in the actual response. The addition of a burst mechanism to the IF model was necessary to accurately predict the entire LGN response. These results suggest that LGN bursts are an important part of the neural code, providing a nonlinear amplification of stimulus features that are typical of the natural environment.

The impact of chronic blood–brain barrier breach on intracortical electrode function

Brain-computer interfaces (BCIs) have allowed control of prosthetic limbs in paralyzed patients. Unfortunately, the electrodes of the BCI that interface with the brain only function for a short period of time before the signal quality on these electrodes becomes substantially diminished. To truly realize the potential of BCIs, it is imperative to have electrodes that function chronically. In order to elucidate the physiological determinants of a chronically functional neural interface, we studied the role of the blood-brain barrier (BBB) in electrode function, because it is a key mediator of neuronal hemostasis. We monitored the status of the BBB and the consequences of BBB breach on electrode function using non-invasive imaging, electrophysiology, genomic, and histological analyses. Rats implanted with commercially available intracortical electrodes demonstrated an inverse correlation between electrode performance and BBB breach over a period of 16 weeks. Genomic analysis showed that chronically functional electrodes elicit an enhanced wound healing response. Conversely, in poorly functioning electrodes, chronic BBB breach led to local accumulation of neurotoxic factors and an influx of pro-inflammatory myeloid cells, which negatively affect neuronal health. These findings were further verified in a subset of electrodes with graded electrophysiological performance. In this study, we determine the mechanistic link between intracortical electrode function and failure. Our results indicate that BBB status is a critical physiological determinant of intracortical electrode function and can inform future electrode design and biochemical intervention strategies to enhance the functional longevity of BCIs.

Thalamic synchrony and the adaptive gating of information flow to cortex

Although it has long been posited that sensory adaptation serves to enhance information flow in sensory pathways, the neural basis remains elusive. Simultaneous single-unit recordings in the thalamus and cortex in anesthetized rats showed that adaptation differentially influenced thalamus and cortex in a manner that fundamentally changed the nature of information conveyed about vibrissa motion. Using an ideal observer of cortical activity, we found that performance in detecting vibrissal deflections degraded with adaptation while performance in discriminating among vibrissal deflections of different velocities was enhanced, a trend not observed in thalamus. Analysis of simultaneously recorded thalamic neurons did reveal, however, an analogous adaptive change in thalamic synchrony that mirrored the cortical response. An integrate-and-fire model using experimentally measured thalamic input reproduced the observed transformations. The results here suggest a shift in coding strategy with adaptation that directly controls information relayed to cortex, which could have implications for encoding velocity signatures of textures.

Relationship between intracortical electrode design and chronic recording function.

Intracortical electrodes record neural signals directly from local populations of neurons in the brain, and conduct them to external electronics that control prosthetics. However, the relationship between electrode design, defined by shape, size and tethering; and long-term (chronic) stability of the neuron-electrode interface is poorly understood. Here, we studied the effects of various commercially available intracortical electrode designs that vary in shape (cylindrical, planar), size (15 μm, 50 μm and 75 μm), and tethering [electrode connections to connector with (tethered) and without tethering cable (untethered)] using histological, transcriptomic, and electrophysiological analyses over acute (3 day) and chronic (12 week) timepoints. Quantitative analysis of histological sections indicated that Michigan 50 μm (M50) and Michigan tethered (MT) electrodes induced significantly (p < 0.01) higher glial scarring, and lesser survival of neurons in regions of blood-brain barrier (BBB) breach when compared to microwire (MW) and Michigan 15 μm (M15) electrodes acutely and chronically. Gene expression analysis of the neurotoxic cytokines interleukin (Il)1 (Il1α, Il1β), Il6, Il17 (Il17a, Il17b, Il17f), and tumor necrosis factor alpha (Tnf) indicated that MW electrodes induced significantly (p < 0.05) reduced expression of these transcripts when compared to M15, M50 and FMAA electrodes chronically. Finally, electrophysiological assessment of electrode function indicated that MW electrodes performed significantly (p < 0.05) better than all other electrodes over a period of 12 weeks. These studies reveal that intracortical electrodes with smaller size, cylindrical shape, and without tethering cables produce significantly diminished inflammatory responses when compared to large, planar and tethered electrodes. These studies provide a platform for the rational design and assessment of chronically functional intracortical electrode implants in the future.

Adaptation to stimulus contrast and correlations during natural visual stimulation

In this study, we characterize the adaptation of neurons in the cat lateral geniculate nucleus to changes in stimulus contrast and correlations. By comparing responses to high- and low-contrast natural scene movie and white noise stimuli, we show that an increase in contrast or correlations results in receptive fields with faster temporal dynamics and stronger antagonistic surrounds, as well as decreases in gain and selectivity. We also observe contrast- and correlation-induced changes in the reliability and sparseness of neural responses. We find that reliability is determined primarily by processing in the receptive field (the effective contrast of the stimulus), while sparseness is determined by the interactions between several functional properties. These results reveal a number of adaptive phenomena and suggest that adaptation to stimulus contrast and correlations may play an important role in visual coding in a dynamic natural environment.

Computational subunits of visual cortical neurons revealed by artificial neural networks

A crucial step toward understanding visual processing is to obtain a comprehensive description of the relationship between visual stimuli and neuronal responses. Many neurons in the visual cortex exhibit nonlinear responses, making it difficult to characterize their stimulus–response relationships. Here, we recorded the responses of primary visual cortical neurons of the cat to spatiotemporal random-bar stimuli and trained artificial neural networks to predict the response of each neuron. The random initial connections in the networks consistently converged to regular patterns. Analyses of these connection patterns showed that the response of each complex cell to the random-bar stimuli could be well approximated by the sum of a small number of subunits resembling simple cells. The direction selectivity of each complex cell measured with drifting gratings was also well predicted by the combination of these subunits, indicating the generality of the model. These results are consistent with a simple functional model for complex cells and demonstrate the usefulness of the neural network method for revealing the stimulus–response transformations of nonlinear neurons.

Dynamic encoding of natural luminance sequences by LGN bursts

In the lateral geniculate nucleus (LGN) of the thalamus, visual stimulation produces two distinct types of responses known as tonic and burst. Due to the dynamics of the T-type Ca 2+ channels involved in burst generation, the type of response evoked by a particular stimulus depends on the resting membrane potential, which is controlled by a network of modulatory connections from other brain areas. In this study, we use simulated responses to natural scene movies to describe how modulatory and stimulus-driven changes in LGN membrane potential interact to determine the luminance sequences that trigger burst responses. We find that at low resting potentials, when the T channels are de-inactivated and bursts are relatively frequent, an excitatory stimulus transient alone is sufficient to evoke a burst. However, to evoke a burst at high resting potentials, when the T channels are inactivated and bursts are relatively rare, prolonged inhibitory stimulation followed by an excitatory transient is required. We also observe evidence of these effects in vivo, where analysis of experimental recordings demonstrates that the luminance sequences that trigger bursts can vary dramatically with the overall burst percentage of the response. To characterize the functional consequences of the effects of resting potential on burst generation, we simulate LGN responses to different luminance sequences at a range of resting potentials with and without a mechanism for generating bursts. Using analysis based on signal detection theory, we show that bursts enhance detection of specific luminance sequences, ranging from the onset of excitatory sequences at low resting potentials to the offset of inhibitory sequences at high resting potentials. These results suggest a dynamic role for burst responses during visual processing that may change according to behavioral state.

Cortical Excitation and Inhibition following Focal Traumatic Brain Injury

Cortical compression can be a significant problem in many types of brain injuries, such as brain trauma, localized brain edema, hematoma, focal cerebral ischemia, or brain tumors. Mechanical and cellular alterations can result in global changes in excitation and inhibition on the neuronal network level even in the absence of histologically significant cell injury, often manifesting clinically as seizures. Despite the importance and prevalence of this problem, however, the precise electrophysiological effects of brain injury have not been well characterized. In this study, the changes in electrophysiology were characterized following sustained cortical compression using large-scale, multielectrode measurement of multiunit activity in primary somatosensory cortex in a sensory-evoked, in vivo animal model. Immediately following the initiation of injury at a distal site, there was a period of suppression of the evoked response in the rat somatosensory cortex, followed by hyper-excitability that was accompanied by an increase in the spatial extent of cortical activation. Paired-pulse tactile stimulation revealed a dramatic shift in the excitatory/inhibitory dynamics, suggesting a longer term hyperexcitability of the cortical circuit following the initial suppression that could be linked to the disruption of one or more inhibitory mechanisms of the thalamocortical circuit. Together, our results showed that the use of a sensory-evoked response provided a robust and repeatable functional marker of the evolution of the consequences of mild injury, serving as an important step toward in vivo quantification of alterations in excitation and inhibition in the cortex in the setting of traumatic brain injury.