John E. W. Mayhew

PMF: A Stereo Correspondence Algorithm Using a Disparity Gradient Limit

The advantages of solving the stereo correspondence problem by imposing a limit on the magnitude of allowable disparity gradients are examined. It is shown how the imposition of such a limit can provide a suitable balance between the twin requirements of disambiguating power and the ability to deal with a wide range of surfaces. Next, the design of a very simple stereo algorithm called PMF is described. In conjunction with certain other constraints used in many other stereo algorithms, PMF employs a limit on allowable disparity gradients of 1, a value that coincides with that reported for human stereoscopic vision. The excellent performance of PMF is illustrated on a series of natural and artificial stereograms. Finally, the differences between the theoretical justification for the use of disparity gradients for solving the stereo correspondence problems presented in the paper and others that exist in the stereo algorithm literature are discussed.

Psychophysical and computational studies towards a theory of human stereopsis

Psychophysical studies are described which pose a strong challenge to models of human stereopsis based on the processing of disparity information within independent spatial frequency tuned binocular channels. These studies support instead the proposal that the processes of human binocular combination integrally relate the extraction of disparity information with the construction of raw primal sketch assertions. This proposal implies global binocular combination rules using principles of figural continuity and cross-channel correspondence to disambiguate local matches found independently within spatial frequency channels. Exploratory small-scale computational experiments with stereo algorithms based on these rules are described and found to be successful in dealing with a variety of stereo inputs. The constraints presented by objects which are exploited by these algorithms are discussed.

Concurrent optical imaging spectroscopy and laser-Doppler flowmetry: the relationship between blood flow, oxygenation, and volume in rodent barrel cortex.

Functional magnetic resonance imaging (fMRI) is based on the coupling between neural activity and changes in the concentration of the endogenous paramagnetic contrast agent deoxygenated hemoglobin. Changes in the blood oxygen level-dependent (BOLD) signal result from a complex interplay of blood volume, flow, and oxygen consumption. Optical imaging spectroscopy (OIS) has been used to measure changes in blood volume and saturation in response to increased neural activity, while laser Doppler Flowmetry (LDF) can be used to measure flow changes and is now commonplace in neurovascular research. Here, we use concurrent OIS and LDF to examine the hemodynamic response in rodent barrel cortex using electrical stimulation of the whisker pad at varying intensities. Spectroscopic analysis showed that stimulation produced a biphasic early increase in deoxygenated hemoglobin (Hbr), followed by a decrease below baseline, reaching minima at approximately 3.7 s. There was no evidence for a corresponding early decrease in oxygenated hemoglobin (HbO(2)), which simply increased after stimulation, reaching maximum at approximately 3.2 s. The time courses of changes in blood volume (CBV) and blood flow (CBF) were similar. Both increased within a second of stimulation onset and peaked at approximately 2.7 s, after which CBV returned to baseline at a slower rate than CBF. The changes in Hbr, Hbt, and CBF were used to estimate changes in oxygen consumption (CMRO(2)), which increased within a second of stimulation and peaked approximately 2.2 s after stimulus onset. Analysis of the relative magnitudes of CBV and CBF indicates that the fractional changes of CBV could be simply scaled to match those of CBF. We found the relationship to be well approximated by CBV = CBF(0.29). A similar relationship was found using the response to elevated fraction of inspired carbon dioxide (FICO(2)).

Investigating neural-hemodynamic coupling and the hemodynamic response function in the awake rat.

An understanding of the relationship between changes in neural activity and the accompanying hemodynamic response is crucial for accurate interpretation of functional brain imaging data and in particular the blood oxygen level-dependent (BOLD) fMRI signal. Much physiological research investigating this topic uses anesthetized animal preparations, and yet, the effects of anesthesia upon the neural and hemodynamic responses measured in such studies are not well understood. In this study, we electrically stimulated the whisker pad of both awake and urethane anesthetized rats at frequencies of 1-40 Hz. Evoked field potential responses were recorded using electrodes implanted into the contralateral barrel cortex. Changes in hemoglobin oxygenation and concentration were measured using optical imaging spectroscopy, and cerebral blood flow changes were measured using laser Doppler flowmetry. A linear neural-hemodynamic coupling relationship was found in the awake but not the anesthetized animal preparation. Over the range of stimulation conditions studied, hemodynamic response magnitude increased monotonically with summed neural activity in awake, but not in anesthetized, animals. Additionally, the temporal structure of the hemodynamic response function was different in awake compared to anesthetized animals. The responses in each case were well approximated by gamma variates, but these were different in terms of mean latency (approximately 2 s awake; 4 s anesthetized) and width (approximately 0.6 s awake; 2.5 s anesthetized). These findings have important implications for research into the intrinsic signals that underpin BOLD fMRI and for biophysical models of cortical hemodynamics and neural-hemodynamic coupling.

Spectroscopic analysis of neural activity in brain: increased oxygen consumption following activation of barrel cortex.

This research investigates the hemodynamic response to stimulation of the barrel cortex in anaesthetized rats using optical imaging and spectroscopy (Bonhoeffer and Grinvald, 1996; Malonek and Grinvald, 1996; Mayhew et al., 1999). A slit spectrograph was used to collect spectral image data sequences. These were analyzed using an algorithm that corrects for the wavelength dependency in the optical path lengths produced by the light scattering properties of tissue. The analysis produced the changes in the oxy- and deoxygenation of hemoglobin following stimulation. Two methods of stimulation were used. One method mechanically vibrated a single whisker, the other electrically stimulated the whisker pad. The electrical stimulation intensity varied from 0.4 to 1.6 mA. The hemodynamic responses to stimulation increased as a function of intensity. At 0.4 mA they were commensurate with those from the mechanical stimulation; however, the responses at the higher levels were greater by a factor of approximately 10. For both methods of data collection, the results of the spectroscopic analysis showed an early increase in deoxygenated hemoglobin (Hbr) with no evidence for a corresponding decrease in oxygenated hemoglobin (HbO(2)). Evidence for increased oxygen consumption (CMRO(2)) was obtained by converting the fractional changes in blood volume (Hbt) into estimates of changes in blood flow (Grubb et al., 1974) and using the resulting time course to scale the fractional changes in Hbr. The results show an early increase CMRO(2) peaking approximately 2 s after stimulation onset. Using these methods, we find evidence for increased oxygen consumption following increased neural activity even at low levels of stimulation intensity.

Spectroscopic analysis of changes in remitted illumination: the response to increased neural activity in brain.

Imaging of neural activation has been used to produce maps of functional architecture and metabolic activity. There is some uncertainty associated with the sources underlying the intrinsic signals. It has been reported that following increased neural activity there was little increased oxygen consumption ( approximately 5%), although glucose consumption increased by approximately 50%. The research we describe uses a modification of the Beer-Lambert Law called path-length scaling analysis (PLSA) to analyze the spectra of the hemodynamic and metabolic responses to vibrissal stimulation in rat somatosensory cortex. The results of the PLSA algorithm were compared with those obtained using a linear spectrographic analysis method (we refer to this as LMCA). There are differences in the results of the analysis depending on which of the two algorithms (PLSA or LMCA) is used. Using the LMCA algorithm, we obtain results showing an increase in the volume of Hbr at approximately 2 s, following onset of stimulation but no complementary decrease in oxygenated haemoglobin (HbO(2)). These results are similar to a previous report. In contrast, after using the PLSA algorithm, the time series of the chromophore changes shows no evidence for an increase in the volume of deoxygenated haemoglobin (Hbr). However, after further analysis of the time series from the PLSA using general linear models (GLM) to remove contributions from low frequency baseline oscillations, both the HbO(2) and Hbr times series of the response to stimulation were found to be biphasic with an early decrease in saturation peaking approximately 1 s after onset of stimulation followed by a larger increase in saturation peaking at approximately 3 s. Finally, following the PLSA-then-GLM analysis procedure, we do not find convincing evidence for an increase in cytochrome oxidation following stimulation, though we demonstrate the PLSA algorithm to be capable of disassociating changes in cytochrome oxidation state from changes in hemoglobin oxygenation.

The hemodynamic impulse response to a single neural event.

This article investigates the relation between stimulus-evoked neural activity and cerebral hemodynamics. Specifically, the hypothesis is tested that hemodynamic responses can be modeled as a linear convolution of experimentally obtained measures of neural activity with a suitable hemodynamic impulse response function. To obtain a range of neural and hemodynamic responses, rat whisker pad was stimulated using brief (≤2 seconds) electrical stimuli consisting of single pulses (0.3 millisecond, 1.2 mA) combined both at different frequencies and in a paired-pulse design. Hemodynamic responses were measured using concurrent optical imaging spectroscopy and laser Doppler flowmetry, whereas neural responses were assessed through current source density analysis of multielectrode recordings from a single barrel. General linear modeling was used to deconvolve the hemodynamic impulse response to a single “neural event” from the hemodynamic and neural responses to stimulation. The model provided an excellent fit to the empirical data. The implications of these results for modeling schemes and for physiologic systems coupling neural and hemodynamic activity are discussed.

Stereopsis, vertical disparity and relief transformations.

The pattern of retinal binocular disparities acquired by a fixating visual system depends on both the depth structure of the scene and the viewing geometry. This paper treats the problem of interpreting the disparity pattern in terms of scene structure without relying on estimates of fixation position from eye movement control and proprioception mechanisms. We propose a sequential decomposition of this interpretation process into disparity correction, which is used to compute three-dimensional structure up to a relief transformation, and disparity normalization, which is used to resolve the relief ambiguity to obtain metric structure. We point out that the disparity normalization stage can often be omitted, since relief transformations preserve important properties such as depth ordering and coplanarity. Based on this framework we analyse three previously proposed computational models of disparity processing; the Mayhew and Longuet-Higgins model, the deformation model and the polar angle disparity model. We show how these models are related, and argue that none of them can account satisfactorily for available psychophysical data. We therefore propose an alternative model, regional disparity correction. Using this model we derive predictions for a number of experiments based on vertical disparity manipulations, and compare them to available experimental data. The paper is concluded with a summary and a discussion of the possible architectures and mechanisms underling stereopsis in the human visual system.

Learning and maintaining saccadic accuracy: A model of brainstem--cerebellar interactions

Saccadic accuracy requires that the control signal sent to the motor neurons must be the right size to bring the fovea to the target, whatever the initial position of the eyes (and corresponding state of the eye muscles). Clinical and experimental evidence indicates that the basic machinery for generating saccadic eye movements, located in the brainstem, is not accurate: learning to make accurate saccades requires cerebellar circuitry located in the posterior vermis and fastigial nucleus. How do these two circuits interact to achieve adaptive control of saccades? A model of this interaction is described, based on Kawatos principle of feedback-error-learning. Its three components were (1) a simple controller with no knowledge of initial eye position, corresponding to the superior colliculus; (2) Robinsons internal feedback model of the saccadic burst generator, corresponding to preoculomotor areas in the brain-stem; and (3) Albuss Cerebellar Model Arithmetic Computer (CMK), a neural net model of the cerebellum. The connections between these components were (I) the simple feedback controller passed a (usually inaccurate) command to the pulse generator, and (2) a copy of this command to the CMAC; (3) the CMAC combined the copy with information about initial eye position to (4) alter the gain on the pulse generators internal feedback loop, thereby adjusting the size of burst sent to the motor neurons. (5) If the saccade were inaccurate, an error signal from the feedback controller adjusted the weights in the CMAC. It was proposed that connection (2) corresponds to the mossy fiber projection from superior colliculus to oculomotor vermis via the nucleus reticularis tegmenti pontis, and connection (5) to the climbing fiber projection from superior colliculus to the oculomotor vermis via the inferior olive. Plausible initialization values were chosen so that the system produced hypometric saccades (as do human infants) at the start of learning, and position-dependent hypermetric saccades when the cerebellum was removed. Simulations for horizontal eye movements showed that accurate saccades from any starting position could be learned rapidly, even if the error signal conveyed only whether the initial saccade were too large or too small. In subsequent tests the model adapted realistically both to simulated weakening of the eye muscles, and to intrasaccadic displacement of the target, thereby mimicking saccadic plasticity in adults. The architecture of the model may therefore offer a functional explanation of hitherto mysterious tectocerebellar projections, and a framework for investigating in greater detail how the cerebellum adaptively controls saccadic accuracy.

Nonlinear coupling of neural activity and CBF in rodent barrel cortex

The relationship between neural activity and accompanying changes in cerebral blood flow (CBF) and oxygenation must be fully understood before data from brain imaging techniques can be correctly interpreted. Whether signals in fMRI reflect the neural input or output of an activated region is still unclear. Similarly, quantitative relationships between neural activity and changes in CBF are not well understood. The present study addresses these issues by using simultaneous laser Doppler flowmetry (LDF) to measure CBF and multichannel electrophysiology to record neural activity in the form of field potentials and multiunit spiking. We demonstrate that CBF-activation coupling is a nonlinear inverse sigmoid function. Comparing the data with previous work suggests that within a cortical model, CBF shows greatest spatial correlation with a current sink 500 microm below the surface corresponding to sensory input. These results show that care must be exercised when interpreting imaging data elicited by particularly strong or weak stimuli and that hemodynamic changes may better reflect the input to a region rather than its spiking output.