Vincenzo Maffei

Simulated self-motion in a visual gravity field: sensitivity to vertical and horizontal heading in the human brain.

Multiple visual signals are relevant to perception of heading direction. While the role of optic flow and depth cues has been studied extensively, little is known about the visual effects of gravity on heading perception. We used fMRI to investigate the contribution of gravity-related visual cues on the processing of vertical versus horizontal apparent self-motion. Participants experienced virtual roller-coaster rides in different scenarios, at constant speed or 1g-acceleration/deceleration. Imaging results showed that vertical self-motion coherent with gravity engaged the posterior insula and other brain regions that have been previously associated with vertical object motion under gravity. This selective pattern of activation was also found in a second experiment that included rectilinear motion in tunnels, whose direction was cued by the preceding open-air curves only. We argue that the posterior insula might perform high-order computations on visual motion patterns, combining different sensory cues and prior information about the effects of gravity. Medial-temporal regions including para-hippocampus and hippocampus were more activated by horizontal motion, preferably at constant speed, consistent with a role in inertial navigation. Overall, the results suggest partially distinct neural representations of the cardinal axes of self-motion (horizontal and vertical).

Visual gravitational motion and the vestibular system in humans

The visual system is poorly sensitive to arbitrary accelerations, but accurately detects the effects of gravity on a target motion. Here we review behavioral and neuroimaging data about the neural mechanisms for dealing with object motion and egomotion under gravity. The results from several experiments show that the visual estimates of a target motion under gravity depend on the combination of a prior of gravity effects with on-line visual signals on target position and velocity. These estimates are affected by vestibular inputs, and are encoded in a visual-vestibular network whose core regions lie within or around the Sylvian fissure, and are represented by the posterior insula/retroinsula/temporo-parietal junction. This network responds both to target motions coherent with gravity and to vestibular caloric stimulation in human fMRI studies. Transient inactivation of the temporo-parietal junction selectively disrupts the interception of targets accelerated by gravity.

Processing of targets in smooth or apparent motion along the vertical in the human brain: an fMRI study

Neural substrates for processing constant speed visual motion have been extensively studied. Less is known about the brain activity patterns when the target speed changes continuously, for instance under the influence of gravity. Using functional MRI (fMRI), here we compared brain responses to accelerating/decelerating targets with the responses to constant speed targets. The target could move along the vertical under gravity (1g), under reversed gravity (-1g), or at constant speed (0g). In the first experiment, subjects observed targets moving in smooth motion and responded to a GO signal delivered at a random time after target arrival. As expected, we found that the timing of the motor responses did not depend significantly on the specific motion law. Therefore brain activity in the contrast between different motion laws was not related to motor timing responses. Average BOLD signals were significantly greater for 1g targets than either 0g or -1g targets in a distributed network including bilateral insulae, left lingual gyrus, and brain stem. Moreover, in these regions, the mean activity decreased monotonically from 1g to 0g and to -1g. In the second experiment, subjects intercepted 1g, 0g, and -1g targets either in smooth motion (RM) or in long-range apparent motion (LAM). We found that the sites in the right insula and left lingual gyrus, which were selectively engaged by 1g targets in the first experiment, were also significantly more active during 1g trials than during -1g trials both in RM and LAM. The activity in 0g trials was again intermediate between that in 1g trials and that in -1g trials. Therefore in these regions the global activity modulation with the law of vertical motion appears to hold for both RM and LAM. Instead, a region in the inferior parietal lobule showed a preference for visual gravitational motion only in LAM but not RM.

Extrapolation of vertical target motion through a brief visual occlusion

It is known that arbitrary target accelerations along the horizontal generally are extrapolated much less accurately than target speed through a visual occlusion. The extent to which vertical accelerations can be extrapolated through an occlusion is much less understood. Here, we presented a virtual target rapidly descending on a blank screen with different motion laws. The target accelerated under gravity (1g), decelerated under reversed gravity (−1g), or moved at constant speed (0g). Probability of each type of acceleration differed across experiments: one acceleration at a time, or two to three different accelerations randomly intermingled could be presented. After a given viewing period, the target disappeared for a brief, variable period until arrival (occluded trials) or it remained visible throughout (visible trials). Subjects were asked to press a button when the target arrived at destination. We found that, in visible trials, the average performance with 1g targets could be better or worse than that with 0g targets depending on the acceleration probability, and both were always superior to the performance with −1g targets. By contrast, the average performance with 1g targets was always superior to that with 0g and −1g targets in occluded trials. Moreover, the response times of 1g trials tended to approach the ideal value with practice in occluded protocols. To gain insight into the mechanisms of extrapolation, we modeled the response timing based on different types of threshold models. We found that occlusion was accompanied by an adaptation of model parameters (threshold time and central processing time) in a direction that suggests a strategy oriented to the interception of 1g targets at the expense of the interception of the other types of tested targets. We argue that the prediction of occluded vertical motion may incorporate an expectation of gravity effects.

Visual gravity cues in the interpretation of biological movements: neural correlates in humans

Our visual system takes into account the effects of Earth gravity to interpret biological motion (BM), but the neural substrates of this process remain unclear. Here we measured functional magnetic resonance (fMRI) signals while participants viewed intact or scrambled stick-figure animations of walking, running, hopping, and skipping recorded at normal or reduced gravity. We found that regions sensitive to BM configuration in the occipito-temporal cortex (OTC) were more active for reduced than normal gravity but with intact stimuli only. Effective connectivity analysis suggests that predictive coding of gravity effects underlies BM interpretation. This process might be implemented by a family of snapshot neurons involved in action monitoring.

Filling gaps in visual motion for target capture.

A remarkable challenge our brain must face constantly when interacting with the environment is represented by ambiguous and, at times, even missing sensory information. This is particularly compelling for visual information, being the main sensory system we rely upon to gather cues about the external world. It is not uncommon, for example, that objects catching our attention may disappear temporarily from view, occluded by visual obstacles in the foreground. Nevertheless, we are often able to keep our gaze on them throughout the occlusion or even catch them on the fly in the face of the transient lack of visual motion information. This implies that the brain can fill the gaps of missing sensory information by extrapolating the object motion through the occlusion. In recent years, much experimental evidence has been accumulated that both perceptual and motor processes exploit visual motion extrapolation mechanisms. Moreover, neurophysiological and neuroimaging studies have identified brain regions potentially involved in the predictive representation of the occluded target motion. Within this framework, ocular pursuit and manual interceptive behavior have proven to be useful experimental models for investigating visual extrapolation mechanisms. Studies in these fields have pointed out that visual motion extrapolation processes depend on manifold information related to short-term memory representations of the target motion before the occlusion, as well as to longer term representations derived from previous experience with the environment. We will review recent oculomotor and manual interception literature to provide up-to-date views on the neurophysiological underpinnings of visual motion extrapolation.

Gravity in the Brain as a Reference for Space and Time Perception

Moving and interacting with the environment require a reference for orientation and a scale for calibration in space and time. There is a wide variety of environmental clues and calibrated frames at different locales, but the reference of gravity is ubiquitous on Earth. The pull of gravity on static objects provides a plummet which, together with the horizontal plane, defines a three-dimensional Cartesian frame for visual images. On the other hand, the gravitational acceleration of falling objects can provide a time-stamp on events, because the motion duration of an object accelerated by gravity over a given path is fixed. Indeed, since ancient times, man has been using plumb bobs for spatial surveying, and water clocks or pendulum clocks for time keeping. Here we review behavioral evidence in favor of the hypothesis that the brain is endowed with mechanisms that exploit the presence of gravity to estimate the spatial orientation and the passage of time. Several visual and non-visual (vestibular, haptic, visceral) cues are merged to estimate the orientation of the visual vertical. However, the relative weight of each cue is not fixed, but depends on the specific task. Next, we show that an internal model of the effects of gravity is combined with multisensory signals to time the interception of falling objects, to time the passage through spatial landmarks during virtual navigation, to assess the duration of a gravitational motion, and to judge the naturalness of periodic motion under gravity.

Multisensory Integration and Internal Models for Sensing Gravity Effects in Primates

Gravity is crucial for spatial perception, postural equilibrium, and movement generation. The vestibular apparatus is the main sensory system involved in monitoring gravity. Hair cells in the vestibular maculae respond to gravitoinertial forces, but they cannot distinguish between linear accelerations and changes of head orientation relative to gravity. The brain deals with this sensory ambiguity (which can cause some lethal airplane accidents) by combining several cues with the otolith signals: angular velocity signals provided by the semicircular canals, proprioceptive signals from muscles and tendons, visceral signals related to gravity, and visual signals. In particular, vision provides both static and dynamic signals about body orientation relative to the vertical, but it poorly discriminates arbitrary accelerations of moving objects. However, we are able to visually detect the specific acceleration of gravity since early infancy. This ability depends on the fact that gravity effects are stored in brain regions which integrate visual, vestibular, and neck proprioceptive signals and combine this information with an internal model of gravity effects.

Processing of visual gravitational motion in the peri-sylvian cortex: Evidence from brain-damaged patients.

Rich behavioral evidence indicates that the brain estimates the visual direction and acceleration of gravity quite accurately, and the underlying mechanisms have begun to be unraveled. While the neuroanatomical substrates of gravity direction processing have been studied extensively in brain-damaged patients, to our knowledge no such study exists for the processing of visual gravitational motion. Here we asked 31 stroke patients to intercept a virtual ball moving along the vertical under either natural gravity or artificial reversed gravity. Twenty-seven of them also aligned a luminous bar to the vertical direction (subjective visual vertical, SVV). Using voxel-based lesion-symptom mapping as well as lesion subtraction analysis, we found that lesions mainly centered on the posterior insula are associated with greater deviations of SVV, consistent with several previous studies. Instead, lesions mainly centered on the parietal operculum decrease the ability to discriminate natural from unnatural gravitational acceleration with a timed motor response in the interception task. Both the posterior insula and the parietal operculum belong to the vestibular cortex, and presumably receive multisensory information about the gravity vector. We speculate that an internal model estimating the effects of gravity on visual objects is constructed by transforming the vestibular estimates of mechanical gravity, which are computed in the brainstem and cerebellum, into internalized estimates of virtual gravity, which are stored in the cortical vestibular network. The present lesion data suggest a specific role for the parietal operculum in detecting the mismatch between predictive signals from the internal model and the online visual signals.

Unfamiliar Walking Movements Are Detected Early in the Visual Stream: An fMRI Study

Two experiments investigated the network involved in the visual perception of walking. Video clips of forward and backward walk (real walk direction) were shown either as recorded, or reversed in time (rendering). In Experiment 1 (identification task), participants were asked to indicate whether or not the stimulus was time-reversed. In Experiment 2 (free-viewing), participants viewed the video clips passively. Identification accuracy was good with the more familiar scene, that is, when the visual walk was in the direction of the facing orientation, and at chance level in the opposite case. In both experiments, the temporo-occipital junction (TOJ) was activated more strongly by unfamiliar than familiar scenes. Only in Experiment 1 intraparietal, superior temporal, and inferior temporal regions were also activated. TOJ activation signals the detection in unfamiliar scenes of a mismatch between facing orientation and visual movement direction. We argue that TOJ response to a mismatch prevents the further processing of the visual input required to identify temporal inversions. When no mismatch is detected (familiar stimuli), TOJ would, instead, be involved in the kinematic analysis that makes such identification possible. The study demonstrates that unfamiliar walking movements are detected earlier than so far assumed along the visual movement processing stream.