Monica Gori

Young Children Do Not Integrate Visual and Haptic Form Information

Several studies have shown that adults integrate visual and haptic information (and information from other modalities) in a statistically optimal fashion, weighting each sense according to its reliability [1, 2]. When does this capacity for crossmodal integration develop? Here, we show that prior to 8 years of age, integration of visual and haptic spatial information is far from optimal, with either vision or touch dominating totally, even in conditions in which the dominant sense is far less precise than the other (assessed by discrimination thresholds). For size discrimination, haptic information dominates in determining both perceived size and discrimination thresholds, whereas for orientation discrimination, vision dominates. By 8-10 years, the integration becomes statistically optimal, like adults. We suggest that during development, perceptual systems require constant recalibration, for which cross-sensory comparison is important. Using one sense to calibrate the other precludes useful combination of the two sources.

Impairment of auditory spatial localization in congenitally blind human subjects

Several studies have demonstrated enhanced auditory processing in the blind, suggesting that they compensate their visual impairment in part with greater sensitivity of the other senses. However, several physiological studies show that early visual deprivation can impact negatively on auditory spatial localization. Here we report for the first time severely impaired auditory localization in the congenitally blind: thresholds for spatially bisecting three consecutive, spatially-distributed sound sources were seriously compromised, on average 4.2-fold typical thresholds, and half performing at random. In agreement with previous studies, these subjects showed no deficits on simpler auditory spatial tasks or with auditory temporal bisection, suggesting that the encoding of Euclidean auditory relationships is specifically compromised in the congenitally blind. It points to the importance of visual experience in the construction and calibration of auditory spatial maps, with implications for rehabilitation strategies for the congenitally blind.

Poor Haptic Orientation Discrimination in Nonsighted Children May Reflect Disruption of Cross-Sensory Calibration

A long-standing question, going back at least 300 years to Berkeleys famous essay, is how sensory systems become calibrated with physical reality. We recently showed [1] that children younger than 8-10 years do not integrate visual and haptic information optimally, but that one or the other sense prevails: touch for size and vision for orientation discrimination. The sensory dominance may reflect crossmodal calibration of vision and touch, where the more accurate sense calibrates the other. This hypothesis leads to a clear prediction: that lack of clear vision at an early age should affect calibration of haptic orientation discrimination. We therefore measured size and orientation haptic discrimination thresholds in 17 congenitally visually impaired children (aged 5-19). Haptic orientation thresholds were greatly impaired compared with age-matched controls, whereas haptic size thresholds were at least as good, and often better. One child with a late-acquired visual impairment stood out with excellent orientation discrimination. The results provide strong support for our crossmodal calibration hypothesis.

Development of Visuo-Auditory Integration in Space and Time

Adults integrate multisensory information optimally (e.g., Ernst and Banks, 2002) while children do not integrate multisensory visual-haptic cues until 8–10 years of age (e.g., Gori et al., 2008). Before that age strong unisensory dominance occurs for size and orientation visual-haptic judgments, possibly reflecting a process of cross-sensory calibration between modalities. It is widely recognized that audition dominates time perception, while vision dominates space perception. Within the framework of the cross-sensory calibration hypothesis, we investigate visual-auditory integration in both space and time with child-friendly spatial and temporal bisection tasks. Unimodal and bimodal (conflictual and not) audio-visual thresholds and PSEs were measured and compared with the Bayesian predictions. In the temporal domain, we found that both in children and adults, audition dominates the bimodal visuo-auditory task both in perceived time and precision thresholds. On the contrary, in the visual-auditory spatial task, children younger than 12 years of age show clear visual dominance (for PSEs), and bimodal thresholds higher than the Bayesian prediction. Only in the adult group did bimodal thresholds become optimal. In agreement with previous studies, our results suggest that also visual-auditory adult-like behavior develops late. We suggest that the visual dominance for space and the auditory dominance for time could reflect a cross-sensory comparison of vision in the spatial visuo-audio task and a cross-sensory comparison of audition in the temporal visuo-audio task.

Parkinson's disease accelerates age-related decline in haptic perception by altering somatosensory integration.

This study investigated how Parkinsons disease alters haptic perception and the underlying mechanisms of somatosensory and sensorimotor integration. Changes in haptic sensitivity and acuity (the abilities to detect and to discriminate between haptic stimuli) due to Parkinsons disease were systematically quantified and contrasted to the performance of healthy older and young adults. Using a robotic force environment, virtual contours of various curvatures were presented. Participants explored these contours with their hands and indicated verbally whether they could detect or discriminate between two contours. To understand what aspects of sensory or sensorimotor integration are altered by ageing and disease, we manipulated the sensorimotor aspect of the task: the robot either guided the hand along the contour or the participant actively moved the hand. Active exploration relies on multimodal sensory and sensorimotor integration, while passive guidance only requires sensory integration of proprioceptive and tactile information. The main findings of the study are as follows: first, a decline in haptic precision can already be observed in adults before the age of 70 years. Parkinsons disease may lead to an additional decrease in haptic sensitivity well beyond the levels typically seen in middle-aged and older adults. Second, the haptic deficit in Parkinsons disease is general in nature. It becomes manifest as a decrease in sensitivity and acuity (i.e. a smaller perceivable range and a diminished ability to discriminate between two perceivable haptic stimuli). Third, thresholds during both active and passive exploration are elevated, but not significantly different from each other. That is, active exploration did not enhance the haptic deficit when compared to passive hand motion. This implies that Parkinsons disease affects early stages of somatosensory integration that ultimately have an impact on processes of sensorimotor integration. Our results suggest that the known motor problems in Parkinsons disease that are generally characterized as a failure of sensorimotor integration may, in fact, have a sensory origin.

Anti-Glass patterns and real motion perception: same or different mechanisms?

A sequence of anti-Glass patterns, composed by dot pairs with opposite luminance polarity, elicits a clear perception of motion in the direction of the white dot of the pair. This effect can be reversed by introducing a delay in the presentation of white dots, suggesting a faster processing of light dots as a cause of the motion signal (M. M. Del Viva, M. Gori, & D. C. Burr, 2006). If this hypothesis is correct, anti-Glass patterns should interact with real motion signals. In this study, we compare the motion induced by these stimuli to test whether they are analyzed by the same motion mechanism. We found that motion induced by anti-Glass patterns annuls real motion, when they are presented simultaneously in the same display and moving in opposite directions. By lowering the contrast of one of them, motion toward the stimulus with higher contrast prevails. We also found sub-threshold summation of motion induced by anti-Glass patterns and real motion, when presented simultaneously and moving in the same direction. These findings indicate that anti-Glass patterns and moving stimuli are processed by the same, contrast-dependent motion mechanism and lend further support to the proposed explanation of the effect.

Enhanced auditory spatial localization in blind echolocators.

Echolocation is the extraordinary ability to represent the external environment by using reflected sound waves from self-generated auditory pulses. Blind human expert echolocators show extremely precise spatial acuity and high accuracy in determining the shape and motion of objects by using echoes. In the current study, we investigated whether or not the use of echolocation would improve the representation of auditory space, which is severely compromised in congenitally blind individuals (Gori et al., 2014). The performance of three blind expert echolocators was compared to that of 6 blind non-echolocators and 11 sighted participants. Two tasks were performed: (1) a space bisection task in which participants judged whether the second of a sequence of three sounds was closer in space to the first or the third sound and (2) a minimum audible angle task in which participants reported which of two sounds presented successively was located more to the right. The blind non-echolocating group showed a severe impairment only in the space bisection task compared to the sighted group. Remarkably, the three blind expert echolocators performed both spatial tasks with similar or even better precision and accuracy than the sighted group. These results suggest that echolocation may improve the general sense of auditory space, most likely through a process of sensory calibration.

Perceived duration of visual and tactile stimuli depends on perceived speed

It is known that the perceived duration of visual stimuli is strongly influenced by speed: faster moving stimuli appear to last longer. To test whether this is a general property of sensory systems we asked participants to reproduce the duration of visual and tactile gratings, and visuo-tactile gratings moving at a variable speed (3.5–15 cm/s) for three different durations (400, 600, and 800 ms). For both modalities, the apparent duration of the stimulus increased strongly with stimulus speed, more so for tactile than for visual stimuli. In addition, visual stimuli were perceived to last approximately 200 ms longer than tactile stimuli. The apparent duration of visuo-tactile stimuli lay between the unimodal estimates, as the Bayesian account predicts, but the bimodal precision of the reproduction did not show the theoretical improvement. A cross-modal speed-matching task revealed that visual stimuli were perceived to move faster than tactile stimuli. To test whether the large difference in the perceived duration of visual and tactile stimuli resulted from the difference in their perceived speed, we repeated the time reproduction task with visual and tactile stimuli matched in apparent speed. This reduced, but did not completely eliminate the difference in apparent duration. These results show that for both vision and touch, perceived duration depends on speed, pointing to common strategies of time perception.

Cross-Sensory Facilitation Reveals Neural Interactions between Visual and Tactile Motion in Humans

Many recent studies show that the human brain integrates information across the different senses and that stimuli of one sensory modality can enhance the perception of other modalities. Here we study the processes that mediate cross-modal facilitation and summation between visual and tactile motion. We find that while summation produced a generic, non-specific improvement of thresholds, probably reflecting higher-order interaction of decision signals, facilitation reveals a strong, direction-specific interaction, which we believe reflects sensory interactions. We measured visual and tactile velocity discrimination thresholds over a wide range of base velocities and conditions. Thresholds for both visual and tactile stimuli showed the characteristic “dipper function,” with the minimum thresholds occurring at a given “pedestal speed.” When visual and tactile coherent stimuli were combined (summation condition) the thresholds for these multisensory stimuli also showed a “dipper function” with the minimum thresholds occurring in a similar range to that for unisensory signals. However, the improvement of multisensory thresholds was weak and not directionally specific, well predicted by the maximum-likelihood estimation model (agreeing with previous research). A different technique (facilitation) did, however, reveal direction-specific enhancement. Adding a non-informative “pedestal” motion stimulus in one sensory modality (vision or touch) selectively lowered thresholds in the other, by the same amount as pedestals in the same modality. Facilitation did not occur for neutral stimuli like sounds (that would also have reduced temporal uncertainty), nor for motion in opposite direction, even in blocked trials where the subjects knew that the motion was in the opposite direction showing that the facilitation was not under subject control. Cross-sensory facilitation is strong evidence for functionally relevant cross-sensory integration at early levels of sensory processing.

Auditory and proprioceptive spatial impairments in blind children and adults.

It is not clear what role visual information plays in the development of space perception. It has previously been shown that in absence of vision, both the ability to judge orientation in the haptic modality and bisect intervals in the auditory modality are severely compromised (Gori, Sandini, Martinoli & Burr, 2010; Gori, Sandini, Martinoli & Burr, 2014). Here we report for the first time also a strong deficit in proprioceptive reproduction and audio distance evaluation in early blind children and adults. Interestingly, the deficit is not present in a small group of adults with acquired visual disability. Our results support the idea that in absence of vision the audio and proprioceptive spatial representations may be delayed or drastically weakened due to the lack of visual calibration over the auditory and haptic modalities during the critical period of development.