Adria E. N. Hoover

Sensory compensation in sound localization in people with one eye

Some blind people are better at locating sounds than people with normal vision indicating cross-modal plasticity. People who have lost one eye have a unique form of visual deprivation that reduces visual afferent signals by half and can potentially also lead to cross-modal (as well as intra-modal) plasticity. To look for evidence of auditory-visual cross-modal compensation, we measured binaural and monaural sound localization in one-eyed people and compared them with normally sighted controls. One-eyed people showed significantly better binaural sound localization than controls in the central region of space (±78° from straight ahead), but they mislocalized sounds in the far periphery (on both the blind and intact side) by up to 15° towards the centre. One-eyed people showed significantly better monaural sound localization compared with controls. Controls’ performance became asymmetric when they had one eye patched. Patching improved accuracy in the viewing field but decreased accuracy in the occluded field. These results are discussed in terms of cross-modal sensory compensation and the possible contribution of visual depth to interpreting sound localization cues.

How our body influences our perception of the world

Incorporating the fact that the senses are embodied is necessary for an organism to interpret sensory information. Before a unified perception of the world can be formed, sensory signals must be processed with reference to body representation. The various attributes of the body such as shape, proportion, posture, and movement can be both derived from the various sensory systems and can affect perception of the world (including the body itself). In this review we examine the relationships between sensory and motor information, body representations, and perceptions of the world and the body. We provide several examples of how the body affects perception (including but not limited to body perception). First we show that body orientation effects visual distance perception and object orientation. Also, visual-auditory crossmodal-correspondences depend on the orientation of the body: audio “high” frequencies correspond to a visual “up” defined by both gravity and body coordinates. Next, we show that perceived locations of touch is affected by the orientation of the head and eyes on the body, suggesting a visual component to coding body locations. Additionally, the reference-frame used for coding touch locations seems to depend on whether gaze is static or moved relative to the body during the tactile task. The perceived attributes of the body such as body size, affect tactile perception even at the level of detection thresholds and two-point discrimination. Next, long-range tactile masking provides clues to the posture of the body in a canonical body schema. Finally, ownership of seen body parts depends on the orientation and perspective of the body part in view. Together, all of these findings demonstrate how sensory and motor information, body representations, and perceptions (of the body and the world) are interdependent.

The role of the viewpoint on body ownership

People are more sensitive to detecting asynchrony between a self-generated movement of the hand and delayed visual feedback when what they see matches the expected “self” perspective rather than an “other” perspective (Hoover and Harris in Exp Brain Res 222:389–397, 2012). We take this as corresponding to the ability to distinguish self from others and call it the “self-advantage”: a measure of body ownership. What about views of the body that cannot be seen directly? Here, we assessed the effect of familiarity of the view of the body on the self-advantage. Participants performed self-generated hand and head movements viewed directly, in a mirror, and from behind with a variable delay added to the visual feedback. Each view was shown either in the natural perspective or flipped about the vertical or horizontal axes to provide a view from another perspective. Thresholds for detecting a delay in visual feedback were calculated. Dependency of the self-advantage on perspective was most evident for views of the body that are seen most often. Results support the importance of correlating visual feedback with movement information in creating the sense of body ownership.

Superior voice recognition in a patient with acquired prosopagnosia and object agnosia

Anecdotally, it has been reported that individuals with acquired prosopagnosia compensate for their inability to recognize faces by using other person identity cues such as hair, gait or the voice. Are they therefore superior at the use of non-face cues, specifically voices, to person identity? Here, we empirically measure person and object identity recognition in a patient with acquired prosopagnosia and object agnosia. We quantify person identity (face and voice) and object identity (car and horn) recognition for visual, auditory, and bimodal (visual and auditory) stimuli. The patient is unable to recognize faces or cars, consistent with his prosopagnosia and object agnosia, respectively. He is perfectly able to recognize peoples voices and car horns and bimodal stimuli. These data show a reverse shift in the typical weighting of visual over auditory information for audiovisual stimuli in a compromised visual recognition system. Moreover, the patient shows selectively superior voice recognition compared to the controls revealing that two different stimulus domains, persons and objects, may not be equally affected by sensory adaptation effects. This also implies that person and object identity recognition are processed in separate pathways. These data demonstrate that an individual with acquired prosopagnosia and object agnosia can compensate for the visual impairment and become quite skilled at using spared aspects of sensory processing. In the case of acquired prosopagnosia it is advantageous to develop a superior use of voices for person identity recognition in everyday life.

Short and long-term visual deprivation leads to adapted use of audiovisual information for face-voice recognition

Person identification is essential for everyday social interactions. We quickly identify people from cues such as a persons face or the sound of their voice. A change in sensory input, such as losing ones vision, can alter how one uses sensory information. We asked how people with only one eye, who have had reduced visual input during postnatal maturation of the visual system, use faces and voices for person identity recognition. We used an old/new paradigm to investigate unimodal (visual or auditory) and bimodal (audiovisual) identity recognition of people (face, voice and face-voice) and a control category, objects (car, horn and car-horn). Participants learned the identity of 10 pairs of faces and voices (Experiment 1) and 10 cars and horns (Experiment 2) and were asked to identify the learned face/voice or car/horn among 20 distractors. People with one eye were more sensitive to voice identification compared to controls viewing binocularly or with an eye-patch. However, both people with one eye and eye-patched viewing controls use combined audiovisual information for person identification more equally than binocular viewing controls, who favour vision. People with one eye were no different from controls at object identification. The observed visual dominance for binocular controls is larger for person compared to object identification, indicating that faces (vision) play a larger role in person identification and that person identity processing is unique from that for objects. People with long-term visual deprivation from the loss of one eye may have adaptive strategies, such as placing less reliance on vision to achieve intact performance, particularly for face processing.

Greater sensitivity in detecting cross-modal asynchrony for body parts that are seen most often

We have previously shown that people are more sensitive at detecting asynchrony between a self-generated movement and delayed visual feedback when the perspective of the movement matches the ‘natural view’ suggesting an internal, visual, canonical body representation (Hoover and Harris, 2011). Is there a similar variation in sensitivity for parts of the body that cannot be seen in a first-person perspective? To test this, participants made movements with their hands and head (viewing their face or the back of their head) under four viewing conditions: (1) the natural (or direct) view, (2) mirror-reversed, (3) inverted, and (4) inverted and mirror-reversed. Participants indicated which of two periods (one with a minimum delay, the other with an added delay of 33–264 ms) was delayed and their sensitivity to delay was calculated. A significant linear trend was found when comparing sensitivity to detect cross-modal asynchrony in the ‘natural’ or ‘direct’ view condition across body parts; where sensitivity was greatest when viewing body parts seen most often (hands), intermediary for viewing body parts that are seen only indirectly (moving head while viewing face), and least for viewing body parts that are never seen at all (moving head while viewing back of the head). Further, dependency on viewpoint was most evident for body parts that are seen most often or indirectly, but not for body parts that are never seen. Results are discussed in terms of a visual representation of the body.