David J. Lewkowicz

Cross-modal equivalence in early infancy: Auditory–visual intensity matching.

It has been proposed that young infants are attentive to quantitative variations in stimulation to the exclusion of qualitative ones. To the extent that this is so, young infants should ignore differences between lights and sounds and should instead respond to auditory and visual stimuli as more or less similar depending on their intensity. To examine this hypothesis, a cardia c habituation/dishabituation method with a test for stimulus generalization was employed . Three-weekold infants were repeatedly presented with white-light followed by white-noise stimuli of different intensities. A U-shaped relationship between magnitude of cardiac response and loudness was found. In view of previous findings that without prior visual stimulation a monotonic increase in cardiac response to the same range of auditory stimuli results, this finding of a significant quadratic relationship with loudness suggests that the infants were responding to the auditory stimuli in terms of their similarity to the previously presented visual stimulus. A separate group of infants presented with a more intense visual stimulus exhibited a shift in the intensity at which a minimal cardiac response occurred. Results of a study with adults did not show any systematic relationship between cardiac response and loudness, indicating that unlike infants, adults do not spontaneously make cross-modal matches of intensity. Our perception of the world is based to a

Infants deploy selective attention to the mouth of a talking face when learning speech

The mechanisms underlying the acquisition of speech-production ability in human infancy are not well understood. We tracked 4–12-mo-old English-learning infants’ and adults’ eye gaze while they watched and listened to a female reciting a monologue either in their native (English) or nonnative (Spanish) language. We found that infants shifted their attention from the eyes to the mouth between 4 and 8 mo of age regardless of language and then began a shift back to the eyes at 12 mo in response to native but not nonnative speech. We posit that the first shift enables infants to gain access to redundant audiovisual speech cues that enable them to learn their native speech forms and that the second shift reflects growing native-language expertise that frees them to shift attention to the eyes to gain access to social cues. On this account, 12-mo-old infants do not shift attention to the eyes when exposed to nonnative speech because increasing native-language expertise and perceptual narrowing make it more difficult to process nonnative speech and require them to continue to access redundant audiovisual cues. Overall, the current findings demonstrate that the development of speech production capacity relies on changes in selective audiovisual attention and that this depends critically on early experience.

The emergence of multisensory systems through perceptual narrowing

According to conventional wisdom, multisensory development is a progressive process that results in the growth and proliferation of perceptual skills. We review new findings indicating that a regressive process - perceptual narrowing - also contributes in critical ways to perceptual development. These new data reveal that young infants are able to integrate non-native faces and vocalizations, that this broad multisensory perceptual tuning is present at birth, and that this tuning narrows by the end of the first year of life, leaving infants with the ability to integrate only socio-ecologically-relevant multisensory signals. This narrowing process forces us to reconsider the traditional progressive theories of multisensory development and opens up several new evolutionary questions as well.

Narrowing of intersensory speech perception in infancy

The conventional view is that perceptual/cognitive development is an incremental process of acquisition. Several striking findings have revealed, however, that the sensitivity to non-native languages, faces, vocalizations, and music that is present early in life declines as infants acquire experience with native perceptual inputs. In the language domain, the decline in sensitivity is reflected in a process of perceptual narrowing that is thought to play a critical role during the acquisition of a native-language phonological system. Here, we provide evidence that such a decline also occurs in infant response to multisensory speech. We found that infant intersensory response to a non-native phonetic contrast narrows between 6 and 11 months of age, suggesting that the perceptual system becomes increasingly more tuned to key native-language audiovisual correspondences. Our findings lend support to the notion that perceptual narrowing is a domain-general as well as a pan-sensory developmental process.

Infant Perception of Audio-Visual Speech Synchrony.

Three experiments investigated perception of audio-visual (A-V) speech synchrony in 4- to 10-month-old infants. Experiments 1 and 2 used a convergent-operations approach by habituating infants to an audiovisually synchronous syllable (Experiment 1) and then testing for detection of increasing degrees of A-V asynchrony (366, 500, and 666 ms) or by habituating infants to a detectably asynchronous syllable (666 ms; Experiment 2) and then testing for detection of decreasing degrees of asynchrony (500, 366, and 0 ms). Following habituation to the synchronous syllable, infants detected only the largest A-V asynchrony (0 ms vs. 666 ms), whereas following habituation to the asynchronous syllable, infants detected the largest asynchrony (666 ms vs. 0 ms) as well as a smaller one (666 ms vs. 366 ms). Experiment 3 investigated the underlying mechanism of A-V asynchrony detection and indicated that responsiveness was based on a sensitivity to stimulus-energy onsets rather than the dynamic correlation between acoustic and visible utterance attributes. These findings demonstrated that infant perception of A-V speech synchrony is subject to the effects of short-term experience and that it is driven by a low-level, domain-general mechanism.

Developmental changes in infants' bisensory response to synchronous durations☆

Abstract The present set of studies was concerned with the development of bisensory response to synchronous durations. Infants viewed pairs of checkerboards where each member of the pair flashed at the same rate but differed in the duration of each flash. Visual preferences were studied in silence as well as in the presence of a tone whose duration and onset/offset characteristics corresponded to one member of the visual pair of stimuli. Results indicated that 3-month-old infants did not make bisensory matches of duration. In contrast, 6- and 8-month-old infants exhibited evidence of bisensory matching in that, in general, their looking at the visual stimulus corresponding in duration to the auditory stimulus was greater than was their looking at the non-corresponding stimulus. Synchrony played a major part in this matching in that when the corresponding auditory and visual stimuli were put out of phase with one another, no evidence of bisensory matching was obtained.

Infants’ responsiveness to the auditory and visual attributes of a sounding/moving stimulus

Responses to unimodal and multimodal attributes of a compouMauxlitoryMsuat stimulus were investigated in 4-, 6-, 8-, and 10-month-old infants. First, infants were habituated to a compound stimulus consisting of a visual stimulus that moved up anddown on a video monitor and a sound that occurred each time the visual stimulus reversed direction at the bottom. Once each infant met a habituation criterion, a series of test trials was administered to assess responsiveness to the components of the compound stimulus. Response was defined as the total duration of visual fixation in each trial. In the two unimodal test trials, the rate at which the component was presented was changed while the rate of the other component remained the same, whereas in the bimodal test trial the rate of both components was changed simultaneously. Results indicated that infants at each age successfully discriminated the bimodal and the two unimodal changes and that regression to the mean did not account for the results. Results also showed that disruption of the temporal relationship that accompanied the change in rate in the two unimodal test trials was also discriminable, but rate changes appeared to play a greater role in responsiveness than did synchrony changes. Considered together with results from similar prior studies, the current results are consistent with the modality appropriateness hypothesis in showing that discrimination of temporal changes in the auditory and visual modalities is dependent on the specialization of the sensory modalities.

Semantic confusion regarding the development of multisensory integration: a practical solution

There is now a good deal of data from neurophysiological studies in animals and behavioral studies in human infants regarding the development of multisensory processing capabilities. Although the conclusions drawn from these different datasets sometimes appear to conflict, many of the differences are due to the use of different terms to mean the same thing and, more problematic, the use of similar terms to mean different things. Semantic issues are pervasive in the field and complicate communication among groups using different methods to study similar issues. Achieving clarity of communication among different investigative groups is essential for each to make full use of the findings of others, and an important step in this direction is to identify areas of semantic confusion. In this way investigators can be encouraged to use terms whose meaning and underlying assumptions are unambiguous because they are commonly accepted. Although this issue is of obvious importance to the large and very rapidly growing number of researchers working on multisensory processes, it is perhaps even more important to the non‐cognoscenti. Those who wish to benefit from the scholarship in this field but are unfamiliar with the issues identified here are most likely to be confused by semantic inconsistencies. The current discussion attempts to document some of the more problematic of these, begin a discussion about the nature of the confusion and suggest some possible solutions.

Heterogeneity and heterochrony in the development of intersensory perception

It is now well established that a variety of intersensory perceptual skills emerge in early human development. Empirical evidence from studies in the authors as well as other laboratories charting the developmental emergence of these abilities is reviewed. The evidence is considered in terms of the currently dominant theoretical view of intersensory development that assigns the detection of amodal invariants a primary and foundational role. It is argued that this view is inadequate because the detection of amodal invariants is only one of three distinct intersensory integration processes. It is noted that the other two processes, namely, intersensory association of modality-specific cues and non-specific effects of stimulation in one modality on responsiveness to stimulation in another modality, are equally important and that the operation of all three and, in particular, the relation between them, must be studied to attain a complete understanding of intersensory perceptual development. It is suggested that the theoretical approach to the development of intersensory perception should be broadened to include all three types of processes and that developmental studies must respect basic facts and principles of development. To this end, a developmental systems approach is proposed that holds that the development of intersensory integration consists of the heterochronous emergence of heterogeneous perceptual skills.

Sound induces perceptual reorganization of an ambiguous motion display in human infants

Adults who watch an ambiguous visual event consisting of two identical objects moving toward, through, and away from each other and hear a brief sound when the objects overlap report seeing visual bouncing. We conducted three experiments in which we used the habituation/test method to determine whether these illusory effects might emerge early in development. In Experiments 1 and 3 we tested 4-, 6- and 8-month-old infants’ discrimination between an ambiguous visual display presented together with a sound synchronized with the objects’ spatial coincidence and the identical visual display presented together with a sound no longer synchronized with coincidence. Consistent with illusory perception, the 6- and 8-month-old, but not the 4-month-old, infants responded to these events as different. In Experiment 2 infants were habituated to the ambiguous visual display together with a sound synchronized with the objects’ coincidence and tested with a physically bouncing object accompanied by the sound at the bounce. Consistent with illusory perception again, infants treated these two events as equivalent by not exhibiting response recovery. The developmental emergence of this intersensory illusion at 6 months of age is hypothesized to reflect developmental changes in object knowledge and attentional mechanisms.