Lawrence D. Rosenblum

The McGurk effect in infants

In the McGurk effect, perceptual identification of auditory speech syllables is influenced by simultaneous presentation of discrepant visible speech syllables. This effect has been found in subjects of different ages and with various native language backgrounds. But no McGurk tests have been conducted with prelinguistic infants. In the present series of experiments, 5-month-old English-exposed infants were tested for the McGurk effect. Infants were first gaze-habituated to an audiovisual /va/. Two different dishabituation stimuli were then presented: audio /ba/-visual /va/ (perceived by adults as /va/), and audio /da/-visual /va/ (perceived by adults as /da/). The infants showed generalization from the audiovisual /va/ to the audio /ba/-visual /va/ stimulus but not to the audio /da/-visual /va/ stimulus. Follow-up experiments revealed that these generalization differences were not due to a general preference for the audio /da/-visual /va/ stimulus or to the auditory similarity of /ba/ to /va/ relative to /da/. These results suggest that the infants were visually influenced in the same way as Englishspeaking adults are visually influenced.

An audiovisual test of kinematic primitives for visual speech perception.

Isolated kinematic properties of visible speech can provide information for lip reading. Kinematic facial information is isolated by darkening an actors face and attaching dots to various articulators so that only moving dots can be seen with no facial features present. To test the salience of these images, the authors conducted experiments to determine whether the images could visually influence the perception of discrepant auditory syllables. Results showed that these images can influence auditory speech independently of the participants knowledge of the stimuli. In other experiments, single frozen frames of visible syllables were presented with discrepant auditory syllables to test the salience of static facial features. Although the influence of the kinematic stimuli was perceptual, any influence of the static featural stimuli was likely based on participants misunderstanding or postperceptual response bias.

Hefting for a Maximum Distance Throw: A Smart Perceptual Mechanism*

Objects for throwing to a maximum distance were selected by hefting objects varying in size and weight. Preferred weights increased with size reproducing size-weight illusion scaling between weight and volume. In maximum distance throws, preferred objects were thrown the farthest. Throwing was related to hefting as a smart perceptual mechanism. Two strategies for conveying high kinetic energy to projectiles were investigated by studying the kinematics of hefting light, preferred, and heavy objects. Changes in tendon lengths occurring when objects of varying size were grasped corresponded to changes in stiffness at the wrist. Hefting with preferred objects produced an invariant phase between the wrist and elbow. This result corresponded to an optimal relation at peak kinetic energy for the hefting. A paradigm for the study of perceptual properties was compared to size-weight illusion methodology.

Relative Effectiveness of Three Stimulus Variables for Locating a Moving Sound Source

A study is reported in which it is shown that observers can use at least three types of acoustic variables that indicate reliably when a moving sound source is passing: interaural temporal differences, the Doppler effect, and amplitude change. Each of these variables was presented in isolation and each was successful in indicating when a (simulated) moving sound source passed an observer. These three variables were put into competition (with each indicating that closest passage occurred at a different time) in an effort to determine their relative importance. It was found that amplitude change dominated interaural temporal differences which, in turn, dominated the Doppler effect stimulus variable. The results are discussed in terms of two interpretations. First, it is possible that subjects based their judgements on the potential discriminability of each stimulus variable. However, because the stimuli used involved easily discriminable changes, subjects may instead have based their judgements on the independence of a stimulus variable from different environmental situation conditions. The dominance ordering obtained supports the second interpretation.

Visual influences on auditory pluck and bow judgments

In the McGurk effect, visual information specifying a speaker’s articulatory movements can influence auditory judgments of speech. In the present study, we attempted to find an analogue of the McGurk effect by using nonspeech stimuli—the discrepant audiovisual tokens of plucks and bows on a cello. The results of an initial experiment revealed that subjects’ auditory judgments were influenced significantly by the visual pluck and bow stimuli. However, a second experiment in which speech syllables were used demonstrated that the visual influence on consonants was significantly greater than the visual influence observed for pluck-bow stimuli. This result could be interpreted to suggest that the nonspeech visual influence was not a true McGurk effect. In a third experiment, visual stimuli consisting of the wordspluck andbow were found to have no influence over auditory pluck and bow judgments. This result could suggest that the nonspeech effects found in Experiment 1 were based on the audio and visual information’s having an ostensive lawful relation to the specified event. These results are discussed in terms of motor-theory, ecological, and FLMP approaches to speech perception.

On the time allometry of co-ordinated rhythmic movements*

The focus is the power formulae relating periodic time in terrestrial locomotion and flight to mass and length. The periodic timing of limbs and wings oscillating comfortably in absolute co-ordination is viewed as the characteristic period tau 0 of a system in which the free, undamped oscillatory motion of a point mass m at a distance l from a fixed axis does work against two conservative forces. These forces are in the form of gravity g acting on the point mass and a spring of stiffness k acting at a distance b from the axis. The systems characteristic period can be expressed most simply as: tau 0 = 2 pi [ml2/(mlg + kb2)]1/2. In the biological instantiation of this hybrid mass-spring/simple pendulum system, muscular and other tissues function as the spring that elastically stores and releases mechanical energy. Regular oscillations are brought about and sustained by a muscular driving force that ordinarily is close to resonance. The resultant dynamical regime--basically, raising and lowering a mass at regular intervals with respect to gravity--is referred to as the pendular clocking mode of movement organization. The mode is investigated comfortably at a common period and a fixed phase. In absolute co-ordination, two wrist-pendulum systems can be interpreted physically as a virtual single system. The evidence suggests that the scalings of the periodic times of such systems to mass and to length follow directly from the dynamical properties inherent in the resonance equation of the pendular clocking mode. Recourse to biological constants to rationalize the time scale is unnecessary. Experiments on human wrist-pendular activity and detailed analyses of the mass and length dependencies of the locomotory cycle times of quadrupeds, large birds, small passerines, hummingbirds, and insects are performed with respect to the dynamical properties predicted for systems in the pendular clocking mode. The major conclusion is that all the time scales of terrestrial in locomotory time allometries follow systematically from differences in the length scale and differences in the relation of mass to length.

Speech Perception as a Multimodal Phenomenon

Speech perception is inherently multimodal. Visual speech (lip-reading) information is used by all perceivers and readily integrates with auditory speech. Imaging research suggests that the brain treats auditory and visual speech similarly. These findings have led some researchers to consider that speech perception works by extracting amodal information that takes the same form across modalities. From this perspective, speech integration is a property of the input information itself. Amodal speech information could explain the reported automaticity, immediacy, and completeness of audiovisual speech integration. However, recent findings suggest that speech integration can be influenced by higher cognitive properties such as lexical status and semantic context. Proponents of amodal accounts will need to explain these results.

Dynamics and the orientation of kinematic forms in visual event recognition.

The authors investigated event dynamics as a determinant of the perceptual significance of forms of motion. Patch-light displays were recorded for 9 simple events selected to represent rigid-body dynamics, biodynamics, hydrodynamics, and aerodynamics. Observers described events in a free-response task or by circling properties on a list. Cluster analyses performed on descriptor frequencies reflected the dynamics. Observers discriminated hydro- versus aerodynamic events and animate versus inanimate events. The latter result was confirmed by using a forced-choice task. Dynamical models of the events led us to consider energy flows as a determinant of kinematic properties that allowed animacy to be distinguished. Orientation was manipulated in 3 viewing conditions. Descriptions varied with absolute display orientation rather than the relative orientation of display and observer.

Maintenance tendency in co-ordinated rhythmic movements: relative fluctuations and phase.

Evidence from the oscillatory behavior of fish fins and the crayfish swimmeret system suggests that local rhythmic-pattern generators preserve their characteristic properties over the various locomotory co-ordinations in which they participate. This maintenance tendency, as von Holst termed it, was investigated in an experiment in which human subjects swung, through motions at the wrists, hand-held pendulums of variable mass and length. In the experiment (comprising six sessions over 21 months with the same three subjects) the context for the maintenance tendency was steady-state absolute co-ordination: two rhythmic units oscillating at a single, common period and at a bounded phase relation. The experimental methodology permitted systematic control of (a) the characteristic periods of the individual rhythmic units and (b) the deviations from these periods. Relative fluctuations in periodic timing and amplitude were least when a rhythmic units period in absolute co-ordination approximated its characteristic period and increased with departures from the characteristic period. Rates of increase in timing fluctuations were approximately the same for deviations on either side of the characteristic period; the rate of increase in spacing fluctuations was substantially greater for the range in which periods were less than the characteristic period. The phase relation between two co-ordinated rhythmic movement units in absolute co-ordination depended on the difference between their characteristic periods. The intended phase relation of 180 degrees was attained only when the characteristic periods were identical. When the characteristic periods differed, the departure from 180 degrees increased systematically with the difference. The fluctuation results are discussed in terms of the relation between relaxation and harmonic dynamics in producing rhythmic movements, with particular emphasis on the harmonic tuning of relaxation oscillations. The phase results are discussed in terms of whether or not the very many stable phase relations in absolute co-ordination are reflective of the nervous system or of differences in response latencies in left and right muscle systems induced by different degrees of inertial compensation.

Task Dynamics and Resource Dynamics in the Assembly of a Coordinated Rhythmic Activity

Task dynamics corresponding to rhythmic movements emerge from interactions among dynamical resources composed of the musculature, the link segments, and the nervous and circulatory systems. This article investigated whether perturbations of interlimb coordination might be effect over circulatory and nervous elements. Stiffness of wrist-pendulums oscillated at a common tempo and at 180 degrees relative phase was perturbed through the use of tonic activity about an ankle. Left and right stiffnesses, the common period, and the phase relation all changed. Stiffnesses increased with ankle torque in proportion to the wrists inertial load. Despite different changes in stiffness at the two wrists, isochrony was preserved. The stability was shown to be consistent with the proportionality of changes in stiffness to the inertial loads. The phase departed from antiphase in proportion to the asymmetry of inertial loads. The size of departures decreased with increasing ankle torque. An account was developed in terms of muscular, circulatory, and nervous functions.