Anne S. Warlaumont

A Social Feedback Loop for Speech Development and Its Reduction in Autism

We analyzed the microstructure of child-adult interaction during naturalistic, daylong, automatically labeled audio recordings (13,836 hr total) of children (8- to 48-month-olds) with and without autism. We found that an adult was more likely to respond when the child’s vocalization was speech related rather than not speech related. In turn, a child’s vocalization was more likely to be speech related if the child’s previous speech-related vocalization had received an immediate adult response rather than no response. Taken together, these results are consistent with the idea that there is a social feedback loop between child and caregiver that promotes speech development. Although this feedback loop applies in both typical development and autism, children with autism produced proportionally fewer speech-related vocalizations, and the responses they received were less contingent on whether their vocalizations were speech related. We argue that such differences will diminish the strength of the social feedback loop and have cascading effects on speech development over time. Differences related to socioeconomic status are also reported.

NOMINAL CROSS RECURRENCE AS A GENERALIZED LAG SEQUENTIAL ANALYSIS FOR BEHAVIORAL STREAMS

We briefly present lag sequential analysis for behavioral streams, a commonly used method in psychology for quantifying the relationships between two nominal time series. Cross recurrence quantification analysis (CRQA) is shown as an extension of this technique, and we exemplify this nominal application of CRQA to eye-movement data in human interaction. In addition, we demonstrate nominal CRQA in a simple coupled logistic map simulation used in previous communication research, permitting the investigation of properties of nonlinear systems such as bifurcation and onset to chaos, even in the streams obtained by coarse-graining a coupled nonlinear model. We end with a summary of the importance of CRQA for exploring the relationship between two behavioral streams, and review a recent theoretical trend in the cognitive sciences that would be usefully informed by this and similar nonlinear methods. We hope this work will encourage scientists interested in general properties of complex, nonlinear dynamical systems to apply emerging methods to coarse-grained, nominal units of measure, as there is an immediate need for their application in the psychological domain.

Functional flexibility of infant vocalization and the emergence of language

We report on the emergence of functional flexibility in vocalizations of human infants. This vastly underappreciated capability becomes apparent when prelinguistic vocalizations express a full range of emotional content—positive, neutral, and negative. The data show that at least three types of infant vocalizations (squeals, vowel-like sounds, and growls) occur with this full range of expression by 3–4 mo of age. In contrast, infant cry and laughter, which are species-specific signals apparently homologous to vocal calls in other primates, show functional stability, with cry overwhelmingly expressing negative and laughter positive emotional states. Functional flexibility is a sine qua non in spoken language, because all words or sentences can be produced as expressions of varying emotional states and because learning conventional “meanings” requires the ability to produce sounds that are free of any predetermined function. Functional flexibility is a defining characteristic of language, and empirically it appears before syntax, word learning, and even earlier-developing features presumed to be critical to language (e.g., joint attention, syllable imitation, and canonical babbling). The appearance of functional flexibility early in the first year of human life is a critical step in the development of vocal language and may have been a critical step in the evolution of human language, preceding protosyntax and even primitive single words. Such flexible affect expression of vocalizations has not yet been reported for any nonhuman primate but if found to occur would suggest deep roots for functional flexibility of vocalization in our primate heritage.

Effects of Parental Interaction on Infant Vocalization Rate, Variability and Vocal Type

Examination of infant vocalization patterns across interactive and noninteractive contexts may facilitate better understanding of early communication development. In the current study, with 24 infant-parent dyads, infant volubility increased significantly when parent interaction ceased (presenting a “still face,” or SF) after a period of normal interaction (“face-to-face,” or FF). Infant volubility continued at the higher rate than in FF when the parent re-engaged (“reunion,” or RE). Additionally, during SF, the variability in volubility across infants decreased, suggesting the infants adopted relatively similar rates of vocalization to re-engage the parent. The pattern of increasing volubility in SF was seen across all of the most common speech-like vocal types of the first half-year of life (e.g., full vowels, quasivowels, squeals, growls). Parent and infant volubility levels were not significantly correlated. The findings suggest that by six months of age infants have learned that their vocalizations have social value and that changes in volubility can affect parental engagement.

Prespeech motor learning in a neural network using reinforcement

Vocal motor development in infancy provides a crucial foundation for language development. Some significant early accomplishments include learning to control the process of phonation (the production of sound at the larynx) and learning to produce the sounds of ones language. Previous work has shown that social reinforcement shapes the kinds of vocalizations infants produce. We present a neural network model that provides an account of how vocal learning may be guided by reinforcement. The model consists of a self-organizing map that outputs to muscles of a realistic vocalization synthesizer. Vocalizations are spontaneously produced by the network. If a vocalization meets certain acoustic criteria, it is reinforced, and the weights are updated to make similar muscle activations increasingly likely to recur. We ran simulations of the model under various reinforcement criteria and tested the types of vocalizations it produced after learning in the different conditions. When reinforcement was contingent on the production of phonated (i.e. voiced) sounds, the networks post-learning productions were almost always phonated, whereas when reinforcement was not contingent on phonation, the networks post-learning productions were almost always not phonated. When reinforcement was contingent on both phonation and proximity to English vowels as opposed to Korean vowels, the models post-learning productions were more likely to resemble the English vowels and vice versa.

Using nonlinear methods to quantify changes in infant limb movements and vocalizations.

The pairing of dynamical systems theory and complexity science brings novel concepts and methods to the study of infant motor development. Accordingly, this longitudinal case study presents a new approach to characterizing the dynamics of infant limb and vocalization behaviors. A single infants vocalizations and limb movements were recorded from 51-days to 305-days of age. On each recording day, accelerometers were placed on all four of the infants limbs and an audio recorder was worn on the childs chest. Using nonlinear time series analysis methods, such as recurrence quantification analysis and Allan factor, we quantified changes in the stability and multiscale properties of the infants behaviors across age as well as how these dynamics relate across modalities and effectors. We observed that particular changes in these dynamics preceded or coincided with the onset of various developmental milestones. For example, the largest changes in vocalization dynamics preceded the onset of canonical babbling. The results show that nonlinear analyses can help to understand the functional co-development of different aspects of infant behavior.

Influence of musical groove on postural sway.

Timescales of postural fluctuation reflect underlying neuromuscular processes in balance control that are influenced by sensory information and the performance of concurrent cognitive and motor tasks. An open question is how postural fluctuations entrain to complex environmental rhythms, such as in music, which also vary on multiple timescales. Musical groove describes the property of music that encourages auditory-motor synchronization and is used to study voluntary motor entrainment to rhythmic sounds. The influence of groove on balance control mechanisms remains unexplored. We recorded fluctuations in center of pressure (CoP) of standing participants (N = 40) listening to low and high groove music and during quiet stance. We found an effect of musical groove on radial sway variability, with the least amount of variability in the high groove condition. In addition, we observed that groove influenced postural sway entrainment at various temporal scales. For example, with increasing levels of groove, we observed more entrainment to shorter, local timescale rhythmic musical occurrences. In contrast, we observed more entrainment to longer, global timescale features of the music, such as periodicity, with decreasing levels of groove. Finally, musical experience influenced the amount of postural variability and entrainment at local and global timescales. We conclude that groove in music and musical experience can influence the neural mechanisms that govern balance control, and discuss implications of our findings in terms of multiscale sensorimotor coupling. (PsycINFO Database Record

Learning to Produce Syllabic Speech Sounds via Reward-Modulated Neural Plasticity

At around 7 months of age, human infants begin to reliably produce well-formed syllables containing both consonants and vowels, a behavior called canonical babbling. Over subsequent months, the frequency of canonical babbling continues to increase. How the infant’s nervous system supports the acquisition of this ability is unknown. Here we present a computational model that combines a spiking neural network, reinforcement-modulated spike-timing-dependent plasticity, and a human-like vocal tract to simulate the acquisition of canonical babbling. Like human infants, the model’s frequency of canonical babbling gradually increases. The model is rewarded when it produces a sound that is more auditorily salient than sounds it has previously produced. This is consistent with data from human infants indicating that contingent adult responses shape infant behavior and with data from deaf and tracheostomized infants indicating that hearing, including hearing one’s own vocalizations, is critical for canonical babbling development. Reward receipt increases the level of dopamine in the neural network. The neural network contains a reservoir with recurrent connections and two motor neuron groups, one agonist and one antagonist, which control the masseter and orbicularis oris muscles, promoting or inhibiting mouth closure. The model learns to increase the number of salient, syllabic sounds it produces by adjusting the base level of muscle activation and increasing their range of activity. Our results support the possibility that through dopamine-modulated spike-timing-dependent plasticity, the motor cortex learns to harness its natural oscillations in activity in order to produce syllabic sounds. It thus suggests that learning to produce rhythmic mouth movements for speech production may be supported by general cortical learning mechanisms. The model makes several testable predictions and has implications for our understanding not only of how syllabic vocalizations develop in infancy but also for our understanding of how they may have evolved.

Multiple Coordination Patterns in Infant and Adult Vocalizations

The study of vocal coordination between infants and adults has led to important insights into the development of social, cognitive, emotional and linguistic abilities. We used an automatic system to identify vocalizations produced by infants and adults over the course of the day for fifteen infants studied longitudinally during the first two years of life. We measured three different types of vocal coordination: coincidence-based, rate-based, and cluster-based. Coincidence-based and rate-based coordination are established measures in the developmental literature. Cluster-based coordination is new and measures the strength of matching in the degree to which vocalization events occur in hierarchically nested clusters. We investigated whether various coordination patterns differ as a function of vocalization type, whether different coordination patterns provide unique information about the dynamics of vocal interaction, and how the various coordination patterns each relate to infant age. All vocal coordination patterns displayed greater coordination for infant speech-related vocalizations, adults adapted the hierarchical clustering of their vocalizations to match that of infants, and each of the three coordination patterns had unique associations with infant age. Altogether, our results indicate that vocal coordination between infants and adults is multifaceted, suggesting a complex relationship between vocal coordination and the development of vocal communication.

Production and Convergence of Multiscale Clustering in Speech

Language entails the scaling of variability across levels of measurement—small linguistic variations occur at the millisecond level, larger variations occur at the next level, and even larger variations occur over longer timescales. For acoustic onsets in speech signals, small temporal variations occur at the phonetic level, larger variations occur at the phrasal level, and even larger variations occur at the conversational level. Scaling across levels of measurement can be quantified in terms of power law distributions. In this article we review recent investigations into power law clustering of acoustic speech onsets. Studies demonstrate that the multiscale clustering in onsets reflects communicative aspects of speech in adult conversations as well as infant vocalizations. We also review evidence that multiscale clustering in the vocalizations of individuals converges during vocal interactions. We relate multiscale convergence to the notion of complexity matching, that is, the hypothesis that maximal information transfer occurs when the power laws of 2 interacting complex systems are matched. We conclude by discussing potential extensions of this work including estimating the multifractal structure of speech and testing the maximal information transfer prediction of complexity matching.