Michael Schwartze

Cortical speech processing unplugged: a timely subcortico-cortical framework

Speech is inherently tied to time. This fundamental quality has long been deemed secondary, and has consequently not received appropriate recognition in speech processing models. We develop an integrative speech processing framework by synthesizing evolutionary, anatomical and neurofunctional concepts of auditory, temporal and speech processing. These processes converge in a network that extends cortical speech processing systems with cortical and subcortical systems associated with motor control. This subcortico-cortical multifunctional network is based on temporal processing and predictive coding of events to optimize interactions between the organism and the environment. The framework we outline provides a novel perspective on speech processing and has implications for future studies on learning, proficient use, and developmental and acquired disorders of speech production and perception.

The impact of basal ganglia lesions on sensorimotor synchronization, spontaneous motor tempo, and the detection of tempo changes

The basal ganglia (BG) are part of extensive subcortico-cortical circuits that are involved in a variety of motor and non-motor cognitive functions. Accumulating evidence suggests that one specific function that engages the BG and associated cortico-striato-thalamo-cortical circuitry is temporal processing, i.e., the mechanisms that underlie the encoding, decoding and evaluation of temporal relations or temporal structure. In the current study we investigated the interplay of two processes that require precise representations of temporal structure, namely the perception of an auditory pacing signal and manual motor production by means of finger tapping in a sensorimotor synchronization task. Patients with focal lesions of the BG and healthy control participants were asked to align finger taps to tone sequences that either did or did not contain a tempo acceleration or tempo deceleration at a predefined position, and to continue tapping at the final tempo after the pacing sequence had ceased. Performance in this adaptive synchronization-continuation paradigm differed between the two groups. Selective damage to the BG affected the abilities to detect tempo changes and to perform attention-dependent error correction, particularly in response to tempo decelerations. An additional assessment of preferred spontaneous, i.e., unpaced but regular, production rates yielded more heterogeneous results in the patient group. Together these findings provide evidence for less efficient processing in the perception and the production of temporal structure in patients with focal BG lesions. The results also support the functional role of the BG system in attention-dependent temporal processing.

Temporal regularity effects on pre-attentive and attentive processing of deviance

Temporal regularity allows predicting the temporal locus of future information thereby potentially facilitating cognitive processing. We applied event-related brain potentials (ERPs) to investigate how temporal regularity impacts pre-attentive and attentive processing of deviance in the auditory modality. Participants listened to sequences of sinusoidal tones differing exclusively in pitch. The inter-stimulus interval (ISI) in these sequences was manipulated to convey either isochronous or random temporal structure. In the pre-attentive session, deviance processing was unaffected by the regularity manipulation as evidenced in three event-related-potentials (ERPs): mismatch negativity (MMN), P3a, and reorienting negativity (RON). In the attentive session, the P3b was smaller for deviant tones embedded in irregular temporal structure, while the N2b component remained unaffected. These findings confirm that temporal regularity can reinforce cognitive mechanisms associated with the attentive processing of deviance. Furthermore, they provide evidence for the dynamic allocation of attention in time and dissociable pre-attentive and attention-dependent temporal processing mechanisms.

Functional dissociation of pre-SMA and SMA-proper in temporal processing.

The ability to assess temporal structure is crucial in order to adapt to an ever-changing environment. Increasing evidence suggests that the supplementary motor area (SMA) is involved in both sensory and sensorimotor processing of temporal structure. However, it is not entirely clear whether the structural differentiation of the SMA translates into functional specialization, and how the SMA relates to other systems that engage in temporal processing, namely the cerebellum and cortico-striatal circuits. Anatomically, the SMA comprises at least two subareas, the rostral pre-SMA and the caudal SMA-proper. Each displays a characteristic pattern of connections to motor and non-motor structures. Crucially, these connections establish a potential hub among cerebellar and cortico-striatal systems, possibly forming a dedicated subcortico-cortical temporal processing network. To further explore the functional role of each SMA subarea, we performed a meta-analysis of functional neuroimaging studies by contrasting activations according to whether they linked with either sensory, sensorimotor, sequential, non-sequential, explicit, non-explicit, subsecond, or suprasecond temporal processing. This procedure yielded a set of functional differences, which mirror the rostro-caudal anatomical dimension. Activations associated with sensory, non-sequential, and suprasecond temporal processing tend to locate to the rostral SMA, while the opposite is true for the caudal SMA. These findings confirm a functional dissociation of pre-SMA and SMA-proper in temporal processing.

A dual-pathway neural architecture for specific temporal prediction

Efficient behavior depends in part on the ability to predict the type and the timing of events in the environment. Specific temporal predictions require an internal representation of the temporal structure of events. Here we propose that temporal prediction recruits adaptive and non-adaptive oscillatory mechanisms involved in establishing such an internal representation. Partial structural and functional convergence of the underlying mechanisms allows speculation about an extended subcortico-cortical network. This network develops around a dual-pathway architecture, which establishes the basis for preparing the organism for perceptual integration, for the generation of specific temporal predictions, and for optimizing the brains allocation of its limited resources. Key to these functions is rapid cerebellar transmission of an adaptively-filtered, event-based representation of temporal structure. Rapid cerebellar transmission engages a pathway comprising connections from early sensory processing stages to the cerebellum and from there to the thalamus, effectively bypassing more central stages of classical sensory pathways.

Temporal aspects of prediction in audition: Cortical and subcortical neural mechanisms

Tracing the temporal structure of acoustic events is crucial in order to efficiently adapt to dynamic changes in the environment. In turn, regularity in temporal structure may facilitate tracing of the acoustic signal and its likely spatial source. However, temporal processing in audition extends beyond a domain-general facilitatory function. Temporal regularity and temporal order of auditory events correspond to contextually extracted, statistically sampled relations among sounds. These relations are the backbone of prediction in audition, determining both when an event is likely to occur (temporal structure) and also what type of event can be expected at a specific point in time (formal structure, e.g. spectral information). Here, we develop a model of temporal processing in audition and speech that involves a division of labor between the cerebellum and the basal ganglia in tracing acoustic events in time. As for the cerebellum and its associated thalamo-cortical connections, we refer to its role in the automatic encoding of event-based temporal structure with high temporal precision, while the basal ganglia-thalamo-cortical system engages in the attention-dependent evaluation of longer-range intervals. Recent electrophysiological and neurofunctional evidence suggests that neocortical processing of spectral structure relies on concurrent extraction of event-based temporal information. We propose that spectrotemporal predictive processes may be facilitated by subcortical coding of relevant changes in sound energy as temporal event markers.

Differential input of the supplementary motor area to a dedicated temporal processing network: functional and clinical implications.

The ability to track the temporal structure of events in a dynamic environment is crucial to cognition and action alike. In order to guide timely reactive and proactive behavior the individual has to draw upon some internal representation of temporal relations or temporal structure. Here an event may be defined as a perceived change in the formal structure of the environment, i.e., the identity (“what”) or the position (“where”) of an object. In turn, the temporal relation between events may be defined as the temporal structure (“when”) of the environment.

Effects of musically cued gait training in Parkinson's disease: beyond a motor benefit

Auditory stimulation via rhythmic cues can be used successfully in the rehabilitation of motor function in patients with motor disorders. A prototypical example is provided by dysfunctional gait in patients with idiopathic Parkinsons disease (PD). Coupling steps to external rhythmic cues (the beat of music or the sounds of a metronome) leads to long‐term motor improvements, such as increased walking speed and greater stride length. These effects are likely to be underpinned by compensatory brain mechanisms involving cerebellar–thalamocortical networks. Because these areas are also involved in perceptual and motor timing, parallel improvement in timing tasks is expected in PD beyond purely motor benefits. In keeping with this idea, we report here recent behavioral data showing beneficial effects of musically cued gait training (MCGT) on gait performance (i.e., increased stride length and speed), perceptual timing (e.g., discriminating stimulus durations), and sensorimotor timing abilities (i.e., in paced tapping tasks) in PD patients. Particular attention is paid to individual differences in timing abilities in PD, thus paving the ground for an individualized MCGT‐based therapy.

Event-related potential responses to metric violations: rules versus meaning

In stress-timed languages, the alternation of stressed and unstressed syllables (or ‘meter’) is an important formal and temporal cue to guide speech processing. Previous electroencephalography studies have shown that metric violations result in an early negative event-related potential. It is unclear whether this ‘metric’ negativity is an N400 elicited by misplaced stress or whether it responds to error detection. The aim of this study was to investigate the nature of the ‘metric’ negativity as a function of rule-based, predictive sequencing. Our results show that the negativity occurs independent of the lexical-semantic content. We therefore suggest that the metric negativity reflects a rule-based sequencing mechanism.

Cerebellum, temporal predictability and the updating of a mental model

We live in a dynamic and changing environment, which necessitates that we adapt to and efficiently respond to changes of stimulus form (‘what’) and stimulus occurrence (‘when’). Consequently, behaviour is optimal when we can anticipate both the ‘what’ and ‘when’ dimensions of a stimulus. For example, to perceive a temporally expected stimulus, a listener needs to establish a fairly precise internal representation of its external temporal structure, a function ascribed to classical sensorimotor areas such as the cerebellum. Here we investigated how patients with cerebellar lesions and healthy matched controls exploit temporal regularity during auditory deviance processing. We expected modulations of the N2b and P3b components of the event-related potential in response to deviant tones, and also a stronger P3b response when deviant tones are embedded in temporally regular compared to irregular tone sequences. We further tested to what degree structural damage to the cerebellar temporal processing system affects the N2b and P3b responses associated with voluntary attention to change detection and the predictive adaptation of a mental model of the environment, respectively. Results revealed that healthy controls and cerebellar patients display an increased N2b response to deviant tones independent of temporal context. However, while healthy controls showed the expected enhanced P3b response to deviant tones in temporally regular sequences, the P3b response in cerebellar patients was significantly smaller in these sequences. The current data provide evidence that structural damage to the cerebellum affects the predictive adaptation to the temporal structure of events and the updating of a mental model of the environment under voluntary attention.