Gerd Schmitz

Observation of sonified movements engages a basal ganglia frontocortical network.

BackgroundProducing sounds by a musical instrument can lead to audiomotor coupling, i.e. the joint activation of the auditory and motor system, even when only one modality is probed. The sonification of otherwise mute movements by sounds based on kinematic parameters of the movement has been shown to improve motor performance and perception of movements.ResultsHere we demonstrate in a group of healthy young non-athletes that congruently (sounds match visual movement kinematics) vs. incongruently (no match) sonified breaststroke movements of a human avatar lead to better perceptual judgement of small differences in movement velocity. Moreover, functional magnetic resonance imaging revealed enhanced activity in superior and medial posterior temporal regions including the superior temporal sulcus, known as an important multisensory integration site, as well as the insula bilaterally and the precentral gyrus on the right side. Functional connectivity analysis revealed pronounced connectivity of the STS with the basal ganglia and thalamus as well as frontal motor regions for the congruent stimuli. This was not seen to the same extent for the incongruent stimuli.ConclusionsWe conclude that sonification of movements amplifies the activity of the human action observation system including subcortical structures of the motor loop. Sonification may thus be an important method to enhance training and therapy effects in sports science and neurological rehabilitation.

Movement Sonification: Effects on Motor Learning beyond Rhythmic Adjustments

Motor learning is based on motor perception and emergent perceptual-motor representations. A lot of behavioral research is related to single perceptual modalities but during last two decades the contribution of multimodal perception on motor behavior was discovered more and more. A growing number of studies indicates an enhanced impact of multimodal stimuli on motor perception, motor control and motor learning in terms of better precision and higher reliability of the related actions. Behavioral research is supported by neurophysiological data, revealing that multisensory integration supports motor control and learning. But the overwhelming part of both research lines is dedicated to basic research. Besides research in the domains of music, dance and motor rehabilitation, there is almost no evidence for enhanced effectiveness of multisensory information on learning of gross motor skills. To reduce this gap, movement sonification is used here in applied research on motor learning in sports. Based on the current knowledge on the multimodal organization of the perceptual system, we generate additional real-time movement information being suitable for integration with perceptual feedback streams of visual and proprioceptive modality. With ongoing training, synchronously processed auditory information should be initially integrated into the emerging internal models, enhancing the efficacy of motor learning. This is achieved by a direct mapping of kinematic and dynamic motion parameters to electronic sounds, resulting in continuous auditory and convergent audiovisual or audio-proprioceptive stimulus arrays. In sharp contrast to other approaches using acoustic information as error-feedback in motor learning settings, we try to generate additional movement information suitable for acceleration and enhancement of adequate sensorimotor representations and processible below the level of consciousness. In the experimental setting, participants were asked to learn a closed motor skill (technique acquisition of indoor rowing). One group was treated with visual information and two groups with audiovisual information (sonification vs. natural sounds). For all three groups learning became evident and remained stable. Participants treated with additional movement sonification showed better performance compared to both other groups. Results indicate that movement sonification enhances motor learning of a complex gross motor skill—even exceeding usually expected acoustic rhythmic effects on motor learning.

Auditory coding of human movement kinematics.

Although visual perception is dominant on motor perception, control and learning, auditory information can enhance and modulate perceptual as well as motor processes in a multifaceted manner. During last decades new methods of auditory augmentation had been developed with movement sonification as one of the most recent approaches expanding auditory movement information also to usually mute phases of movement. Despite general evidence on the effectiveness of movement sonification in different fields of applied research there is nearly no empirical proof on how sonification of gross motor human movement should be configured to achieve information rich sound sequences. Such lack of empirical proof is given for (a) the selection of suitable movement features as well as for (b) effective kinetic-acoustical mapping patterns and for (c) the number of regarded dimensions of sonification. In this study we explore the informational content of artificial acoustical kinematics in terms of a kinematic movement sonification using an intermodal discrimination paradigm. In a repeated measure design we analysed discrimination rates of six everyday upper limb actions to evaluate the effectiveness of seven different kinds of kinematic-acoustical mappings as well as short-term learning effects. The kinematics of the upper limb actions were calculated based on inertial motion sensor data and transformed into seven different sonifications. Sound sequences were randomly presented to participants and discrimination rates as well as confidence of choice were analysed. Data indicate an instantaneous comprehensibility of the artificial movement acoustics as well as short-term learning effects. No differences between different dimensional encodings became evident thus indicating a high efficiency for intermodal pattern discrimination for the acoustically coded velocity distribution of the actions. Taken together movement information related to continuous kinematic parameters can be transformed into the auditory domain. Additionally, pattern based action discrimination is obviously not restricted to the visual modality. Artificial acoustical kinematics might be used to supplement and/or substitute visual motion perception in sports and motor rehabilitation.

Adaptation of eye and hand movements to target displacements of different size

Previous work has documented that the direction of eye and hand movements can be adaptively modified using the double-step paradigm. Here we report that both motor systems adapt not only to small direction steps (5° gaze angle) but also to large ones (28° gaze angle). However, the magnitude of adaptation did not increase with step size, and the relative magnitude of adaptation therefore decreased from 67% with small steps to 15% with large steps. This decreasing efficiency of adaptation may reflect the participation of directionally selective neural circuits in double-step adaptation.

Effect of rhythmic auditory cueing on parkinsonian gait: A systematic review and meta-analysis

The use of rhythmic auditory cueing to enhance gait performance in parkinsonian patients’ is an emerging area of interest. Different theories and underlying neurophysiological mechanisms have been suggested for ascertaining the enhancement in motor performance. However, a consensus as to its effects based on characteristics of effective stimuli, and training dosage is still not reached. A systematic review and meta-analysis was carried out to analyze the effects of different auditory feedbacks on gait and postural performance in patients affected by Parkinson’s disease. Systematic identification of published literature was performed adhering to PRISMA guidelines, from inception until May 2017, on online databases; Web of science, PEDro, EBSCO, MEDLINE, Cochrane, EMBASE and PROQUEST. Of 4204 records, 50 studies, involving 1892 participants met our inclusion criteria. The analysis revealed an overall positive effect on gait velocity, stride length, and a negative effect on cadence with application of auditory cueing. Neurophysiological mechanisms, training dosage, effects of higher information processing constraints, and use of cueing as an adjunct with medications are thoroughly discussed. This present review bridges the gaps in literature by suggesting application of rhythmic auditory cueing in conventional rehabilitation approaches to enhance motor performance and quality of life in the parkinsonian community.

Adaptation to rotated visual feedback depends on the number and spread of target directions

It has been proposed in the past that adaptation to rotated visual feedback is based on directionally tuned modules. Here, we investigate whether adaptation depends on the number of modules that are concurrently activated. To disambiguate the number of modules and their spatial overlap, we decided to vary the number of target directions and their spacing independently. In light of recent work on the existence of fast and slow adaptive processes, we analyzed the role of target number and spacing separately for the first eight movements under rotated visual feedback and for later movements. We found that during the first eight movements, adaptation progressed three times faster when targets were spaced across a 42° rather than across a 360° range, irrespective of their number. During the subsequent movements, adaptation progressed 1.66 times faster with two than with eight targets, irrespective of their spacing. This differential dependence of early and late adaptation on target metrics confirms the existence of adaptive processes with different time scales and suggests that those processes differ not only by their clocking speed, but also by their functional properties. Specifically, the speed of fast processes seems to be constrained by the directional tuning width of adaptive modules, and the speed of slow processes by the number of presentations per target direction.

Directional adaptation of reactive saccades and hand pointing movements is not independent.

ABSTRACT It is a matter of debate whether reactive saccades and hand pointing movements share common adaptive mechanism. To find out, the authors used a double-step paradigm in which the direction either of eye or of hand movements was adaptively modified in a first block of 300 trials, and the direction of the other motor system was then modified with opposite polarity in a second block of 300 trials. In a third block, single-step stimuli were used to test for after effects. The authors found that subjects adapted in the second block less well than in the first, and that aftereffects were adequate for the first rather than the second block. When the second block was extended to 500 trials, adaptation was still poor but aftereffects were now adequate for the second block. From this the authors concluded that double-step adaptation of the first motor system interferes with the subsequent adaptation of the other motor system (i.e., the adaptive mechanisms for eyes and hand are not independent).

A Comparison of Sensorimotor Adaptation in the Visual and in the Auditory Modality

We compared sensorimotor adaptation in the visual and the auditory modality. Subjects pointed to visual targets while receiving direct spatial information about fingertip position in the visual modality, or they pointed to visual targets while receiving indirect information about fingertip position in the visual modality, or they pointed to auditory targets while receiving indirect information about fingertip position in the auditory modality. Feedback was laterally shifted to induce adaptation, and aftereffects were tested with both target modalities and both hands. We found that aftereffects of adaptation were smaller when tested with the non-adapted hand, i.e., intermanual transfer was incomplete. Furthermore, aftereffects were smaller when tested in the non-adapted target modality, i.e., intermodal transfer was incomplete. Aftereffects were smaller following adaptation with indirect rather than direct feedback, but they were not smaller following adaptation with auditory rather than visual targets. From this we conclude that the magnitude of adaptive recalibration rather depends on the method of feedback delivery (indirect versus direct) than on the modality of feedback (visual versus auditory).

Auditory Proprioceptive Integration: Effects of Real-Time Kinematic Auditory Feedback on Knee Proprioception

The purpose of the study was to assess the influence of real-time auditory feedback on knee proprioception. Thirty healthy participants were randomly allocated to control (n = 15), and experimental group I (15). The participants performed an active knee-repositioning task using their dominant leg, with/without additional real-time auditory feedback where the frequency was mapped in a convergent manner to two different target angles (40 and 75°). Statistical analysis revealed significant enhancement in knee re-positioning accuracy for the constant and absolute error with real-time auditory feedback, within and across the groups. Besides this convergent condition, we established a second divergent condition. Here, a step-wise transposition of frequency was performed to explore whether a systematic tuning between auditory-proprioceptive repositioning exists. No significant effects were identified in this divergent auditory feedback condition. An additional experimental group II (n = 20) was further included. Here, we investigated the influence of a larger magnitude and directional change of step-wise transposition of the frequency. In a first step, results confirm the findings of experiment I. Moreover, significant effects on knee auditory-proprioception repositioning were evident when divergent auditory feedback was applied. During the step-wise transposition participants showed systematic modulation of knee movements in the opposite direction of transposition. We confirm that knee re-positioning accuracy can be enhanced with concurrent application of real-time auditory feedback and that knee re-positioning can modulated in a goal-directed manner with step-wise transposition of frequency. Clinical implications are discussed with respect to joint position sense in rehabilitation settings.

Adaptation of hand movements to double-step targets and to distorted visual feedback: Evidence for shared mechanisms

Visuomotor adaptation of hand movements has been studied with two paradigms: double-step targets and distorted visual feedback. Here we investigate whether both procedures are based on a common adaptive mechanism. Subjects adapted either to double-step targets or to distorted feedback, each requiring a change of response angle by -15°. The magnitude of adaptation was larger with rotated feedback but magnitude of aftereffects was comparable, suggesting that the difference was due to strategic effects rather than visuomotor recalibration. Most importantly, subjects who adapted to double-step targets and were then exposed to rotated feedback performed as well as subjects who had fully adapted to rotated feedback, i.e., there was nearly 100% transfer from double-steps to rotations; likewise, the transfer from rotations to double-steps was almost 100%. From this we conclude that both types of adaptation share a common mechanism for recalibration.