Matthew Rodger

Synthesis of Walking Sounds for Alleviating Gait Disturbances in Parkinson's Disease

Managing gait disturbances in people with Parkinsons disease is a pressing challenge, as symptoms can contribute to injury and morbidity through an increased risk of falls. While drug-based interventions have limited efficacy in alleviating gait impairments, certain nonpharmacological methods, such as cueing, can also induce transient improvements to gait. The approach adopted here is to use computationally-generated sounds to help guide and improve walking actions. The first method described uses recordings of force data taken from the steps of a healthy adult which in turn were used to synthesize realistic gravel-footstep sounds that represented different spatio-temporal parameters of gait, such as step duration and step length. The second method described involves a novel method of sonifying, in real time, the swing phase of gait using real-time motion-capture data to control a sound synthesis engine. Both approaches explore how simple but rich auditory representations of action based events can be used by people with Parkinsons to guide and improve the quality of their walking, reducing the risk of falls and injury. Studies with Parkinsons disease patients are reported which show positive results for both techniques in reducing step length variability. Potential future directions for how these sound approaches can be used to manage gait disturbances in Parkinsons are also discussed.

Timing movements to interval durations specified by discrete or continuous sounds

Understanding how the timing of motor output is coupled to sensory temporal information is largely based on synchronisation of movements through small motion gaps (finger taps) to mostly empty sensory intervals (discrete beats). This study investigated synchronisation of movements between target barriers over larger motion gaps when closing time gaps of intervals were presented as either continuous, dynamic sounds, or discrete beats. Results showed that although synchronisation errors were smaller for discrete sounds, the variability of errors was lower for continuous sounds. Furthermore, finger movement between targets was found to be more sinusoidal when continuous sensory information was presented during intervals compared to discrete. When movements were made over larger amplitudes, synchronisation errors tended to be more positive and movements between barriers more sinusoidal, than for movements over shorter amplitudes. These results show that the temporal control of movement is not independent from the form of the sensory information that specifies time gaps or the magnitude of the movement required for synchronisation.

Refreshing Refreshable Braille Displays

The increased access to books afforded to blind people via e-publishing has given them long-sought independence for both recreational and educational reading. In most cases, blind readers access materials using speech output. For some content such as highly technical texts, music, and graphics, speech is not an appropriate access modality as it does not promote deep understanding. Therefore blind braille readers often prefer electronic braille displays. But, these are prohibitively expensive. The search is on, therefore, for a low-cost refreshable display that would go beyond current technologies and deliver graphical content as well as text. And many solutions have been proposed, some of which reduce costs by restricting the number of characters that can be displayed, even down to a single braille cell. In this paper, we demonstrate that restricting tactile cues during braille reading leads to poorer performance in a letter recognition task. In particular, we show that lack of sliding contact between the fingertip and the braille reading surface results in more errors and that the number of errors increases as a function of presentation speed. These findings suggest that single cell displays which do not incorporate sliding contact are likely to be less effective for braille reading.

Expert players accurately detect an opponent's movement intentions through sound alone

Sounds offer a rich source of information about events taking place in our physical and social environment. However, outside the domains of speech and music, little is known about whether humans can recognize and act upon the intentions of another agent’s actions detected through auditory information alone. In this study we assessed whether intention can be inferred from the sound an action makes, and in turn, whether this information can be used to prospectively guide movement. In 2 experiments experienced and novice basketball players had to virtually intercept an attacker by listening to audio recordings of that player’s movements. In the first experiment participants had to move a slider, while in the second one their body, to block the perceived passage of the attacker as they would in a real basketball game. Combinations of deceptive and nondeceptive movements were used to see if novice and/or experienced listeners could perceive the attacker’s intentions through sound alone. We showed that basketball players were able to more accurately predict final running direction compared to nonplayers, particularly in the second experiment when the interceptive action was more basketball specific. We suggest that athletes present better action anticipation by being able to pick up and use the relevant kinematic features of deceptive movement from event-related sounds alone. This result suggests that action intention can be perceived through the sound a movement makes and that the ability to determine another person’s action intention from the information conveyed through sound is honed through practice.

Bringing sounds into use: Thinking of sounds as materials and a sketch of auditory affordances

We live in a richly structured auditory environment. From the sounds of cars charging towards us on the street to the sounds of music filling a dancehall, sounds like these are generally seen as being instances of things we hear but can also be understood as opportunities for action. In some circumstances, the sound of a car approaching towards us can pro- vide critical information for the avoidance of harm. In the context of a concert venue, sociocultural practices like music can equally afford coordinated activities of movement, such as dancing or music making. Despite how evident the behav- ioral effects of sound are in our everyday experience, they have been sparsely accounted for within the field of psychol- ogy. Instead, most theories of auditory perception have been more concerned with understanding how sounds are pas- sively processed and represented and how they convey information of the world, neglecting than how this information can be used for anything. Here, we argue against these previous rationalizations, suggesting instead that information is instan- tiated through use and, therefore, is an emergent effect of a perceivers interaction with their environment. Drawing on theory from psychology, philosophy and anthropology, we contend that by thinking of sounds as materials, theorists and researchers alike can get to grips with the vast array of auditory affordances that we purposefully bring into use when in- teracting with the environment.

Time to get a move on: Overcoming bradykinetic movement in Parkinson's disease with artificial sensory guidance generated from biological motion

Paradoxical kinesia describes the motor improvement in Parkinsons disease (PD) triggered by the presence of external sensory information relevant for the movement. This phenomenon has been puzzling scientists for over 60 years, both in neurological and motor control research, with the underpinning mechanism still being the subject of fierce debate. In this paper we present novel evidence supporting the idea that the key to understanding paradoxical kinesia lies in both spatial and temporal information conveyed by the cues and the coupling between perception and action. We tested a group of 7 idiopathic PD patients in an upper limb mediolateral movement task. Movements were performed with and without a visual point light display, travelling at 3 different speeds. The dynamic information presented in the visual point light display depicted three different movement speeds of the same amplitude performed by a healthy adult. The displays were tested and validated on a group of neurologically healthy participants before being tested on the PD group. Our data show that the temporal aspects of the movement (kinematics) in PD can be moderated by the prescribed temporal information presented in a dynamic environmental cue. Patients demonstrated a significant improvement in terms of movement time and peak velocity when executing movement in accordance with the information afforded by the point light display, compared to when the movement of the same amplitude and direction was performed without the display. In all patients we observed the effect of paradoxical kinesia, with a strong relationship between the perceptual information prescribed by the biological motion display and the observed motor performance of the patients.

Transposing musical skill: sonification of movement as concurrent augmented feedback enhances learning in a bimanual task.

Concurrent feedback provided during acquisition can enhance performance of novel tasks. The ‘guidance hypothesis’ predicts that feedback provision leads to dependence and poor performance in its absence. However, appropriately structured feedback information provided through sound (‘sonification’) may not be subject to this effect. We test this directly using a rhythmic bimanual shape-tracing task in which participants learned to move at a 4:3 timing ratio. Sonification of movement and demonstration was compared to two other learning conditions: (1) Sonification of task demonstration alone and (2) completely silent practice (control). Sonification of movement emerged as the most effective form of practice, reaching significantly lower error scores than control. Sonification of solely the demonstration, which was expected to benefit participants by perceptually unifying task requirements, did not lead to better performance than control. Good performance was maintained by participants in the Sonification condition in an immediate retention test without feedback, indicating that the use of this feedback can overcome the guidance effect. On a 24-h retention test, performance had declined and was equal between groups. We argue that this and similar findings in the feedback literature are best explained by an ecological approach to motor skill learning which places available perceptual information at the highest level of importance.

(Dis-)Harmony in movement: effects of musical dissonance on movement timing and form

Abstract While the origins of consonance and dissonance in terms of acoustics, psychoacoustics and physiology have been debated for centuries, their plausible effects on movement synchronization have largely been ignored. The present study aimed to address this by investigating whether, and if so how, consonant/dissonant pitch intervals affect the spatiotemporal properties of regular reciprocal aiming movements. We compared movements synchronized either to consonant or to dissonant sounds and showed that they were differentially influenced by the degree of consonance of the sound presented. Interestingly, the difference was present after the sound stimulus was removed. In this case, the performance measured after consonant sound exposure was found to be more stable and accurate, with a higher percentage of information/movement coupling (tau coupling) and a higher degree of movement circularity when compared to performance measured after the exposure to dissonant sounds. We infer that the neural resonance representing consonant tones leads to finer perception/action coupling which in turn may help explain the prevailing preference for these types of tones.

Beyond the Metronome: Auditory Events and Music May Afford More than Just Interval Durations as Gait Cues in Parkinson's Disease

Among the most apparent and adverse symptoms of Parkinsons disease (PD) are disturbances in gait. These include shuffling (small amplitude steps), instability (asymmetry and variability between steps), freezing of gait (cessation of movement and difficulty with initiation), and general disfluencies in walking movements and posture (Morris et al., 1996; Bloem et al., 2004; Grabli et al., 2012). Limitations of pharmacological interventions to alleviate gait disturbances (Lord et al., 2011), have led to interest in exploring non-pharmacological means of enhancing walking in PD, to complement drugs-based approaches. Sensory cueing, in which perceptual guides for movement are presented visually, acoustically, or haptically, is one such approach. While sensory cueing, in particular rhythmic auditory cueing, is a viable and promising approach to enhancing gait in PD, it is our opinion that this approach could be expanded by developing a more action-focussed framework for understanding the information available to patients in sound (cues) and how this information influences gait.

Sonification as Concurrent Augmented Feedback for Motor Skill Learning and the Importance of Mapping Design

In recent years, sonification of movement has emerged as a viable method for the provision of feedback in mo- tor learning. Despite some experimental validation of its utility, controlled trials to test the usefulness of sonification in a motor learning context are still rare. As such, there are no accepted conventions for dealing with its implementation. This article addresses the question of how continuous movement information should be best presented as sound to be fed back to the learner. It is proposed that to establish effective approaches to using sonification in this context, consideration must be given to the processes that underlie motor learning, in particular the nature of the perceptual information available to the learner for performing the task at hand. Although sonification has much potential in movement performance enhance- ment, this potential is largely unrealised as of yet, in part due to the lack of a clear framework for sonification mapping: the relationship between movement and sound. By grounding mapping decisions in a firmer understanding of how percep- tual information guides learning, and an embodied cognition stance in general, it is hoped that greater advances in use of sonification to enhance motor learning can be achieved.