Simone Spagnol

Effect of task-related continuous auditory feedback during learning of tracking motion exercises.

BackgroundThis paper presents the results of a set of experiments in which we used continuous auditory feedback to augment motor training exercises. This feedback modality is mostly underexploited in current robotic rehabilitation systems, which usually implement only very basic auditory interfaces. Our hypothesis is that properly designed continuous auditory feedback could be used to represent temporal and spatial information that could in turn, improve performance and motor learning.MethodsWe implemented three different experiments on healthy subjects, who were asked to track a target on a screen by moving an input device (controller) with their hand. Different visual and auditory feedback modalities were envisaged. The first experiment investigated whether continuous task-related auditory feedback can help improve performance to a greater extent than error-related audio feedback, or visual feedback alone. In the second experiment we used sensory substitution to compare different types of auditory feedback with equivalent visual feedback, in order to find out whether mapping the same information on a different sensory channel (the visual channel) yielded comparable effects with those gained in the first experiment. The final experiment applied a continuously changing visuomotor transformation between the controller and the screen and mapped kinematic information, computed in either coordinate system (controller or video), to the audio channel, in order to investigate which information was more relevant to the user.ResultsTask-related audio feedback significantly improved performance with respect to visual feedback alone, whilst error-related feedback did not. Secondly, performance in audio tasks was significantly better with respect to the equivalent sensory-substituted visual tasks. Finally, with respect to visual feedback alone, video-task-related sound feedback decreased the tracking error during the learning of a novel visuomotor perturbation, whereas controller-task-related sound feedback did not. This result was particularly interesting, as the subjects relied more on auditory augmentation of the visualized target motion (which was altered with respect to arm motion by the visuomotor perturbation), rather than on sound feedback provided in the controller space, i.e., information directly related to the effective target motion of their arm.ConclusionsOur results indicate that auditory augmentation of visual feedback can be beneficial during the execution of upper limb movement exercises. In particular, we found that continuous task-related information provided through sound, in addition to visual feedback can improve not only performance but also the learning of a novel visuomotor perturbation. However, error-related information provided through sound did not improve performance and negatively affected learning in the presence of the visuomotor perturbation.

Effects of Complementary Auditory Feedback in Robot-Assisted Lower Extremity Motor Adaptation

This study investigates how complementary auditory feedback may affect short-term gait modifications induced by four training sessions with a robotic exoskeleton. Healthy subjects walked on a treadmill and were instructed to match a modified gait pattern derived from their natural one, while receiving assistance by the robot (kinetic guidance). The main question we wanted to answer is whether the most commonly used combination of feedback (i.e., haptic and visual) could be either enhanced by adding auditory feedback or successfully substituted with a combination of kinetic guidance and auditory feedback. Participants were randomly assigned to one of four groups, all of which received kinetic guidance. The control group received additional visual feedback, while the three experimental groups were each provided with a different modality of auditory feedback. The third experimental group also received the same visual feedback as the control group. Differences among the training modalities in gait kinematics, timing and symmetry were assessed in three post-training sessions.

Enhancing vertical localization with image-guided selection of non-individual head-related transfer functions

A novel approach to the selection of generic head-related transfer functions (HRTFs) for binaural audio rendering through headphones is formalized and described in this paper. A reflection model applied to the users ear picture facilitates extraction of the relevant anthropometric cues that are used for selecting two HRTF sets in a database fitting that user, whose localization performances are evaluated in a complete psychoacoustic experiment. The proposed selection increases the average elevation performances of 17% (with a peak of 34%) with respect to generic HRTFs from an anthropomorphic mannequin. It also significantly enhances externalization and reduces the number of up/down reversals.

A Head-Related Transfer Function Model for Real-Time Customized 3-D Sound Rendering

This paper addresses the problem of modeling head-related transfer functions (HRTFs) for 3-D audio rendering in the front hemisphere. Following a structural approach, we build a model for real-time HRTF synthesis which allows to control separately the evolution of different acoustic phenomena such as head diffraction, ear resonances, and reflections through the design of distinct filter blocks. Parameters to be fed to the model are both derived from mean spectral features in a collection of measured HRTFs and anthropometric features of the specific subject (taken from a photograph of his/her outer ear), hence allowing model customization. Visual analysis of the synthesized HRTFs reveals a convincing correspondence between original and reconstructed spectral features in the chosen spatial range. Furthermore, a possible experimental setup for dynamic psycho acoustical evaluation of such model is depicted.

Improving robotics for neurorehabilitation: Enhancing engagement, performance, and learning with auditory feedback

This paper reports on an ongoing research collaboration between the University of Padua and the University of California Irvine, on the use of continuous auditory-feedback in robot-assisted neurorehabilitation of post-stroke patients. This feedback modality is mostly underexploited in current robotic rehabilitation systems, that usually implement very basic auditory feedback interfaces. The results of this research show that generating a proper sound cue during robot assisted movement training can help patients in improving engagement, performance and learning in the exercise.

Fitting pinna-related transfer functions to anthropometry for binaural sound rendering

This paper faces the general problem of modeling pinna-related transfer functions (PRTFs) for 3-D sound rendering. Following a structural approach, we aim at constructing a model for PRTF synthesis which allows to control separately the evolution of ear resonances and spectral notches through the design of two distinct filter blocks. Taking such model as endpoint, we propose a method based on the McAulay-Quatieri partial tracking algorithm to extract the frequencies of the most important spectral notches. Ray-tracing analysis performed on the so obtained tracks reveals a convincing correspondence between extracted frequencies and pinna geometry of a bunch of subjects.

Designing sensory-substitution devices: Principles, pitfalls and potential1

An exciting possibility for compensating for loss of sensory function is to augment deficient senses by conveying missing information through an intact sense. Here we present an overview of techniques that have been developed for sensory substitution (SS) for the blind, through both touch and audition, with special emphasis on the importance of training for the use of such devices, while highlighting potential pitfalls in their design. One example of a pitfall is how conveying extra information about the environment risks sensory overload. Related to this, the limits of attentional capacity make it important to focus on key information and avoid redundancies. Also, differences in processing characteristics and bandwidth between sensory systems severely constrain the information that can be conveyed. Furthermore, perception is a continuous process and does not involve a snapshot of the environment. Design of sensory substitution devices therefore requires assessment of the nature of spatiotemporal continuity for the different senses. Basic psychophysical and neuroscientific research into representations of the environment and the most effective ways of conveying information should lead to better design of sensory substitution systems. Sensory substitution devices should emphasize usability, and should not interfere with other inter- or intramodal perceptual function. Devices should be task-focused since in many cases it may be impractical to convey too many aspects of the environment. Evidence for multisensory integration in the representation of the environment suggests that researchers should not limit themselves to a single modality in their design. Finally, we recommend active training on devices, especially since it allows for externalization, where proximal sensory stimulation is attributed to a distinct exterior object.

Mixed structural modeling of head-related transfer functions for customized binaural audio delivery

A novel approach to the modeling of head-related transfer functions (HRTFs) for binaural audio rendering is formalized and described in this paper. Mixed structural modeling (MSM) can be seen as the generalization and extension of the structural modeling approach first defined by Brown and Duda back in 1998. Possible solutions for building partial HRTFs (pHRTFs) of the head, torso, and pinna of a specific listener are first described and then used in the construction of two possible mixed structural models of a KEMAR mannequin. Thanks to the flexibility of the MSM approach, an exponential number of solutions for building custom binaural audio displays can be considered and evaluated, the final aim of the process being the achievement of a HRTF model fully customizable by the listener.

On distance dependence of pinna spectral patterns in head-related transfer functions.

The aim of this letter is to address a little understood question in sound source localization: Can the distance of a near sound source affect our own perception of its elevation? The issue is studied by means of an objective analysis of a database of distance-dependent head-related transfer functions (HRTFs) of a KEMAR (Knowles Electronic Manikin for Acoustic Research) mannequin with different pinnae on a dense spatial grid. Iso-directional HRTFs are compared through spectral error metrics; results indicate significant distance-dependent HRTF modifications due to the pinna occur when the source is close to the interaural axis.

Synthetic individual binaural audio delivery by pinna image processing

Purpose – The purpose of this paper is to present a system for customized binaural audio delivery based on the extraction of relevant features from a 2-D representation of the listener’s pinna. Design/methodology/approach – The most significant pinna contours are extracted by means of multi-flash imaging, and they provide values for the parameters of a structural head-related transfer function (HRTF) model. The HRTF model spatializes a given sound file according to the listener’s head orientation, tracked by sensor-equipped headphones, with respect to the virtual sound source. Findings – A preliminary localization test shows that the model is able to statically render the elevation of a virtual sound source better than non-individual HRTFs. Research limitations/implications – Results encourage a deeper analysis of the psychoacoustic impact that the individualized HRTF model has on perceived elevation of virtual sound sources. Practical implications – The model has low complexity and is suitable for implem...