Sofia Cavaco

Robust phoneme recognition for a speech therapy environment

Traditional speech therapy approaches for speech sound disorders have a lot of advantages to gain from computer-based therapy systems. In this paper, we propose a robust phoneme recognition solution for an interactive environment for speech therapy. With speech recognition techniques the motivation elements of computer-based therapy systems can be automated in order to get an interactive environment that motivates the therapy attendee towards better performances. The contribution of this paper is a robust phoneme recognition to control the feedback provided to the patient during a speech therapy session. We compare the results of hierarchical and flat classification, with naive Bayes, support vector machines and kernel density estimation on linear predictive coding coefficients and Mel-frequency cepstral coefficients.

A voice-controlled serious game for the sustained vowel exercise

Speech is the main form of human communication. Thus it is important to detect and treat speech sound disorders as early as possible during childhood. When children need to attend speech therapy it is critical to keep them motivated on doing the therapy exercises. Software systems for speech therapy can be a useful tool to keep the child interested in keep practicing the therapy exercises. Several software systems have been developed to assist speech and language therapists during the therapy sessions. However most software focus on articulation disorders while voice disorders have been mostly neglected. Here we propose a voice-controlled serious computer game for the sustained vowel exercise, which is an exercise commonly used in speech therapy to treat voice disorders. The main novelty of this application is the combination of real time speech processing, with the gamification of the speech therapy exercises and the parameterization of the difficulty level.

An interactive toolset for speech therapy

This paper proposes a novel approach to include biofeedback in speech and language therapy by providing the patient with a visual self-monitoring of his/her performance combined with a reward mechanism in an entertainment environment. We propose a toolset that includes an in-session interactive environment to be used during the therapy sessions. This insession environment provides instantaneous biofeedback and assists the therapist during the session with rewards for the patients good performance. It also allows to make audio-visual recordings and annotations of the session for later analysis. The toolset also provides an off-line multimedia application for post-session analysis where the session audio-visual recordings can be examined through browsing, searching, and visualization techniques to plan the future session.

An orientation game with 3D spatialized audio for visually impaired children

In this article, we propose an educational mobile game designed to help visually impaired children to develop orientation skills. These skills are usually trained at orientation and mobility classes for special needs children. The proposed game can be played on touch screen mobile devices and can be used in class or after classes. It uses a child appropriate theme and it aims at training children to perform accurate sound localization, while distinguishing concepts like front/back, left/right, close/far, etc. We have had very promising results from a preliminary test with blind and low vision students. Training these concepts on an entertaining environment can have very positive outcomes as it motivates children to spend more time training and at the same time allows children to forget that they need to train due to their special needs.

From pixels to pitches: Unveiling the world of color for the blind

We propose a tool, SonarX, which converts color information from still images or video frames into sound. The tool converts the hue, saturation and value parameters into sound parameters that influence the perception of pitch, timbre and loudness. The goal of SonarX is to help visually impaired individuals to perceive characteristics of the environment that are usually not easily acquired without vision. The tool has been experimented by visually impaired individuals, who confirmed that it can be used to give them information about the range of colors present in the images, presence or absence of light sources as well the location and shape of the objects.

Mathematics for all: A game-based learning environment for visually impaired students

Visually impaired students who reach higher education have a preference for human sciences. The difficulties felt with courses such as mathematics, during their school years, may lead them to abandon the possibility of pursuing higher education in exact sciences and engineering courses. We propose a game-based learning environment that can be used in class and at home by visually impaired students. The game was developed for middle school mathematics but can be adapted for primary school and secondary school. The main goal of the game is to help and motivate blind teenage students to learn and like mathematics. The graphics are designed for low vision students but to guarantee accessibility to blind students, the game can be played without the need to see the graphics. All the features are complemented with audio and the game uses 2D spatialized audio. Having graphics also contributes to the integration of visually impaired students with their sighted colleagues. The game was tested in two schools with blind and low vision children. Both teachers and students were very enthusiastic with the game and gave us very positive feedback. The results of the test suggest that the game can have a positive influence in mathematics performance.

Classification of similar impact sounds

Several sound classifiers have been developed throughout the years. The accuracy provided by these classifiers is influenced by the features they use and the classification method implemented. While there are many approaches in sound feature extraction and in sound classification, most have been used to classify sounds with very different characteristics. Here, we propose a similar sound classifier that is able to distinguish sounds with very similar properties, namely sounds produced by objects with similar geometry and that only differ in material. The classifier applies independent component analysis to learn temporal and spectral features of the sounds, which are then used by a 1-nearest neighbor algorithm. We concluded that the features extracted in this way are powerful enough for classifying similar sounds. Finally, a user study shows that the classifier achieves better performance than humans in the classification of the sounds used here.

A BIOLOGICALLY PLAUSIBLE ACOUSTIC MOTION DETECTION NEURAL NETWORK

In this paper we present an acoustic motion detection system to be used in a small mobile robot. While the first purpose of the system has been to be a reliable computational implementation, cheap enough to be built in hardware, effort has also been taken to construct a biologically plausible solution. The motion detector consists of a neural network composed of motion-direction sensitive neurons with a preferred direction and a preferred region of the azimuth. The system was designed to produce a higher response when stimulated by motion in the preferred direction than in the null direction and that is in fact what the system does, which means that, as desired, the system can detect motion and distinguish its direction.

NavMol 3.0: enabling the representation of metabolic reactions by blind users

Summary The representation of metabolic reactions strongly relies on visualization, which is a major barrier for blind users. The NavMol software renders the communication and interpretation of molecular structures and reactions accessible by integrating chemoinformatics and assistive technology. NavMol 3.0 provides a molecular editor for metabolic reactions. The user can start with templates of reactions and build from such cores. Atom-to-atom mapping enables changes in the reactants to be reflected in the products (and vice-versa) and the reaction centres to be automatically identified. Blind users can easily interact with the software using the keyboard and text-to-speech technology. Availability and implementation NavMol 3.0 is free and open source under the GNU general public license (GPLv3), and can be downloaded at http://sourceforge.net/projects/navmol as a JAR file. Contact joao@airesdesousa.com.

Visual-feedback in an interactive environment for speech-language therapy.

By combining visual-feedback and motivational elements, a speech therapy computer-based system can offer new approaches with various advantages when compared to traditional speech therapy techniques. Through visual-feedback and adaptation of traditional speech sound exercises, it is possible to create an engaging environment with motivation focused elements. These elements can be used in an interactive environment that motivates the therapy attendee towards better performances. Hereby we present an interactive gamified environment for speech therapy that combines visual-feedback and motivational components. The results from a survey and a usability study suggest that children can show more interest in the speech therapy sessions when the proposed environment is used.