Charles Jorgensen

Gestures as input: neuroelectric joysticks and keyboards

EMG (electromyogram) technology helps capture gestures as input for virtual joysticks and keyboards and thus could lead to new applications in flight control, space, and the video game industry.

Speech interfaces based upon surface electromyography

This paper discusses the use of surface electromyography (EMG) to recognize and synthesize speech. The acoustic speech signal can be significantly corrupted by high noise in the environment or impeded by garments or masks. Such situations occur, for example, when firefighters wear pressurized suits with self-contained breathing apparatus (SCBA) or when astronauts perform operations in pressurized gear. In these conditions it is important to capture and transmit clear speech commands in spite of a corrupted or distorted acoustic speech signal. One way to mitigate this problem is to use surface electromyography to capture activity of speech articulators and then, either recognize spoken commands from EMG signals or use these signals to synthesize acoustic speech commands. We describe a set of experiments for both speech recognition and speech synthesis based on surface electromyography and discuss the lessons learned about the characteristics of the EMG signal for these domains. The experiments include speech recognition in high noise based on 15 commands for firefighters wearing self-contained breathing apparatus, a sub-vocal speech robotic platform control experiment based on five words, a speech recognition experiment testing recognition of vowels and consonants, and a speech synthesis experiment based on an articulatory speech synthesizer.

Small-vocabulary speech recognition using surface electromyography

We present results of electromyographic (EMG) speech recognition on a small vocabulary of 15 English words. EMG speech recognition holds promise for mitigating the effects of high acoustic noise on speech intelligibility in communication systems, including those used by first responders (a focus of this work). We collected 150 examples per word of single-channel EMG data from a male subject, speaking normally while wearing a firefighters self-contained breathing apparatus. The signal processing consisted of an activity detector, a feature extractor, and a neural network classifier. Testing produced an overall average correct classification rate on the 15 words of 74% with a 95% confidence interval of (71%, 77%). Once trained, the subject used a classifier as part of a real-time system to communicate to a cellular phone and to control a robotic device. These tasks were performed under an ambient noise level of approximately 95 decibels. We also describe ongoing work on phoneme-level EMG speech recognition.

Cerebellar neurocontroller project, for aerospace applications, in a civilian neurocomputing initiative in the 'decade of the brain'

A key question is how to utilize civilian government agencies along with an industrial consortium to successfully complement the so far primarily defense-oriented neural network research. Civilian artificial neural system projects, such as artificial cerebellar neurocontrollers aimed at duplicating natures existing neural network solutions for adaptive sensorimotor coordination, are proposed by such a synthesis. The cerebellum provides an intelligent interface between higher possibly symbolic levels of human intelligence and repetitious demands of real world conventional controllers. The generation of such intelligent interfaces could be crucial to the economic feasibility of the human settlement of space and an improvement in telerobotics techniques to permit the cost-effective exploitation of nonterrestrial materials and planetary exploration and monitoring. The authors propose a scientific framework within which such interagency activities could effectively cooperate.<<ETX>>

Associative neural networks for machine consciousness: Improving existing AI technologies

In this research we look at ways for improving existing AI techniques by the use of associative neural networks, proposed by Haikonen for machine consciousness. We find that all examined technologies do profit from such an approach: speech recognition, emotion recognition in speech, EMG data analysis for multilingual speech processing, the simulation of bistable perception and the generation of random numbers. EMG data analysis for multilingual speech processing (silent speech recognition) is selected as the main example in this paper for its simple yet complete architecture. We discuss the development of a test bench and give an overview of results obtained.