Augmented Humans Conference 2021 | 2021
SilentMask: Mask-type Silent Speech Interface with Measurement of Mouth Movement
Abstract
Silent Speech Interaction (SSI) is a non-speech interaction used as an input method for speech recognition devices such as smartphones and as a support tool for people with speech difficulties. Conventional SSI methods using lip reading, electromyography(EMG), ultrasonic echo, and electrostatic positioning in the palate have been proposed, but there have been issues such as not being able to use one hand and being easily noticeable. In this study, we propose a mask-based SSI that recognizes silent speech by measuring the motion around the mouth using acceleration and angular velocity sensors attached to mask. Using two acceleration and angular velocity sensors to acquire 12-dimensional motion information around the mouth and analyzing it using deep learning, we were able to identify a total of 22 states (21 types of voice commands and no speech) with 79.9% accuracy. The results also showed that the device can be worn for a longer period of time compared to the method of applying the sensors directly to the skin. This research presents new possibilities for masks, as they are a non-contact, unobtrusive interface that does not use camera images and is therefore independent of lighting conditions.