Ji-hai Yang

A Framework for Hand Gesture Recognition Based on Accelerometer and EMG Sensors

This paper presents a framework for hand gesture recognition based on the information fusion of a three-axis accelerometer (ACC) and multichannel electromyography (EMG) sensors. In our framework, the start and end points of meaningful gesture segments are detected automatically by the intensity of the EMG signals. A decision tree and multistream hidden Markov models are utilized as decision-level fusion to get the final results. For sign language recognition (SLR), experimental results on the classification of 72 Chinese Sign Language (CSL) words demonstrate the complementary functionality of the ACC and EMG sensors and the effectiveness of our framework. Additionally, the recognition of 40 CSL sentences is implemented to evaluate our framework for continuous SLR. For gesture-based control, a real-time interactive system is built as a virtual Rubiks cube game using 18 kinds of hand gestures as control commands. While ten subjects play the game, the performance is also examined in user-specific and user-independent classification. Our proposed framework facilitates intelligent and natural control in gesture-based interaction.

Hand gesture recognition and virtual game control based on 3D accelerometer and EMG sensors

This paper describes a novel hand gesture recognition system that utilizes both multi-channel surface electromyogram (EMG) sensors and 3D accelerometer (ACC) to realize user-friendly interaction between human and computers. Signal segments of meaningful gestures are determined from the continuous EMG signal inputs. Multi-stream Hidden Markov Models consisting of EMG and ACC streams are utilized as decision fusion method to recognize hand gestures. This paper also presents a virtual Rubiks Cube game that is controlled by the hand gestures and is used for evaluating the performance of our hand gesture recognition system. For a set of 18 kinds of gestures, each trained with 10 repetitions, the average recognition accuracy was about 91.7% in real application. The proposed method facilitates intelligent and natural control based on gesture interaction.

Automatic recognition of sign language subwords based on portable accelerometer and EMG sensors

Sign language recognition (SLR) not only facilitates the communication between the deaf and hearing society, but also serves as a good basis for the development of gesture-based human-computer interaction (HCI). In this paper, the portable input devices based on accelerometers and surface electromyography (EMG) sensors worn on the forearm are presented, and an effective fusion strategy for combination of multi-sensor and multi-channel information is proposed to automatically recognize sign language at the subword classification level. Experimental results on the recognition of 121 frequently used Chinese sign language subwords demonstrate the feasibility of developing SLR system based on the presented portable input devices and that our proposed information fusion method is effective for automatic SLR. Our study will promote the realization of practical sign language recognizer and multimodal human-computer interfaces.

Research on gesture definition and electrode placementin pattern recognition of hand gesture action SEMG

The goal of this study is to explore the effects of electrode placement on the hand gesture pattern recognition performance. We have conducted experiments with surface EMG sensors using two detecting electrode channels. In total 25 different hand gestures and 10 different electrode positions for measuring muscle activities have been evaluated. Based on the experimental results, dependencies between surface EMG signal detection positions and hand gesture recognition performance have been analyzed and summarized as suggestions how to define hand gestures and select suitable electrode positions for a myoelectric control system. This work provides useful insight for the development of a medical rehabilitation system based on EMG technique.

Interpreting sign components from accelerometer and sEMG data for automatic sign language recognition

The identification of constituent components of each sign gesture is a practical way of establishing large-vocabulary sign language recognition (SLR) system. Aiming at developing such a system using portable accelerometer (ACC) and surface electromyographic (sEMG) sensors, this work proposes a method for automatic SLR at the component level. The preliminary experimental results demonstrate the effectiveness of the proposed method and the feasibility of interpreting sign components from ACC and sEMG data. Our study improves the performance of SLR based on ACC and sEMG sensors and will promote the realization of a large-vocabulary portable SLR system.

A Method of Hand Gesture Recognition Based on Multiple Sensors

This paper presents a new method of gesture recognition based on multiple sensors fusion technique. Three kinds of sensors, namely surface Electromyography (sEMG) sensor, 3-axis accelerometer (ACC) and camera, are used together to capture the dynamic hand gesture firstly. Then four types of features are extracted from the three kinds of sensory data to depict the static hand posture and dynamic gesture trajectory characteristics of hand gesture. Finally decision-level multi-classifier fusion method is implemented for hand gesture pattern classification. Experimental results of 4 subjects demonstrate that each kind of sensor data has its advantages and disadvantages in representing hand gestures. And the proposed method could fuse effectively the complementary information from these three types of sensors for dynamic hand gesture recognition.

An adaptive feature extractor for gesture SEMG recognition

This paper proposes an adaptive feature extraction method for pattern recognition of hand gesture action sEMG to enhance the reusability of myoelectric control. The feature extractor is based on wavelet packet transform and Local Discriminant Basis (LDB) algorithms to select several optimized decomposition subspaces of origin SEMG waveforms caused by hand gesture motions. Then the square roots of mean energy of signal in those subspaces are calculated to form the feature vector. In data acquisition experiments, five healthy subjects implement six kinds of hand motions every day for a week. The recognition results of hand gesture on the basis of the measured SEMG signals from different use sessions demonstrate that the feature extractor is effective. Our work is valuable for the realization of myoelectric control system in rehabilitation and other medical applications.

Simulation Analysis on the MUAPs Classification Using a Combined Detection Method

For exploring the recognition performance of MUAPs, using the synthesized signal, the candidate MUAPs were detected by the combined method of continuous wavelet transform and hypothesis testing, the primary clustering of MUAPs was done by the fuzzy k-means algorithm, and the superimposed waveforms were decomposed by the modeling approach. The experimental results showed that the simulated MUAPs could be correctly extracted and classified by the proposed strategy.

The Decomposition of Surface EMG Signals Based on Blind Source Separation of Convolved Mixtures

The decomposition of surface EMG signals can provide valuable information about the recruitment and firing of motor units from surface EMG recordings. According to the physiologic characteristic of the surface EMG signals generation, a method of the decomposition of SEMG signals based on the technique of convolved mixing blind source separation was proposed. Using simulated SEMG signals, the performance of the decomposition algorithm was analyzed and compared with that of the decomposition technique adopting independent component analysis (ICA). The experiment results show that the proposed method could decompose SEMG signals effectively, and its performance is better than the ICA decomposition method, no matter for the simulated or recorded SEMG signals