Mahmoud Elmezain

A Hidden Markov Model-based continuous gesture recognition system for hand motion trajectory

In this paper, we propose an automatic system that recognizes both isolated and continuous gestures for Arabic numbers (0-9) in real-time based on hidden Markov model (HMM). To handle isolated gestures, HMM using ergodic, left-right (LR) and left-right banded (LRB) topologies with different number of states ranging from 3 to 10 is applied. Orientation dynamic features are obtained from spatio-temporal trajectories and then quantized to generate its codewords. The continuous gestures are recognized by our novel idea of zero-codeword detection with static velocity motion. Therefore, the LRB topology in conjunction with forward algorithm presents the best performance and achieves average rate recognition 98.94% and 95.7% for isolated and continuous gestures, respectively.

Hand trajectory-based gesture spotting and recognition using HMM

In this paper, we propose an automatic system that executes hand gesture spotting and recognition simultaneously without any time delay based on Hidden Markov Models (HMM). Our system is based on three main stages; preprocessing, feature extraction and classification. In preprocessing stage, color and 3D depth map are used to detect hands. The hand trajectory will take place in further steps using Mean-shift algorithm and Kalman filter. The second stage, Orientation dynamic features are obtained from spatio-temporal trajectories and then are quantized to generate its codewords. In the final stage, the gestures are segmented by finding the start and the end points of meaningful gestures that are embedded in the input stream and then are recognized by Viterbi algorithm. Experimental results demonstrate that, our system can successfully recognize spotted hand gestures with a 95.87% recognition rate for Arabic numbers from 0 to 9.

A Robust Method for Hand Gesture Segmentation and Recognition Using Forward Spotting Scheme in Conditional Random Fields

This paper proposes a forward spotting method that handles hand gesture segmentation and recognition simultaneously without time delay. To spot meaningful gestures of numbers (0-9) accurately, a stochastic method for designing a non-gesture model using Conditional Random Fields (CRFs) is proposed without training data. The non-gesture model provides a confidence measures that are used as an adaptive threshold to find the start and the end point of meaningful gestures. Experimental results show that the proposed method can successfully recognize isolated gestures with 96.51% and meaningful gestures with 90.49% reliability.

Robust methods for hand gesture spotting and recognition using Hidden Markov Models and Conditional Random Fields

This paper proposes an automatic method that handles hand gesture spotting and recognition simultaneously. To spot meaningful gestures of numbers (0–9) accurately, a stochastic method for designing a non-gesture model with Hidden Markov Models (HMMs) versus Conditional Random Fields (CRFs) is proposed without training data. The non-gesture model provides a confidence measure that is used as an adaptive threshold to find the start and the end point of meaningful gestures, which are embedded in the input video stream. To reduce the states number of the non-gesture model with HMMs, similar probability distributions states are merged based on relative entropy measure. Additionally, the weights of self-transition feature functions are increased for short gesture to further improve the accuracy of gesture spotting and recognition with CRFs. Experimental results show that; the proposed method can successfully spot and recognize meaningful gestures with 93.31% and 90.49% reliability for HMMs and CRFs respectively. In addition, the model inference by HMMs are faster and the saving time is 66.42% using relative entropy. The reliability of CRFs method is improved from 86.12% to 90.49% using short gesture detector.

Data Gathering for Gesture Recognition Systems Based on Mono Color-, Stereo Color- and Thermal Cameras

In this paper, we present our results to build an automatic gesture recognition system using different types of cameras to compare them in reference to their features for segmentation. Normally, the images of a mono color camera system are mostly used as input data in the research area of gesture recognition. In comparison to that, the analysis results of a stereo color camera and a thermal camera system are used to determine the advantages and disadvantages of these camera systems. With this basics, a real-time gesture recognition system is build to classify alphabets (A-Z) and numbers (0-9) with an average recognition rate of 98% using Hidden Markov Models (HMM).

Gesture Recognition for Alphabets from Hand Motion Trajectory Using Hidden Markov Models

This paper describes a method to recognize the alphabets from a single hand motion using Hidden Markov Models (HMM). In our method, gesture recognition for alphabets is based on three main stages; preprocessing, feature extraction and classification. In preprocessing stage, color and depth information are used to detect both hands and face in connection with morphological operation. After the detection of the hand, the tracking will take place in further step in order to determine the motion trajectory so-called gesture path. The second stage, feature extraction enhances the gesture path which gives us a pure path and also determines the orientation between the center of gravity and each point in a pure path. Thereby, the orientation is quantized to give a discrete vector that used as input to HMM. In the final stage, the gesture of alphabets is recognized by using Left-Right Banded model (LRB) in conjunction with Baum-Welch algorithm (BW) for training the parameters of HMM. Therefore, the best path is obtained by Viterbi algorithm using a gesture database. In our experiment, 520 trained gestures are used for training and also 260 tested gestures for testing. Our method recognizes the alphabets from A to Z and achieves an average recognition rate of 92.3%.

Spatio-temporal feature extraction-based hand gesture recognition for isolated American Sign Language and Arabic numbers

This paper proposes a system to recognize isolated American Sign Language and Arabic numbers in real-time from stereo color image sequences using Hidden Markov Models (HMMs). Our system is based on three main stages; preprocessing, feature extraction and classification. In preprocessing stage, color and 3D depth map are used to detect and track the hand. The second stage, 3D combined features of location, orientation and velocity with respected to Cartesian and Polar systems are used. Additionally, k-means clustering is employed for HMMs code-word. In the final stage, the hand gesture path is recognized using Left-Right Banded topology (LRB) in conjunction Viterbi path. Experimental results demonstrate that, our system can successfully recognize isolated hand gestures with 98.33% recognition rate.

Improving Hand Gesture Recognition Using 3D Combined Features

In this paper, we propose a system to recognize alphabet characters (A-Z) and numbers (0-9) in real-time from stereo color image sequences using Hidden Markov Models (HMMs). Additionally, a robust method for hand tracking in a complex environment using Mean-shift analysis in conjunction with 3D depth map is introduced. The depth information solve the overlapping problem between hands and face, which is obtained by passive stereo measuring based on cross correlation and the known calibration data of the cameras. 3D combined features of location, orientation and velocity with respected to Cartesian systems are used. And then, k-means clustering is employed for HMMs codeword. The hand gesture path is recognized using Left-Right Banded topology (LRB) in conjunction Viterbi path. Experimental results demonstrate that, our system can successfully recognize hand gestures with 98.33% recognition rate.

Discriminative Models-Based Hand Gesture Recognition

In this paper, we study the discriminative models like CRFs, HCRFs and LDCRFs to recognize alphabet characters (A-Z) and numbers (0-9) in real-time from stereo color image sequences. To handle isolated gesture, CRFs, HCRFs and LDCRFs with different number of window size are applied on 3D combined features of location, orientation and velocity. The gesture recognition rate is improved initially as the window size increase, but degrades as window size increase further. In contrast to generative approaches such as HMMs, experimental results show that the LDCRFs are the best in terms of results than CRFs, HCRFs and HMMs at window size equal 4. Additionally, our results show that; an overall recognition rates are 91.52%, 95.28% and 98.05% for CRFs, HCRFs, and LDCRFs respectively.

Human activity recognition via temporal moment invariants

Temporal invariant shape moments intuitively seem to provide an important visual cue to human activity recognition in video sequences. In this paper, an SVM based method for human activity recognition is introduced. With this method, the feature extraction is carried out based on a small number of computationally-cheap invariant shape moments. When tested on the popular KTH action dataset, the obtained results are promising and compare favorably with that reported in the literature. Furthermore our proposed method achieves real-time performance, and thus can provide latency guarantees to real-time applications and embedded systems.