Tamer Shanableh

Spatio-Temporal Feature-Extraction Techniques for Isolated Gesture Recognition in Arabic Sign Language

This paper presents various spatio-temporal feature-extraction techniques with applications to online and offline recognitions of isolated Arabic Sign Language gestures. The temporal features of a video-based gesture are extracted through forward, backward, and bidirectional predictions. The prediction errors are thresholded and accumulated into one image that represents the motion of the sequence. The motion representation is then followed by spatial-domain feature extractions. As such, the temporal dependencies are eliminated and the whole video sequence is represented by a few coefficients. The linear separability of the extracted features is assessed, and its suitability for both parametric and nonparametric classification techniques is elaborated upon. The proposed feature-extraction scheme was complemented by simple classification techniques, namely, K nearest neighbor (KNN) and Bayesian, i.e., likelihood ratio, classifiers. Experimental results showed classification performance ranging from 97% to 100% recognition rates. To validate our proposed technique, we have conducted a series of experiments using the classical way of classifying data with temporal dependencies, namely, hidden Markov models (HMMs). Experimental results revealed that the proposed feature-extraction scheme combined with simple KNN or Bayesian classification yields comparable results to the classical HMM-based scheme. Moreover, since the proposed scheme compresses the motion information of an image sequence into a single image, it allows for using simple classification techniques where the temporal dimension is eliminated. This is actually advantageous for both computational and storage requirements of the classifier

H.264/AVC to HEVC Video Transcoder Based on Dynamic Thresholding and Content Modeling

The new video coding standard, High Efficiency Video Coding (HEVC), was developed to succeed the current standard, H.264/AVC, as the state of the art in video compression. However, there is a lot of legacy content encoded with H.264/AVC. This paper proposes and evaluates several transcoding algorithms from the H.264/AVC to the HEVC format. In particular, a novel transcoding architecture, in which the first frames of the sequence are used to compute the parameters so that the transcoder can learn the mapping for that particular sequence, is proposed. Then, two types of mode mapping algorithms are proposed. In the first solution, a single H.264/AVC coding parameter is used to determine the outgoing HEVC partitions using dynamic thresholding. The second solution uses linear discriminant functions to map the incoming H.264/AVC coding parameters to the outgoing HEVC partitions. This paper contains experiments designed to study the impact of the number of frames used for training in the transcoder. Comparisons with existing transcoding solutions reveal that the proposed work results in lower rate-distortion loss at a competitive complexity performance.

Data Hiding in MPEG Video Files Using Multivariate Regression and Flexible Macroblock Ordering

This paper proposes two data hiding approaches using compressed MPEG video. The first approach hides message bits by modulating the quantization scale of a constant bitrate video. A payload of one message bit per macroblock is achieved. A second order multivariate regression is used to find an association between macroblock-level feature variables and the values of a hidden message bit. The regression model is then used by the decoder to predict the values of the hidden message bits with very high prediction accuracy. The second approach uses the flexible macroblock ordering feature of H.264/AVC to hide message bits. Macroblocks are assigned to arbitrary slice groups according to the content of the message bits to be hidden. A maximum payload of three message bits per macroblock is achieved. The proposed solutions are analyzed in terms of message extraction accuracy, message payload, excessive bitrate and quality distortion. Comparisons with previous work reveal that the proposed solutions are superior in terms of message payload while causing less distortion and compression overhead.

MPEG-2 to HEVC Video Transcoding With Content-Based Modeling

This paper proposes an efficient MPEG-2 to High Efficiency Video Coding (HEVC) video transcoder. The objective of the transcoder is to migrate the abundant MPEG-2 video content to the emerging HEVC video coding standard. The transcoder introduces a content-based machine learning solution to predict the depth of the HEVC coding units. The proposed transcoder utilizes full re-encoding to find a mapping between the incoming MPEG-2 coding information and the outgoing HEVC depths of the coding units. Once the model is built, a switch to transcoding mode occurs. Hence, the model is content based and varies from one video sequence to another. The transcoder is compared against full re-encoding using the default HEVC fast motion estimation. Using HEVC test sequences, it is shown that a speedup factor of up to 3 is achieved, while reducing the bitrate of the incoming video by around 50%. In comparison to full re-encoding, an average of 3.9% excessive bitrate is encountered with an average PSNR drop of 0.1 dB. Since this is the first work to report on MPEG-2 to HEVC video transcoding, the reported results can be used as a benchmark for future transcoding research.

Hybrid DCT/pixel domain architecture for heterogeneous video transcoding

Abstract In this paper, the pixel domain heterogeneous video transcoder proposed by the authors in Shanableh and Ghanabari (IEEE Trans. Multimedia 2(2) (2000) 101) is implemented in the DCT domain. Consequently, the motion compensation (MC) and its inverse and the image down-sampling functions of the pixel domain transcoder are implemented in the frequency domain whilst eliminating the DCT and IDCT pairs. Moreover, the paper proposes two transcoding architectures. In one, the transcoder is simplified by implementing both its MC loops in the DCT domain. While in the other, image decimation is realised through a modified inverse transformation of the top left 4×4 coefficients. The input and output domains of the mentioned decimator render the decoders and the encoders MC loops to be in the DCT domain and the pixel domain respectively. This results in a unique hybrid DCT, pixel domain transcoding architecture. Various methods for accelerating the process of the DCT domain MC are reviewed and classified into lossless and lossy methods. It is shown that both picture quality and performance are enhanced by utilising shared information with successive motion compensated macroblocks. The superiority of the hybrid architecture is then assessed in terms of preserving image quality, feasible functionality and tolerance to lossy acceleration of the DCT domain MC.

Feature modeling using polynomial classifiers and stepwise regression

In polynomial networks, feature vectors are mapped to a higher dimensional space through a polynomial function. The expanded vectors are then passed to a single layer network to compute the model parameters. However, as the dimensionality of the feature vectors grows with polynomial expansion, polynomial training and classification become impractical due to the prohibitive number of expanded variables. This problem is more prominent in vision-based systems where high dimensionality feature vectors are extracted from digital images and/or video. In this paper we propose to reduce the dimensionality of the expanded vector through the use of stepwise regression. We compare our work to the reduced-model multinomial networks where the dimensionality of the expanded feature vectors grows linearly whilst preserving the classification ability. We also compare the proposed work to standard polynomial classifiers and to established techniques of polynomial classifiers with dimensionality reduction. Two application scenarios are used to test the proposed solution, namely; image-based hand recognition and video-based recognition of isolated sign language gestures. Various datasets from the UCI machine learning repository are also used for testing. Experimental results illustrate the effectiveness of the proposed dimensionality reduction technique in comparison to published methods.

Glove-Based Continuous Arabic Sign Language Recognition in User-Dependent Mode

In this paper, we propose a glove-based Arabic sign language recognition system using a novel technique for sequential data classification. We compile a sensor-based dataset of 40 sentences using an 80-word lexicon. In the dataset, hand movements are captured using two DG5-VHand data gloves. Data labeling is performed using a camera to synchronize hand movements with their corresponding sign language words. Low-complexity preprocessing and feature extraction techniques are applied to capture and emphasize the temporal dependence of the data. Subsequently, a Modified k-Nearest Neighbor (MKNN) approach is used for classification. The proposed MKNN makes use of the context of feature vectors for the purpose of accurate classification. The proposed solution achieved a sentence recognition rate of 98.9%. The results are compared against an existing vision-based approach that uses the same set of sentences. The proposed solution is superior in terms of classification rates while eliminating restrictions of vision-based systems.

Detection of frame deletion for digital video forensics

The abundance of digital video forms a potential piece of evidence in courtrooms. Augmenting subjective assessment of digital video evidence by an automated objective assessment helps increase the accuracy of deciding whether or not to admit the digital video as legal evidence. This paper examines the authenticity of digital video evidence and in particular it proposes a machine learning approach to detecting frame deletion. A number of discriminative features are extracted from the video bit stream and its reconstructed images. The features are based on prediction residuals, percentage of intra-coded macroblocks, quantization scales and reconstruction quality. The importance of these features is verified by using stepwise regression. Consequently, the dimensionality of the feature vectors is reduced using spectral regression where it is shown that the projected features of unaltered and forged videos are nearly separable. Machine learning techniques are used to report the true positive and false negative rates of the proposed solution. It is shown that the proposed solution works for detecting forged videos regardless of the number of deleted frames, as long as it is not a multiple of the length of a group of pictures. It is also shown that the proposed solution is applicable for the two modes of video compression, variable and constant bitrate coding.

User-independent recognition of Arabic sign language for facilitating communication with the deaf community

This paper presents a solution for user-independent recognition of isolated Arabic sign language gestures. The video-based gestures are preprocessed to segment out the hands of the signer based on color segmentation of the colored gloves. The prediction errors of consecutive segmented images are then accumulated into two images according to the directionality of the motion. Different accumulation weights are employed to further help preserve the directionality of the projected motion. Normally, a gesture is represented by hand movements; however, additional user-dependent head and body movements might be present. In the user-independent mode we seek to filter out such user-dependent information. This is realized by encapsulating the movements of the segmented hands in a bounding box. The encapsulated images of the projected motion are then transformed into the frequency domain using Discrete Cosine Transformation (DCT). Feature vectors are formed by applying Zonal coding to the DCT coefficients with varying cutoff values. Classification techniques such as KNN and polynomial classifiers are used to assess the validity of the proposed user-independent feature extraction schemes. An average classification rate of 87% is reported.

Continuous Arabic Sign Language Recognition in User Dependent Mode

Existing work on Arabic Sign Language recognition focuses on finger spelling and isolated gestures. In this work we extend vision-based existing solutions to recognition of continuous signing. As such we have collected and labeled the first video-based continuous Arabic Sign Language dataset. We intend to make the collected dataset available for the research community. The proposed solution extracts the motion from the video-based sentences by means of thresholding the forward prediction error between consecutive images. Such prediction errors are then transformed into the frequency domain and Zonal coded. We use Hidden Markov Models for model training and classification. The experimental results show an average word recognition rate of 94%, keeping in the mind the use of a high perplexity vocabulary and unrestrictive grammar.