Jeroen Lichtenauer

Sign Language Recognition by Combining Statistical DTW and Independent Classification

To recognize speech, handwriting, or sign language, many hybrid approaches have been proposed that combine dynamic time warping (DTW) or hidden Markov models (HMMs) with discriminative classifiers. However, all methods rely directly on the likelihood models of DTW/HMM. We hypothesize that time warping and classification should be separated because of conflicting likelihood modeling demands. To overcome these restrictions, we propose using statistical DTW (SDTW) only for time warping, while classifying the warped features with a different method. Two novel statistical classifiers are proposed - combined discriminative feature detectors (CDFDs) and quadratic classification on DF Fisher mapping (Q-DFFM) - both using a selection of discriminative features (DFs), and are shown to outperform HMM and SDTW. However, we have found that combining likelihoods of multiple models in a second classification stage degrades performance of the proposed classifiers, while improving performance with HMM and SDTW. A proof-of-concept experiment, combining DFFM mappings of multiple SDTW models with SDTW likelihoods, shows that, also for model-combining, hybrid classification can provide significant improvement over SDTW. Although recognition is mainly based on 3D hand motion features, these results can be expected to generalize to recognition with more detailed measurements such as hand/body pose and facial expression.

Exhaustive geometrical search and the false positive watermark detection probability

One way of recovering watermarks in geometrically distorted images is by performing a geometrical search. In addition to the computational cost required for this method, this paper considers the more important problem of false positives. The maximal number of detections that can be performed in a geometrical search is bounded by the maximum false positive detection probability required by the watermark application. We show that image and key dependency in the watermark detector leads to different false positive detection probabilities for geometrical searches for different images and keys. Furthermore, the image and key dependency of the tested watermark detector increases the random-image-random-key false positive detection probability, compared to the Bernoulli experiment that was used as a model.

Isophote properties as features for object detection

Usually, object detection is performed directly on (normalized) gray values or gray primitives like gradients or Haar-like features. In that case the learning of relationships between gray primitives, that describe the structure of the object, is the complete responsibility of the classifier. We propose to apply more knowledge about the image structure in the preprocessing step, by computing local isophote directions and curvatures, in order to supply the classifier with much more informative image structure features. However, a periodic feature space, like orientation, is unsuited for common classification methods. Therefore, we split orientation into two more suitable components. Experiments show that the isophote features result in better detection performance than intensities, gradients or Haar-like features.

Influence of the observation likelihood function on particle filtering performance in tracking applications

Since the introduction of particle filtering for object tracking, a lot of improvements have been suggested. However, the definition of the observation likelihood function, needed for determining the particle weights, has received little attention. Because particle weights determine how the particles are re-sampled, the likelihood function has a strong influence on the tracking performance. We show experimental results for three different tracking tasks for different parameter values of the assumed observation model. The results show a large influence of the model parameters on the tracking performance. Optimizing the likelihood function can give significant tracking improvement. Different optimal parameter settings are observed for the three different tracking tasks. Consequently, when performing multiple tasks a trade-off must be made for the parameter setting. In practical situations where robust tracking must be achieved with a limited amount of particles, the true observation probability is not always the optimal likelihood function.

FPGA accelerator for real-time skin segmentation

Many real-time image processing applications are confronted with performance limitations when implemented in software. The skin segmentation algorithm utilized in hand gesture recognition as developed by the ICT department of Delft University of Technology presents an example of such an application. This paper presents the design of an FPGA based accelerator which alleviates the host PCs computational effort required for real-time skin segmentation. We show that our design utilizes no more than 88% of the resources available within the targeted XC2VP30 device. In addition, the proposed approach is highly portable and not limited to the considered real-time image processing algorithm only

Hiding Correlation-Based Watermark Templates using Secret Modulation

A possible solution to the difficult problem of geometrical distortion of watermarked images in a blind watermarking scenario is to use a template grid in the autocorrelation function. However, the important drawback of this method is that the watermark itself can be estimated and subtracted, or the peaks in the Fourier magnitude spectrum can be removed. A recently proposed solution is to modulate the watermark with a pattern derived from the image content and a secret key. This effectively hides the watermark pattern, making malicious attacks much more difficult. However, the algorithm to compute the modulation pattern is computationally intensive. We propose an efficient implementation, using frequency domain filtering, to make this hiding method more practical. Furthermore, we evaluate the performance of different kinds of modulation patterns. We present experimental results showing the influence of template hiding on detection and payload extraction performance. The results also show that modulating the ACF based watermark improves detection performance when the modulation signal can be retrieved sufficiently accurately. Modulation signals with small average periods between zero crossings provide the most watermark detection improvement. Using these signals, the detector can also make the most errors in retrieving the modulation signal until the detection performance drops below the performance of the watermarking method without modulation.

Person-Independent 3D Sign Language Recognition

In this paper, we present a person independent 3D system for judging the correctness of a sign. The system is camera-based, using computer vision techniques to track the hand and extract features. 3D co-ordinates of the hands and other features are calculated from stereo images. The features are then modeled statistically and automatic feature selection is used to build the classifiers. Each classifier is meant to judge the correctness of one sign. We tested our approach using a 120-sign vocabulary and 75 different signers. Overall, a true positive rate of 96.5% at a false positive rate of 3.5% is achieved. The systems performance in a real-world setting largely agreed with human expert judgement.

Learning to recognize a sign from a single example

We present a method to automatically construct a sign language classifier for a previously unseen sign. The only required input of a new sign is one example, performed by a sign language tutor. The method works by comparing the measurements of the new sign to signs that have been trained on a large number of persons. The parameters of the respective trained classifier models are used to construct a classification model for the new sign. We show that the performance of a classifier constructed from an instructed sign is significantly better than that of dynamic time warping (DTW) with the same sign. Using only a single example, the proposed method has a performance comparable to a regular training with five examples, while being more stable because of the larger source of information.

A learning environment for sign language

We have developed a prototype for a learning environment for deaf and hard of hearing children. This demonstration consists of hands-on experience with the prototype. In total, there are three exercises: 1) an introduction of all pictures and corresponding signs, 2) multiple choice sign-to-picture and 3) performing the sign that corresponds to the picture shown on the screen. The live recognition from a wide-angle stereo camera provides immediate feedback for the third exercise where the sign must be performed.

Acceptability ratings by humans and automatic gesture recognition for variations in sign productions

In this study we compare human and machine acceptability judgments for extreme variations in sign productions. We gathered acceptability judgments of 26 signers and scores of three different automatic gesture recognition (AGR) algorithms that could potentially be used for automatic acceptability judgments, in which case the correlation between human ratings and AGR scores may serve as an dasiaacceptability performancepsila measure. We found high human-human correlations, high AGR-AGR correlations, but low human-AGR correlations. Furthermore, in a comparison between acceptability and classification performance of the different AGR methods, classification performance was found to be an unreliable predictor of acceptability performance.