Michał Lech

Fuzzy Rule-Based Dynamic Gesture Recognition Employing Camera and Multimedia Projector

In this chapter the system based on camera and multimedia projector enabling a user to control computer applications by dynamic hand gestures is presented. The main objective is to present the gesture recognition methodology which bases on representing hand movement trajectory by motion vectors analyzed using fuzzy rule-based inference. The approach was engineered in the system developed with J2SE and C++ / OpenCV technology. OpenCV was used for image processing and J2SE with jFuzzyLogic package for gesture interpretation. Results of fuzzy rule-based and fixed threshold-based gesture recognition effectiveness are provided. As an example of system usage the so-called Interactive Whiteboard application is presented. Details on the application engineered are provided in the context of fuzzy inference processing.

Hand gesture recognition supported by fuzzy rules and Kalman filters

The paper presents a system based on camera and multimedia projector enabling a user to control computer applications by dynamic hand gestures. Gesture recognition methodology based on representing hand movement trajectory by motion vectors analysed using fuzzy rule-based inference is first given. For effective hand position tracking Kalman filters are employed. The system engineered is developed using J2SE and C++/OpenCV technology. In addition, OpenCV is used for image processing and J2SE with jFuzzyLogic package is employed for gesture interpretation. Results of fuzzy rule-based and fixed threshold-based gesture recognition effectiveness are provided. Additionally, for fuzzy rule-based gesture recognition the system efficacy after utilising Kalman filters is examined. The so-called interactive whiteboard application is given as an example of the system usage.

Examining Classifiers Applied to Static Hand Gesture Recognition in Novel Sound Mixing System

The main objective of the chapter is to present the methodology and results of examining various classifiers (Nearest Neighbor-like algorithm with non-nested generalization (NNge), Naive Bayes, C4.5 (J48), Random Tree, Random Forests, Artificial Neural Networks (Multilayer Perceptron), Support Vector Machine (SVM) used for static gesture recognition. A problem of effective gesture recognition is outlined in the context of the system based on a camera and a multimedia projector enabling a user to process sound in audio mixing domain by hand gestures. The image processing method and hand shape parameterization method are described in relation to the specificity of the input and data classifiers. The SVM classifier is considered the optimum choice for the engineered gesture-based sound mixing system.

Pilot testing of developed multimodal biometric identity verification system

The bank client identity verification system developed in the course of the IDENT project is presented. The total number of five biometric modalities including: dynamic signature proofing, voice recognition, face image verification, face contour extraction and hand blood vessels distribution comparison have been developed and studied. The experimental data were acquired employing multiple biometric sensors installed at engineered biometric terminals. The biometric portraits of 125 subjects were registered and stored in the database during the presented pilot study and then verified experimentally. The analysis of FAR and FRR measures obtained for developed biometric applications was made. Problem-specific survey was done on the basis of questionnaires completed by the subjects in order to assess the look and feel of the developed biometric system as well as to collect opinions concerning its implementation in banking outlets. A discussion concerning the quality of registered signals and results achieved in the pilot study is included.

A handwritten signature verification method employing a tablet

A signature verification system based on static features and time-domain functions of signals obtained using a tablet has been presented in the paper. The signature verification method, based mainly on dynamic time warping coupled with some signature image features, has been described. The FRR measures reflecting the methods efficiency have been evaluated for verification attempts performed directly after obtaining model signatures and for reiterated attempts made after two days. The FAR measures have been assessed both: for simple and for skilled forgeries. Obtained results are presented and discussed in the paper.

Multimodal human-computer interfaces based on advanced video and audio analysis

Multimodal interfaces development history is reviewed briefly in the introduction. Examples of applications of multimodal interfaces to education software and for the disabled people are presented, including interactive electronic whiteboard based on video image analysis, application for controlling computers with mouth gestures and the audio interface for speech stretching for hearing impaired and stuttering people. The Smart Pen providing a tool for supporting therapy of developmental dyslexia is presented and results achieved with its application are discussed. The eye-gaze tracking system named “Cyber-Eye” developed at the Multimedia Systems Department employed to many kinds of experiments is presented including analysis of visual activity of patients remaining in vegetative state and their awareness evaluation. The scent emitting multimodal computer interface provides an important supplement of the polysensoric stimulation process, playing an essential role in education and therapy of children with certain developmental disorders. A new approach to diagnosing Parkinsons disease is shown. The progression of the disease can be measured employing the UPDRS (Unified Parkinson Disease Rating Scale) scale which is used to evaluate motor and behavioral symptoms of Parkinsons disease, based on the multimodal interface called Virtual-Touchpad (VTP) used for supporting medical diagnosis. The paper is concluded with some general remarks concerning the role of multimodal computer interfaces applied to learning, therapy and everyday usage of computerized devices.

Virtual whiteboard: a gesture-controlled pen-free tool emulating school whiteboard

In the paper the so-called Virtual Whiteboard is presented which may be an alternative solution for modern electronic whiteboards based on electronic pens and sensors. The presented tool enables the user to write, draw and handle whiteboard contents using his/her hands only. An additional equipment such as infrared diodes, infrared cameras or cyber gloves is not needed. The users interaction with the Virtual Whiteboard computer application is based on dynamic hand gesture recognition. Gestures are recognized in the process of analyzing video stream obtained from a webcam coupled with a multimedia projector displaying whiteboard contents. The tracking positions of hands in the image is supported by Kalman filtering. In the paper the hardware and software of the Virtual Whiteboard is presented with a special focus on utilizing Kalman filters for prediction of consecutive hand positions. For the gestures applied to handle whiteboard contents, examined efficacy of Kalman filter supported recognition and the efficacy without using the filtering is given. In addition, the results of system efficiency tests are provided.

Comparison of Classification Methods for EEG Signals of Real and Imaginary Motion

The classification of EEG signals provides an important element of brain-computer interface (BCI) applications, underlying an efficient interaction between a human and a computer application. The BCI applications can be especially useful for people with disabilities. Numerous experiments aim at recognition of motion intent of left or right hand being useful for locked-in-state or paralyzed subjects in controlling computer applications. The chapter presents an experimental study of several methods for real motion and motion intent classification (rest/upper/lower limbs motion, and rest/left/right hand motion). First, our approach to EEG recordings segmentation and feature extraction is presented. Then, 5 classifiers (Naive Bayes, Decision Trees, Random Forest, Nearest-Neighbors NNge, Rough Set classifier) are trained and tested using examples from an open database. Feature subsets are selected for consecutive classification experiments, reducing the number of required EEG electrodes. Methods comparison and obtained results are presented, and a study of features feeding the classifiers is provided. Differences among participating subjects and accuracies for real and imaginary motion are discussed. It is shown that though classification accuracy varies from person to person, it could exceed 80% for some classifiers.

The Project IDENT: Multimodal Biometric System for Bank Client Identity Verification

Biometric identity verification methods are implemented inside a real banking environment comprising: dynamic handwritten signature verification, face recognition, bank client voice recognition and hand vein distribution verification. A secure communication system based on an intra-bank client-server architecture was designed for this purpose. Hitherto achieved progress within the project is reported in this paper with a focus on the design and implementation of the developed biometric authentication system. Implemented multimodal biometric client identity verification methods are briefly outlined and results of hitherto obtained biometric sample acquisition and analysis are reported.

Validating data acquired with experimental multimodal biometric system installed in bank branches

An experimental system was engineered and implemented in 100 copies inside a real banking environment comprising: dynamic handwritten signature verification, face recognition, bank client voice recognition and hand vein distribution verification. The main purpose of the presented research was to analyze questionnaire responses reflecting user opinions on: comfort, ergonomics, intuitiveness and other aspects of the biometric enrollment process. The analytical studies and experimental work conducted in the course of this work will lead towards methodologies and solutions of the multimodal biometric technology, which is planned for further development. Before this stage is achieved a study on the data usefulness acquired from a variety of biometric sensors and from survey questionnaires filled in by banking tellers and clients was done. The decision-related sets were approximated by the Rough Set method offering efficient algorithms and tools for finding hidden patterns in data. Prediction of evaluated biometric data quality, based on enrollment samples and on user subjective opinions was made employing the developed method. After an introduction to the principles of applied biometric identity verification methods, the knowledge modelling approach is presented together with achieved results and conclusions.