Bartosz Kunka

UPDRS Tests for Diagnosis of Parkinson's Disease Employing Virtual-Touchpad

This paper presents a new approach to diagnosing Parkinsons disease. The progression of the disease can be measured by the UPDRS (Unified Parkinson Disease Rating Scale) scale which is used to evaluate behavioral and motor symptoms of Parkinsons disease. Hitherto the evaluation of the advancement of the disease in the UPDRS scale is made by a specialist through medical observation. The authors suggest a partial automation of this process, i.e. using a multimodal interface called Virtual-Touchpad (VTP) may support the medical diagnosis. The VTP is based on processing the image of hand acquired by a camera. Therefore, a patient is not constrained by any equipment. Two types of UPDRS tests that can be supported by VTP are presented in the paper. In the Finger Taps test the patient taps thumb with the index finger in rapid succession. The Rapid Alternating Movement of Hands test consists in pronation-supination movements of hands. VTP captures the image of the patients hand while performing the test and, based on this image, it assigns hand gesture to the given class. The hand movement velocity is measured and used in the diagnostic process.

Gaze-tracking-based audio-visual correlation analysis employing quality of experience methodology

This paper investigates a new approach to audio-visual correlation assessment based on the gaze-tracking system developed at the Multimedia Systems Department (MSD) of Gdansk University of Technology (GUT). The gaze-tracking methodology, having roots in Human-Computer Interaction borrows the relevance feedback through gaze-tracking and applies it to the new area of interests, which is Quality of Experience (QoE). Results of subjective tests carried out at the MSD showed a strong dependency between video presented in the screen and the perceived audio. It has also been shown that the application of gaze-tracking to the audio-visual correlation analysis allows for the objectivization of results obtained in subjective tests. Therefore this research study concentrates on the possibility to apply this methodology to the area of Quality of Experience.

Gaze-tracking and Acoustic Vector Sensors Technologies for PTZ Camera Steering and Acoustic Event Detection

An innovative application of gaze-tracking and acoustic vector sensors (AVS) technologies for guidance of moving pan-tilt-zoom (PTZ) monitoring camera is presented. Gaze-tracking is used to steer and to zoom the camera to the gaze focus area. Additionally, it is combined with audio processing in two scenarios. First is called “audio slave”: directional acoustic monitoring is adjusted automatically to the camera direction. Second is called “audio master”: automatic detection of sound events directions is performed to take priority over user control and steer the camera towards sound source. An approach to gaze tracking is presented, utilizing new algorithmic methods for both image processing and PTZ camera steering. Then application of AVS for directional filtering of sound, and for detection of acoustic events direction is discussed. The implemented application is described, and user experience is reported. Finally, future work is discussed.

An new method of audio-visual correlation analysis

This paper presents a new methodology of conducting the audio-visual correlation analysis employing the gaze tracking system. Interaction between two perceptual modalities, seeing and hearing, their interaction and mutual reinforcement in a complex relationship was a subject of many research studies. Earlier stage of the carried out experiments at the Multimedia Systems Department (MSD) showed that there exists a relationship between perception of video presented in the screen and accompanying audio signals, both stereo and spatial. These results were based on subjective tests. Applying the gaze tracking system to the subjective domain may be a step towards objectivization of results obtained during such tests. In the paper, first a short review of audio-visual correlation examination methods is presented. Then, a gaze tracking system engineered at the MSD is shortly presented. The system calibration is also shown. Assumptions of the preliminary experiments are outlined, and the realization of some preliminary tests is described. Conclusions concerning the objective methodology of audio-visual correlation analysis proposed are also included.

Multimodal human-computer interfaces based on advanced video and audio analysis

Multimodal interfaces development history is reviewed briefly in the introduction. Examples of applications of multimodal interfaces to education software and for the disabled people are presented, including interactive electronic whiteboard based on video image analysis, application for controlling computers with mouth gestures and the audio interface for speech stretching for hearing impaired and stuttering people. The Smart Pen providing a tool for supporting therapy of developmental dyslexia is presented and results achieved with its application are discussed. The eye-gaze tracking system named “Cyber-Eye” developed at the Multimedia Systems Department employed to many kinds of experiments is presented including analysis of visual activity of patients remaining in vegetative state and their awareness evaluation. The scent emitting multimodal computer interface provides an important supplement of the polysensoric stimulation process, playing an essential role in education and therapy of children with certain developmental disorders. A new approach to diagnosing Parkinsons disease is shown. The progression of the disease can be measured employing the UPDRS (Unified Parkinson Disease Rating Scale) scale which is used to evaluate motor and behavioral symptoms of Parkinsons disease, based on the multimodal interface called Virtual-Touchpad (VTP) used for supporting medical diagnosis. The paper is concluded with some general remarks concerning the role of multimodal computer interfaces applied to learning, therapy and everyday usage of computerized devices.

Eye-Gaze Tracking-Based Telepresence System for Videoconferencing

An approach to the teleimmersive videoconferencing system enhanced by the pan-tilt-zoom (PTZ) camera, controlled by the eye-gaze tracking system, is presented in this paper. An overview of the existing telepresence systems, especially dedicated to videoconferencing is included. The presented approach is based on the CyberEye eye-gaze tracking system engineered at the Multimedia Systems Department (MSD) of Gdansk University of Technology (GUT), as well as on a standard PTZ security camera communicating with the computer by the TCP/IP protocol. Technical aspects of the developed system prototype including two different use cases (one-way and two-way configuration of system) are described. Moreover, a discussion related to the gathered user’s experience as well as to difficulties and opportunities concerning the proposed approach are included.

Consciousness Study of Subjects with Unresponsive Wakefulness Syndrome Employing Multimodal Interfaces

The paper presents a novel multimodal-based methodology for consciousness study of individuals with unresponsive wakefulness syndrome. Two interfaces were employed in the experiments: eye gaze tracking system – CyberEye developed at the Multimedia Systems Department, and EEG device with electrode placement in the international 10-20 standard. It was a pilot study for checking if it is possible to determine objective methods based on multimodal techniques which could replace or support current expensive and difficult to access neuroimaging techniques, like fMRI, PET, utilizing in evaluation of consciousness state. The multimodal-based methodology consists of several phases of research involving subjects. Hearing examination based on objective methods (OAE, ABR), consciousness test based on analysis of visual activity, examination of visual neural pathway with Steady State Visually Evoked Potentials and EEG-based comprehension test were proposed. The results obtained within conducted experiments and presented in this paper suggest that proposed objective-subjective methodology could potentially be introduced into clinical facilities after further validation.