Piotr Odya

Smart pen - new multimodal computer control tool for graphomotorical therapy

Numerous researches indicate that dyslexia and dysgraphia are nowadays major problems in schools. Smart Pen is a tool for supporting the therapy of developmental dyslexia, with particular regard to dysgraphia. It comprises a display monitor equipped with a high-sensitivity touchpad and specially designed writing tool equipped with pressure sensors. The paper put emphasis on issues related to the design of the device and the development of software providing a vital part of the interface. The software allows monitoring some interface parameters that are important from the therapy point of view, such as pen grip or time taken to complete an activity being a part of an exercise. The interface designed allows interesting (play and learn) activities to be performed with kids (ie. learning of proper handling writing tools, basic writing etc.). Tests have been carried out to verify usability of the Smart Pen. The test results showed that children and therapists are keen on using the new tool. Furthermore, using Smart Pen it is possible to distinguish children without writing problems from those who have some motoric disruptions. A description of the tests carried out and of their results is included in the paper.

Determining Influence of Visual Cues on the Perception of Surround Sound Using Soft Computing

Contemporary digital video, film or multimedia presentations are often accompanied by the surround sound. Techniques and standards involved in digital video processing are much more developed than concepts underlying creating recording and mixing of the multichannel sound. The main challenge in the sound processing in the multichannel system is to create an appropriate basis for the relating multimodal context of visual and sound domains. Therefore, one of the purposes of experiments is to study in which way and how the surround sound interferes or is associated with the visual context. This kind of study was hitherto carried out when two-channel sound technique was associated with a stereo TV. However, there is not much study done yet that associates surround sound and digital video presented at the TV screen. The main issue in such experiments is the analysis of the influence of visual cues on perception of the surround sound. This problem will be solved with the application of fuzzy logic to the processing of subjective test results.

Influence of visual cues on the perception of surround sound

Contemporary digital video, film, or multimedia presentations are often accompanied by the surround sound. Techniques and standards involved in digital video processing are much more developed than concepts underlying creating recording and mixing of the multichannel sound. The main challenge in the sound processing in the multichannel system is to create an appropriate basis for connecting multimodal context of visual and sound domains. Therefore one of the purposes of experiments is to study in which way and how the surround sound interferes or is associated with the visual context. This kind of study was hitherto carried out when two‐channel sound technique was associated with a stereo TV. However, there are not many studies done yet that associate surround sound and digital video presented at the TV screen. The main issue in such experiments is the analysis of the influence of visual cues on perception of the surround sound. This problem will be addressed in the paper.

Multimodal human-computer interfaces based on advanced video and audio analysis

Multimodal interfaces development history is reviewed briefly in the introduction. Examples of applications of multimodal interfaces to education software and for the disabled people are presented, including interactive electronic whiteboard based on video image analysis, application for controlling computers with mouth gestures and the audio interface for speech stretching for hearing impaired and stuttering people. The Smart Pen providing a tool for supporting therapy of developmental dyslexia is presented and results achieved with its application are discussed. The eye-gaze tracking system named “Cyber-Eye” developed at the Multimedia Systems Department employed to many kinds of experiments is presented including analysis of visual activity of patients remaining in vegetative state and their awareness evaluation. The scent emitting multimodal computer interface provides an important supplement of the polysensoric stimulation process, playing an essential role in education and therapy of children with certain developmental disorders. A new approach to diagnosing Parkinsons disease is shown. The progression of the disease can be measured employing the UPDRS (Unified Parkinson Disease Rating Scale) scale which is used to evaluate motor and behavioral symptoms of Parkinsons disease, based on the multimodal interface called Virtual-Touchpad (VTP) used for supporting medical diagnosis. The paper is concluded with some general remarks concerning the role of multimodal computer interfaces applied to learning, therapy and everyday usage of computerized devices.

Smart pen: new multimodal computer control tool for dyslexia therapy

Dyslexia (dysgraphia) therapy is often boring for children and, what even worse, its results can be unsatisfactory. Hence, many therapists insist on development new methods which would be more interesting for young patients. The Smart Pen is such a tool. It is designed for supporting the therapy of developmental dyslexia, with particular regard to dysgraphia.

Measurements and visualization of sound field distribution around organ pipe

Measurements and visualization of acoustic field around an organ pipe are presented. Sound intensity technique was applied for this purpose. Measurements were performed in free field. The organ pipe was activated with a constant air flow, produced by an external compressor, aimed at obtaining long-term steady state responses of generated acoustic signal. Sound energy distribution was measured in a defined fixed grid of points by means of multichannel acoustic vector sensor. For a precise positioning of the acoustic probe the specialized Cartesian robot was applied. Obtained results made it possible to produce visualizations of sound intensity distribution of radiated acoustic energy around the considered organ pipe.

An Approach to Determining Tinnitus Acoustical Characteristic

For many treatment methods, accurate estimation of Tinnitus (ringing in ears) concerning sound type, level, and bandwidth or frequency is inevitable. The proposed way of obtaining Tinnitus parameters is described in this paper. The method employs sound synthesis, aimed at obtaining sound which is closest to perceived Tinnitus. The proposed method assumes running a designed application on a multimedia PC provided with a special graphical user interface to facilitate sound generation and identification. Emphasis is put on issues related to the implementation of the proposed diagnostic procedure. The method was verified during preliminary tests in which people suffering from Tinnitus participated. The obtained results are presented and discussed in this paper.

Determining the influence of visual stimuli on the peception of surround sound using data mining algorithms

A short description of experiments that aim to determine visual cues influence on the perception of spatial sound is provided in the paper. The earlier stage of the carried out experiments showed that there exists a relationship between the perception of video presented in the screen and sound signals reproduced in a surround system. However, this relationship is dependent on the type of audio–visual signals. Thus a series of subjective tests has been performed on dozens of experts in order to discover these dependencies. The main issue in such experiments is the analysis of the influence of visual cues on the perception of the surround sound. This problem is solved with the application of genetic algorithm and rule searching mechanism to the processing of subjective test results. Some results and conclusions concerning the complexity of the investigated problem are included. [Research sponsored by the Committee for Scientific Research, Warsaw, Poland, Grant No. 8 T11D 00218.]

Multimodal System for Diagnosis and Polysensory Stimulation of Subjects with Communication Disorders

An experimental multimodal system, designed for polysensory diagnosis and stimulation of persons with impaired communication skills or even non-communicative subjects is presented. The user interface includes an eye tracking device and the EEG monitoring of the subject. Furthermore, the system consists of a device for objective hearing testing and an autostereoscopic projection system designed to stimulate subjects through their immersion in a virtual environment. Data analysis methods are described, and experiments associated with classification of mental states during listening exercises as well as audio-visual stimuli are presented and discussed. Feature extraction was based on discrete wavelet transformation and clustering employing the k-means algorithm was designed. All algorithms were implemented in the Python programming language with the use of Open Source libraries. Tests of the proposed system were performed in a Special School and Educational Center in Kościerzyna, Poland. Results and comparison with data gathered from the control group of healthy people are presented and discussed.

Survey on Applications of Multimedia Technology to Examine Impact of Roadside Advertising on Drivers

The correct location of ads, both static and moving, in close proximity of the roadway is an issue of high significance in the context of road safety. This publication aims to provide support in solving these issues by presenting a range of options for the implementation of extensive, multi-faceted research, using modern technology to allow an objective assessment of the risks arising from the presence of advertising spots in the roadway. The chosen research tools include the drivers’ reaction tracking systems based on the use of advanced multimedia technology. These systems may be integrated in the actual vehicle, allowing for performing the tests in real-life conditions or as part of an extended driving simulator. In addition, a part of the proposed approaches to researching the problem is to check drivers’ opinion using questionnaires and to analyze the traffic accidents taking place in close proximity to road advertising.