Otilia Kocsis

PlayMancer: A Serious Gaming 3D Environment

Serious games are computer games used as educational technology or as a vehicle for presenting or promoting a point of view. Serious games are intended to provide an engaging, self-reinforcing context in which to motivate and educate the players towards non-game events or processes, including business operations, training, marketing and advertisement. The potential of games for entertainment and learning has been demonstrated thoroughly from research and clearly in the market place. Unfortunately, the investments committed to entertainment dwarfs that which is committed for more serious purposes. Furthermore, game development has become more complex, expensive, and burdened with a long development cycle. In this paper we introduce PlayMancer, a work in progress that aims to overcome such barriers by augmenting existing 3D gaming engines with new possibilities and thusly creating a novel development framework. In section I of this paper we briefly survey the serious games market. In section II we introduce the PlayMancer project and its objectives, whereas in section III we present the platform architecture. Section VI describes an application scenario where a PlayMancer-based game is being used as additional therapeutic tool to treat chronic mental disorders, such as eating disorders and behavioral addiction.

Affective speech interface in serious games for supporting therapy of mental disorders

We describe a novel design, implementation and evaluation of a speech interface, as part of a platform for the development of serious games. The speech interface consists of the speech recognition component and the emotion recognition from speech component. The speech interface relies on a platform designed and implemented to support the development of serious games, which supports cognitive-based treatment of patients with mental disorders. The implementation of the speech interface is based on the Olympus/RavenClaw framework. This framework has been extended for the needs of the specific serious games and the respective application domain, by integrating new components, such as emotion recognition from speech. The evaluation of the speech interface utilized purposely collected domain-specific dataset. The speech recognition experiments show that emotional speech moderately affects the performance of the speech interface. Furthermore, the emotion detectors demonstrated satisfying performance for the emotion states of interest, Anger and Boredom, and contributed towards successful modelling of the patients emotion status. The performance achieved for speech recognition and for the detection of the emotional states of interest was satisfactory. Recent evaluation of the serious games showed that the patients started to show new coping styles with negative emotions in normal stress life situations.

Context-adaptive pre-processing scheme for robust speech recognition in fast-varying noise environment

Based on the observation that dissimilar speech enhancement algorithms perform differently for different types of interference and noise conditions, we propose a context-adaptive speech pre-processing scheme, which performs adaptive selection of the most advantageous speech enhancement algorithm for each condition. The selection process is based on an unsupervised clustering of the acoustic feature space and a subsequent mapping function that identifies the most appropriate speech enhancement channel for each audio input, corresponding to unknown environmental conditions. Experiments performed on the MoveOn motorcycle speech and noise database validate the practical value of the proposed scheme for speech enhancement and demonstrate a significant improvement in terms of speech recognition accuracy, when compared to the one of the best performing individual speech enhancement algorithm. This is expressed as accuracy gain of 3.3% in terms of word recognition rate. The advance offered in the present work reaches beyond the specifics of the present application, and can be beneficial to spoken interfaces operating in fast-varying noise environments.

A tool for designing digital test objects for module performance evaluation in medical digital imaging

Currently, medical digital imaging systems are characterized by the introduction of additional modules such as digital display, image compression and image processing, as well as film printing and digitization. These additional modules require performance evaluation to ensure high image quality. A tool for designing computer-generated test objects applicable to performance evaluation of these modules is presented. The test objects can be directly used as digital images in the case of film printing, display, compression and image processing, or indirectly as images on film in the case of digitization. The performance evaluation approach is quality control protocol based. Digital test object design is user-driven according to specifications related to the requirements of the modules being tested. The available quality control parameters include input/output response curve, high contrast resolution, low contrast discrimination, noise, geometric distortion and field uniformity. The tool has been designed and implemented according to an object oriented approach in Visual C++ 5.0, and its user interface is based on the Microsoft Foundation Class Library version 4.2, which provides interface items such as windows, dialog boxes, lists, buttons, etc. The compatibility with DICOM 3.0 part 10 image formats specifications allows the integration of the tool in the existing software framework for medical digital imaging systems. The capability of the tool is demonstrated by direct use of the test objects in case of image processing, and indirect use of the test objects in case of film digitization.

Robust speech interaction in motorcycle environment

Aiming at robust spoken dialogue interaction in motorcycle environment, we investigate various configurations for a speech front-end, which consists of speech pre-processing, speech enhancement and speech recognition components. These components are implemented as agents in the Olympus/RavenClaw framework, which is the core of a multimodal dialogue interaction interface of a wearable solution for information support of the motorcycle police force on the move. In the present effort, aiming at optimizing the speech recognition performance, different experimental setups are considered for the speech front-end. The practical value of various speech enhancement techniques is assessed and, after analysis of their performances, a collaborative scheme is proposed. In this collaborative scheme independent speech enhancement channels operate in parallel on a common input and their outputs are fed to the multithread speech recognition component. The outcome of the speech recognition process is post-processed by an appropriate fusion technique, which contributes for a more accurate interpretation of the input. Investigating various fusion algorithms, we identified the Adaboost.M1 algorithm as the one performing best. Utilizing the fusion collaborative scheme based on the Adaboost.M1 algorithm, significant improvement of the overall speech recognition performance was achieved. This is expressed in terms of word recognition rate and correctly recognized words, as accuracy gain of 8.0% and 5.48%, respectively, when compared to the performance of the best speech enhancement channel, alone. The advance offered in the present work reaches beyond the specifics of the present application, and can be beneficial to spoken interfaces operating in non-stationary noise environments.

Automated Acoustic Classification of Bird Species from Real -Field Recordings

We report on a recent progress with the development of an automated bioacoustic bird recognizer, which is part of a long-term project, aiming at the establishment of an automated biodiversity monitoring system at the Hymettus Mountain near Athens. In particular, employing a classical audio processing strategy, which has been proved quite successful in various audio recognition applications, we evaluate the appropriateness of six classifiers on the bird species recognition task. In the experimental evaluation of the acoustic bird recognizer, we made use of real-field audio recordings for seven bird species, which are common for the Hymettus Mountain. Encouraging recognition accuracy was obtained on the real-field data, and further experiments with additive noise demonstrated significant noise robustness in low SNR conditions.

SPEECH ENHANCEMENT FOR ROBUST SPEECH RECOGNITION IN MOTORCYCLE ENVIRONMENT

In the present work, we investigate the performance of a number of traditional and recent speech enhancement algorithms in the adverse non-stationary conditions, which are distinctive for motorcycles on the move. The performance of these algorithms is ranked in terms of the improvement they contribute to the speech recognition accuracy, when compared to the baseline performance, i.e. without speech enhancement. The experiments on the MoveOn motorcycle speech and noise database indicated that there is no equivalence between the ranking of algorithms based on the human perception of speech quality and the speech recognition performance. The Multi-band spectral subtraction method was observed to lead to the highest speech recognition performance.

Assessing machine learning algorithms for self-management of asthma

Control and monitoring of asthma progress is highly important for patients quality of life and healthcare management. Emerging tools for self-management of various chronic diseases have the potential to support personalized patient guidance. This work presents the design aspects of the myAirCoach decision support system, with focus on the assessment of three machine learning approaches as support tools the first prototype implementation.

Dynamic selection of a speech enhancement method for robust speech recognition in moving motorcycle environment

We present a speech pre-processing scheme (SPPS) for robust speech recognition in the moving motorcycle environment. The SPPS is dynamically adapted during the run-time operation of the speech front-end, depending on short-time characteristics of the acoustic environment. In detail, the fast varying acoustic environment is modeled by GMM clusters based on which a selection function determines the speech enhancement method to be applied. The correspondence between input audio and speech enhancement method is learned during the training of the selection function. The SPPS was found to outperform the best performing speech enhancement method by approximately 3.3% in terms of word recognition rate (WRR).

Multi-modal System Architecture for Serious Gaming

Human-computer interaction (HCI), especially in the games domain, targets to mimic as much as possible the natural human-to-human interaction, which is multimodal, involving speech, vision, haptic, etc. Furthermore, the domain of serious games, aiming to value-added games, makes use of additional inputs, such as biosensors, motion tracking equipment, etc. In this context, game development has become complex, expensive and burdened with a long development cycle. This creates barriers to independent game developers and inhibits the introduction of innovative games, or new game genres. In this paper the PlayMancer platform is introduced, a work in progress aiming to overcome such barriers by augmenting existing 3D game engines with innovative modes of interaction. Playmancer integrates open source existing systems, such as a game engine and a spoken dialog management system, extended by newly implemented components, supporting innovative interaction modalities, such as emotion recognition from audio data, motion tracking, etc, and advanced configuration tools.