Dusan Starcevic

Universal accessibility as a multimodal design issue

Creating adaptable solutions to address diverse users and situations.

A model-driven approach to content repurposing

The multimedia metamodel defines platform-independent multimedia concepts, opening the way for novel approaches to designing content repurposing solutions. Designers can use the metamodel to create content and add metadata to existing content, simplifying content analysis and repurposing. This model-driven approach and proposed design solutions are useful not only for many multimedia designers and researchers, where the model-driven tools can help them create better multimedia interfaces, but also for lecturers and students of multimedia courses. In the latter case, the unified multimedia metamodel offers context for sometimes subtle relationships between multimedia concepts. The metamodel can also facilitate the collaborative creation of broader knowledge about multimedia phenomena. In our future work, we plan to extend the proposed metamodel and include domains from related fields, such as user modeling and intelligent tutoring systems that deal with high-level user models. We are also designing multimodal test environments, reusable multimedia components, and data mining tools for evaluating various aspects of multimedia and multimodal communication.

Tactical audio and acoustic rendering in biomedical applications

Complexity of biomedical data requires novel sophisticated analysis and presentation methods. Sonification is used as a new information display in augmented reality systems to overcome problems of existing human-computer interfaces (e.g., opaque or heavy head-mounted displays, slow computer graphics, etc.). A novel taxonomy of sonification methods and techniques is introduced. We present our experience with tactical audio and acoustic rendering in biomedical applications. Tactical audio as an audio feedback is used as support for precise manual positioning of a surgical instrument in the operating room. Acoustic rendering is applied as an additional information channel and/or warning signal in biomedical signal analysis and data presentation.

EEG analysis in a telemedical virtual world

Telemedicine creates virtual medical collaborative environments. We propose here a novel concept of virtual medical devices (VMD) for telemedical applications. VMDs provide different views on biomedical recordings and efficient signal analysis. In this paper we present a telemedical EEG analysis environment based on virtual reality technologies. The same EEG signal/recording could be viewed either as a waveform or as animated topographic maps on a 3D head model. In addition to visualization, sonification is used as a secondary presentation modality. The environment is based on a VRML head model animated with Java applets, and allows the use of standard web browser.

Walking in Colors: Human Gait Recognition Using Kinect and CBIR

This article proposes a new method of recognizing human gait. The proposed method is based on the idea that the problem of human gait recognition can be transformed from a spatio-temporal problem into the spatial domain, specifically the 2D image domain. This is done by representing a sample of a human gait as a still image. By doing so, all the recorded information is kept while enabling the use of proven content-based image retrieval (CBIR) techniques for recognition. The proposed method uses Microsoft Kinect human-computer interaction technology for data acquisition. To prove the validity of the proposed approach, the authors conducted a study with 50 participants.

Motivational aspects of different learning contexts: “My mom won't let me play this game...”

Learning motivation should describe mechanisms that direct toward the learning goals, initiate learning activity and hold ones learning ability on a level that enables person to attain new information and more complex knowledge. Educational games are emerging as a new form of learning that should improve students desire for knowledge. Idea that educational games can be used as an effective teaching tool is what inspires our work in this field. In our study we attempted to explore the connection between motivational effects and application of educational game design as a learning tool. Additionally, we took into consideration effects of this approach on improving final exam results. Our results indicate that designing computer programs can raise enough curiosity and can be seen as a relevant learning tool for students.

Perceptualization of biomedical data. An experimental environment for visualization and sonification of brain electrical activity

Complex experiments require sophisticated visualization of a large number of parameters. However, physiological characteristics of human perception limit the number of visualized parameters and their dynamics. Multimodal data presentation could increase throughput, introduce new data streams, and improve temporal resolution. The future of biomedical data presentation will probably be marked with multimodal data presentation and integration of multiple diagnostic procedures. Here, the authors present sonification as a second presentation modality. During analysis, multimodal parameters must be set to maximize the distance of changes in the perceptual domain. The authors implemented sonification in their 3D EEG visualization environment to render synesthetic extension of a selected data channel or to introduce new composite parameters. Sonification improved the ability to assess genuine dynamics of brain electrical activity and to perceive inherent spatio-temporal patterns of brain electrical activity.

Motivation and Multimodal Interaction in Model-Driven Educational Game Design

In this correspondence, we present an approach to identifying and constructing profiles of user interfaces for educational games. Our approach is based on framing games as educational tools that incorporate fun and learning through motivation as the key ingredient in the learning process and multimodal interaction as the medium for conveying educational material. The proposed solution formalizes the design process, describing educational games in terms of estimated effects that they produce on players. Building upon research on learning and motivation theory, we are connecting these effects with player learning preferences and motivation states. The essence of our solution is the educational game metamodel (EGM), which defines platform-independent educational game concepts. Using the EGM, we have explored novel design approaches for educational games. The metamodel can be used as a conceptual basis for creation of platform-independent educational games, allowing authoring for device and network independence.

Adventure Game Learning Platform

Educational games display great potential as an active form of knowledge transfer. This research field is young, but some patterns in educational game development can be recognized. In this paper, the authors present a new approach to educational game development that overcomes some downsides of more traditional systems. The paper provides the opportunity to create an educational adventure game by using specialized software tool as well as integrating knowledge in that specific game instance. As a result of that process, game definition is created as a form of XML document. On the other side, a web-based interpreter is used to present the adventure game to user in runtime. XML format provides us with platform independency. By use of this tool, the educator gains the ability to create an educational game without programming knowledge, and to reuse some previously created knowledge.

Educational Games Design Issues: Motivation and Multimodal Interaction

In this paper we present an approach to identifying and constructing profiles of user interfaces for educational games. Our approach is based on framing games as educational tools that incorporate fun and learning through motivation as the key ingredient in the learning process, and multimodal interaction as the medium for conveying educational material. To date, there is no common standard to design this kind of games and individual solutions are usually carried out by an ad hocprocess. Proposed solution formalizes design process describing educational games in terms of estimated effects they produce on players. Building upon research on learning and motivation theory, we are connecting these effects with player learning preferences and motivation states. Our main contribution is to suggest design issues that should be taken into account when designing educational games. We exemplify the approach through our ongoing VStrat project, framework for designing educational games.