Emmanuelle Richard

Multi-modal virtual environments for education with haptic and olfactory feedback

It has been suggested that immersive virtual reality (VR) technology allows knowledge-building experiences and in this way provides an alternative educational process. Important key features of constructivist educational computer-based environments for science teaching and learning, include interaction, size, transduction and reification. Indeed, multi-sensory VR technology suits very well the needs of sciences that require a higher level of visualization and interaction. Haptics that refers to physical interactions with virtual environments (VEs) may be coupled with other sensory modalities such as vision and audition but are hardly ever associated with other feedback channels, such as olfactory feedback. A survey of theory and existing VEs including haptic or olfactory feedback, especially in the field of education is provided. Our multi-modal human-scale VE VIREPSE (virtual reality platform for simulation and experimentation) that provides haptic interaction using a string-based interface called SPIDAR (space interface device for artificial reality), olfactory and auditory feedbacks is described. An application that allows students experiencing the abstract concept of the Bohr atomic model and the quantization of the energy levels has been developed. Different configurations that support interaction, size and reification through the use of immersive and multi-modal (visual, haptic, auditory and olfactory) feedback are proposed for further evaluation. Haptic interaction is achieved using different techniques ranging from desktop pseudo-haptic feedback to human-scale haptic interaction. Olfactory information is provided using different fan-based olfactory displays (ODs). Significance of developing such multi-modal VEs for education is discussed.

Augmented Reality for Rehabilitation of Cognitive Disabled Children: A Preliminary Study

We have designed a non-immersive recreational and educational augmented reality application (ARVe - Augmented Reality applied to Vegetal field) that allows young children to handle 2D and 3D plant entities in a simple and intuitive way. This application involves a task of pairing and provides visual, olfactory or auditory cues to help children in decision making. 93 children from a French elementary school (including 11 cognitive disabled ones) participated in a preliminary study. The objectives of this study were: (1) to investigate children performance and behaviour in using AR techniques, and (2) to examine specific attitudes of cognitive disabled children confronted to such techniques. We have observed that disabled children were very enthusiastic when using the application and showed a high motivation compared to most other pupils. Moreover, autistic and trisomic children were able to express some positive emotions when confronted to the application. These very encouraging results promote a widespread use of such tools for cognitive disabled children.

Using olfactive virtual environments for learning organic molecules

A multi-modal virtual reality application that aims to investigate the effect of olfaction on learning, retention, and recall of complex 3D structures such as organic molecules, is presented. Students interact with molecules in either desktop or immersive configuration. In the latter case, visual immersion is achieved through the use of a large rear-projected stereoscopic screen. In both configurations motion parallax is provided using a camera-based head tracking technique. Both desktop and large-scale fan-based devices that allow real-time smell diffusion are used.

A dual-modal virtual reality kitchen for (re)learning of everyday cooking activities in alzheimer's disease

Everyday action impairment is one of the diagnostic criteria of Alzheimers disease and is associated with many serious consequences, including loss of functional autonomy and independence. It has been shown that the (re)learning of everyday activities is possible in Alzheimers disease by using error reduction teaching approaches in naturalistic clinical settings. The purpose of this study is to develop a dual-modal virtual reality platform for training in everyday cooking activities in Alzheimers disease and to establish its value as a training tool for everyday activities in these patients. Two everyday tasks and two error reduction learning methods were implemented within a virtual kitchen. Two patients with Alzheimers disease and two healthy elderly controls were tested. All subjects were trained in two learning sessions on two comparable cooking tasks. Within each group (i.e., patients and controls), the order of the training methods was counterbalanced. Repeated measure analysis before and after learning was performed. A questionnaire of presence and a verbal interview were used to obtain information about the subjective responses of the participants to the VR experience. The results in terms of errors, omissions, and perseverations (i.e., repetitive behaviors) indicate that the patients performed worse than the controls before learning, but that they reached a level of performance similar to that of the controls after a short learning session, regardless of the learning method employed. This finding provides preliminary support for the value of the dual-modal virtual reality platform for training in everyday cooking activities in Alzheimers disease. However, further work is needed before it is ready for clinical application.

Multi-modal virtual environments for education: from illusion to immersion

A multi-modal Virtual Environment that provides human-scale haptic feedback is described. Users immersion is achieved using a large rear-projected stereoscopic screen, a 5.1 audio system, and fan-based olfactory displays. An educational Virtual Reality application that allows students to experiment the electron bound state in the Bohr atom model and the quantification of the energy levels through the haptic channel, is presented. This application can be run in both immersive and desktop configurations including haptic interaction, ranging from pseudo-haptic illusion to human-scale haptic immersion.

RTIL-system: a Real-Time Interactive L-system for 3D interactions with virtual plants

The L-system is a rewriting process based on formal grammar and is used to generate 3D, dynamic structures such as virtual plants and fractal graphics. In previous works, we highlighted that existing L-system software applications and programs are limited, either in terms of human interaction or in terms of modelling. In particular, few of them allow the user to interact with virtual plants during their growth. Our own L-system engine was developed and called the real-time interactive L-system (RTIL-system). The RTIL-system covers most important L-system extensions such as parametric and context-sensitive features. Furthermore, real-time interactions with the user and the environment with respect to L-system formalism are available. This paper presents an RTIL-system focusing on human interaction, the Partial Interactive Derivation (PID) concept and further progress by the extension of PID to context-sensitive rules. To illustrate the potential of the RTIL-system, the effect of various interactive tasks such as sub-axis additions, pruning and bending on the subsequent dynamic development of virtual plants is described.

Modeling dynamic interaction in virtual environments and the evaluation of dynamic virtual fixtures

Virtual reality (VR) is a technology covering a large field of applications among which are sports and video games. In both gaming and sporting VR applications, interaction techniques involve specific gestures such as catching or striking. However, such dynamic gestures are not currently being recognized as elementary task primitives, and have therefore not been investigated as such. In this paper, we propose a framework for the analysis of interaction in dynamic virtual environments (DVEs). This framework is based on three dynamic interaction primitives (DIPs) that are common to many sporting activities: catching, throwing, and striking. For each of these primitives, an original modeling approach is proposed. Furthermore, we introduce and formalize the concept of dynamic virtual fixtures (DVFs). These fixtures aim to assist the user in tasks involving interaction with moving objects or with objects to be set in movement. Two experiments have been carried out to investigate the influence of different DVFs on human performance in the context of ball catching and archery. The results reveal a significant positive effect of the DVFs, and that DVFs could be either classified as performance-assisted or learning-assisted.