Julia Fröhlich

The Visual, the Auditory and the Haptic – A User Study on Combining Modalities in Virtual Worlds

In order to make a step further towards understanding the impact of multi-modal stimuli in Virtual Reality we conducted a user study with 80 participants performing tasks in a virtual pit environment. Participants were divided into four groups, each presented a different combination of multi-sensory stimuli. Those included real-time 3D graphics, audio stimuli (ambient, static and event sounds), and haptics consisting of wind and tactile feedback when touching objects. A presence questionnaire was used to evaluate subjectively reported presence on the one hand, and on the other physiological sensors were used to measure heart rate and skin conductance as an objective measure. Results strongly indicate that an increase of modalities does not automatically result in an increase of presence.

Acoustically enriched virtual worlds with minimum effort

To improve user experiences and immersion within virtual environments auditory experience has long been claimed to be of notable importance [1]. This paper introduces a framework, in which objects, enriched with information about their sound properties, are being processed to generate virtual sound sources. This is done with an automatic processing of the 3D-scene and therefore minimizes the effort needed to develop a multimodal virtual world. In order to create a comprehensive auditory experience different types of sound sources have to be distinguished. We propose a differentiation into three classes: locally bound static sounds, dynamically created event based sounds, and ambient sounds to create spatial atmosphere.

A Phong-based concept for 3d-audio generation

Intelligent virtual objects gain more and more significance in the development of virtual worlds. Although this concept has high potential in generating all kinds of multimodal output, so far it is mostly used to enrich graphical properties. This paper proposes a framework, in which objects, enriched with information about their sound properties, are being processed to generate virtual sound sources. To create a sufficient surround sound experience not only single sounds but also environmental properties have to be considered. We introduce a concept, transferring features from the Phong lighting model to sound rendering.

Combining Multi-Sensory Stimuli in Virtual Worlds --- A Progress Report

In order to make a significant step towards more realistic virtual experiences, we created a multi-sensory stimuli display for a CAVE-like environment. It comprises graphics, sound, tactile feedback, wind and warmth. In the present report we discuss the possibilities and constraints tied to such an enhancement. To use a multi-modal display in a proper way, many considerations have to be taken into account. This includes safety requirements, hardware devices and software integration. For each stimulus different possibilities are reviewed with regard to their assets and drawbacks. Eventually the resulting setup realized in our lab is described --- to our knowledge one of the most comprehensive systems. Technical evaluations as well as user studies accompanied the development and gave hints with respect to necessities and chances.