Ingo Assenmacher

Virtual reality system with integrated sound field simulation and reproduction

A real-time audio rendering system is introduced which combines a full room-specific simulation, dynamic crosstalk cancellation, and multitrack binaural synthesis for virtual acoustical imaging. The system is applicable for any room shape (normal, long, flat, coupled), independent of the a priori assumption of a diffuse sound field. This provides the possibility of simulating indoor or outdoor spatially distributed, freely movable sources and a moving listener in virtual environments. In addition to that, near-to-head sources can be simulated by using measured near-field HRTFs. The reproduction component consists of a headphone-free reproduction by dynamic crosstalk cancellation. The focus of the project is mainly on the integration and interaction of all involved subsystems. It is demonstrated that the system is capable of real-time room simulation and reproduction and, thus, can be used as a reliable platform for further research on VR applications.

Interactive data annotation in virtual environments

Note-taking is an integral part of scientific data analysis. In particular, it is vital for explorative analysis, as the expression and transformation of ideas is a necessary precondition for gaining insight. However, in the case of interactive data exploration in virtual environments it is not possible to keep a pen and pencil at hand. Additionally, data analysis in virtual environments allows the multi-modal exploration of complex and time varying data. We propose the toolkit independent content generation system IDEA that features a defined process model, a generic annotation model with a variety of content types as well as specially developed interaction metaphors for their input and output handling. This allows the user to note ideas, e.g., in form of text, images or voice without interfering with the analysis process. In this paper we present the basic concepts for this system. We describe the context-content model which allows to tie annotation content to logical objects that are part of the scene and stores specific information for the special interaction in virtual environments. The IDEA system is already applied in a prototypical implementation for the exploration of air flows in the human nasal cavity where it is used for data analysis as well as interdisciplinary communication.

A Time Model for Time‐Varying Visualization

The analysis of unsteady phenomena is an important topic for scientific visualization. Several time‐dependent visualization techniques exist, as well as solutions for dealing with the enormous size of time‐varying data in interactive visualization. Many current visualization toolkits support displaying time‐varying data sets. However, for the interactive exploration of time‐varying data in scientific visualization, no common time model that describes the temporal properties which occur in the visualization process has been established. In this work, we propose a general time model which classifies the time frames of simulation phenomena and the connections between different time scales in the analysis process. This model is designed for intuitive interaction with time in visualization applications for the domain expert as well as for the developer of visualization tools. We demonstrate the benefits of our model by applying it to two use cases with different temporal properties.

Integrating real-time binaural acoustics into VR applications

Common research in the field of Virtual Reality (VR) considers acoustic stimulation as a highly important necessity for enhanced immersion into virtual scenes. However, most common VR toolkits do only marginally support the integration of sound for the application programmer. Furthermore, the quality of stimulation that is provided usually ranges from system sounds (e.g. beeps while selecting a menu) to simple 3D panning. In the latter case, these approaches do only allow the user to correctly detect sounds that are at quite a distance from his current position. Binaural synthesis is an interesting way to allow the spatial auditory representation by using few loudspeakers or headphones. This paper describes a system that combines the efforts of creating a binaural representation for the listener who is interacting in a common visual VR application in real-time, thus allowing the research on interaction between visual and auditory human perception systems. It will describe the theoretical background to establishing a binaural representation of a sound and the necessary hardware set-up for this. Afterwards, the infrastructure and software interface which will allow the connection of the audio renderer to a visual VR toolkit is discussed.

A true spatial sound system for CAVE-like displays using four loudspeakers

The paper introduces an audio rendering system based on the binaural approach, which allows a real-time simulation of spatially distributed sound sources and, in addition to that, near-to-head sources in room-mounted virtual environments. We have been developing a dynamic crosstalk cancellation, allowing the listener to freely move around without wearing any headphones. The paper gives a comprehensive description of the system, concentrating on the dual dynamic crosstalk cancellation and aspects of the integration of a real-time room acoustical simulation. Finally, two applications are described to show the wide applicability of the system.

3D Sketch Recognition for Interaction in Virtual Environments

We present a comprehensive 3D sketch recognition framework for interaction within Virtual Environments that allows to trigger commands by drawing symbols, which are recognized by a multi-level analysis. It proceeds in three steps: The segmentation partitions each input line into meaningful segments, which are then recognized as a primitive shape, and finally analyzed as a whole sketch by a symbol matching step. The whole framework is configurable over well-defined interfaces, utilizing a fuzzy logic algorithm for primitive shape learning and a textual description language to define compound symbols. It allows an individualized interaction approach that can be used without much training and provides a good balance between abstraction and intuition. We show the real-time applicability of our approach by performance measurements.

Binaural acoustics for CAVE-like environments without headphones

The human auditory system, in contrast to the human visual system, can perceive input from all directions and has no limited field of view. As such, it provides valuable cues for navigation and orientation in virtual environments. However, audio stimuli are not that common in todays Virtual Reality applications, and this might result from the lack of middleware or user acceptance due to the need for specialized or costly hardware. Surprisingly, the lack of headphone-less near body acoustics is widely accepted, and simple intensity panning approaches that enable plausible spatial audio are used. This paper describes a networked environment for sophisticated binaural synthesis-based audio rendering in visual VR applications for a freely moving listener in a CAVE-like environment without the use of headphones. It describes the binaural acoustics rendering technique and a dynamic crosstalk cancellation system for four loudspeakers. In addition to that, synchronization issues and network coupling together with performance measurements that proof the applicability of the system in interactive Virtual Environments are discussed.

Multimedia and VR Support for Direct Communication of Designers

The development of design processes in chemical engineering and plastics processing requires close cooperation between designers of different companies or working groups. A large number of communication relationships is established, e.g. for the clarification of problems within single tasks, or within project meetings covering the discussion about interim results. With the ongoing development of a process, different types of communication relationships will occur, as the required communication form as well as the extent of communication are depending on the work task. For an efficient support of the communication between designers, support tools are needed to enable cooperative computer-aided work tailored to an actual task and to obtain a speed-up of the development of a process. This section discusses several activities to improve the communication between engineers by multimedia and Virtual Reality tools and protocol mechanisms for supporting new forms of cooperative work in the design of a process. To ease the usage of those tools and protocols within the work processes, they are integrated into the normal working environments of the designers. For communication processes involving geographically distributed designers, the communication platform KomPaKt was developed, which integrates the new communication and cooperation tools for different purposes in a single, configuration-free, and intuitive user interface. As a specially interesting case of cooperation between designers, the technology of immersive Virtual Reality for simulation sciences was examined in more detail using the example of compound extruders, as Virtual Reality technology plays a key role in interdisciplinary communication processes.

On device driver architectures for virtual reality toolkits

One major goal for the development of virtual reality (VR) toolkits is to provide interfaces for novel input or output hardware to support multimodal interaction. The research community has produced several implementations that feature a large variety of device interfaces and abstractions. As a lesson learned from existing approaches, we sum up the requirements for the design of a driver layer that is the basis for a multimodal input and output system in this paper. We derive a general model for driver architectures based on these requirements. This model can be used for reasoning about different implementations of available architectures. As the flow of data through the system is of interest, we take a closer look at common patterns of data processing. Finally, we discuss a number of openly accessible driver architectures currently used for VR development.

Design Support of Reaction and Compounding Extruders

This section describes the different dimensions of integration inside the plastics processing domain as well as cross-organizational integration and collaboration issues. The presented results range from work process modeling up to technical process analysis for the design of compounding extruders in the chemical engineering context. Standard practices for the design of polymer compounding extruders were analyzed and afterwards formalized in cooperation with subproject I1 using methods and tools developed and used in the CRC 476. Fragments of these workflows were redesigned using innovative informatics functionality provided by the CRCs B-projects which provided the novel tool functionality. Exemplarily, the extruder simulation tool MOREX was integrated with the process-integrated modeling environment PRIME and coupled with BEMflow. The distributed analysis of 3D simulation results using KOMPAKT and TRAMP was developed, and a scenario showing the integration of the project management system AHEAD with the plastics engineering design tools was designed participatively. Another focus was set on an integrated visualization environment using Virtual Reality technology for different data from a number of simulation tools.