Gerd Hesina

The studierstube augmented reality project

Our starting point for developing the Studierstube system was the belief that augmented reality, the less obtrusive cousin of virtual reality, has a better chance of becoming a viable user interface for applications requiring manipulation of complex three-dimensional information as a daily routine. In essence, we are searching for a 3-D user interface metaphor as powerful as the desktop metaphor for 2-D. At the heart of the Studierstube system, collaborative augmented reality is used to embed computer-generated images into the real work environment. In the first part of this paper, we review the user interface of the initial Studierstube system, in particular the implementation of collaborative augmented reality, and the Personal Interaction Panel, a two-handed interface for interaction with the system. In the second part, an extended Studierstube system based on a heterogeneous distributed architecture is presented. This system allows the user to combine multiple approaches augmented reality, projection displays, and ubiquitous computingto the interface as needed. The environment is controlled by the Personal Interaction Panel, a twohanded, pen-and-pad interface that has versatile uses for interacting with the virtual environment. Studierstube also borrows elements from the desktop, such as multitasking and multi-windowing. The resulting software architecture is a user interface management system for complex augmented reality applications. The presentation is complemented by selected application examples.

Distributed Open Inventor: a practical approach to distributed 3D graphics

Distributed Open Inventor is an extension to the popular Open Inventor toolkit for interactive 3D graphics. The toolkit is extended with the concept of a distributed shared scene graph, similar to distributed shared memory. From the application programmers perspective, multiple workstations share a common scene graph. The proposed system introduces a convenient mechanism for writing distributed graphical applications based on a popular tool in an almost transparent manner. Local variations in the scene graph allow for a wide range of possible applications, and local low latency interaction mechanisms called input streams enable high performance while saving the programmer from network peculiarities.

Occlusion in Collaborative Augmented Environments

Abstract Augmented environments superimpose computer enhancements on the real world. Such augmented environments are well suited for collaboration of multiple users. To improve the quality and consistency of the augmentation the occlusion of real objects by computer-generated objects and vice versa has to be implemented. We present methods how this can be done for a tracked users body and other real objects and how irritating artifacts due to misalignments can be reduced. Our method is based on simulating the occlusion of virtual objects by a representation of the user modeled as kinematic chains of articulated solids. Smoothing the border between virtual world and occluding real reduces registration and modeling errors of this model. Finally, an implementation in our augmented environment and the resulting improvements are presented.

Bridging multiple user interface dimensions with augmented reality

Studierstube is an experimental user interface system which uses collaborative augmented reality to incorporate true 3D interaction into a productivity environment. This concept is extended to bridge multiple user interface dimensions by including multiple users, multiple host platforms, multiple display types, multiple concurrent applications and a multi-context (i.e. 3D document) interface into a heterogeneous distributed environment. With this architecture, we can explore the user interface design space between pure augmented reality and the popular ubiquitous computing paradigm. We report on our design philosophy, which is centered around the notion of contexts and locales, as well as the underlying software and hardware architecture. Contexts encapsulate a live application together with 3D (visual) and other data, while locales are used to organize geometric reference systems. By separating geometric relationships (locales) from semantic relationships (contexts), we achieve a great amount of flexibility in the configuration of displays. To illustrate our claims, we present several applications, including a cinematographic design tool which showcases many features of our system.

A network architecture for remote rendering

Internet-based virtual environments (VEs) let users explore multiple virtual worlds with many different geometric models, which are downloaded rather than pre-distributed. To avoid long download times, we have developed a method that optimally utilizes network bandwidth by downloading only the exact portion of geometry that is necessary for rendering. The solution is based on progressive geometry data structures (smooth levels of detail) and on selective downloading.

Guided visibility sampling

This paper addresses the problem of computing the triangles visible from a region in space. The proposed aggressive visibility solution is based on stochastic ray shooting and can take any triangular model as input. We do not rely on connectivity information, volumetric occluders, or the availability of large occluders, and can therefore process any given input scene. The proposed algorithm is practically memoryless, thereby alleviating the large memory consumption problems prevalent in several previous algorithms. The strategy of our algorithm is to use ray mutations in ray space to cast rays that are likely to sample new triangles. Our algorithm improves the sampling efficiency of previous work by over two orders of magnitude.

Distributed applications for collaborative augmented reality

This paper focuses on the distributed architecture of the collaborative augmented reality system Studierstube. The system allows multiple users to experience a shared 3D workspace populated by multiple applications using see-through head mounted displays or other presentation media such as projection systems. The system design is based on a distributed shared scene graph that alleviates the application programmer from explicitly considering distribution, and avoids a separation of graphical and application data. The idea of unifying all system data in the scene graph is taken to its logical consequence by implementing application instances as nodes in the scene graph. Through the distributed shared scene graph mechanism, consistency of scene graph replicas and the contained application nodes is assured. Multi-user 3D widgets allow concurrent interaction with minimal coordination effort from the application. Special interest is paid to migration of application nodes from host to host allowing dynamic workgroup management, such as load balancing, late joining and early exit of hosts, and some firms of ubiquitous computing.

Distributed applications for collaborative three-dimensional workspaces

This paper focuses on the distributed architecture of the collaborative threedimensional user interface management system, Studierstube. The system allows multiple users to experience a shared 3D workspace populated by multiple applications using see-through head-mounted displays or other presentation media such as projection systems. Building large, ubiquitous, or mobile workspaces requires distribution of applications over several hosts in varying and dynamic configurations. The system design is based on a distributed shared scene graph that alleviates the application programmer from explicitly considering distribution and that avoids a separation of graphical and application data. The idea of unifying all system data in the scene graph is taken to its logical consequence by implementing application instances as nodes in the scene graph. Through the distributed shared scene graph mechanism, consistency of scene graph replicas and the contained application nodes is assured. Dynamic configuration management is based on application migration between participating hosts and a spatial model of locales allowing dynamic workgroup management. We describe a number of experimental workspaces that demonstrate the use of these configuration management techniques.

Vis-A-Ware: Integrating Spatial and Non-Spatial Visualization for Visibility-Aware Urban Planning

3D visibility analysis plays a key role in urban planning for assessing the visual impact of proposed buildings on the cityscape. A call for proposals typically yields around 30 candidate buildings that need to be evaluated with respect to selected viewpoints. Current visibility analysis methods are very time-consuming and limited to a small number of viewpoints. Further, analysts neither have measures to evaluate candidates quantitatively, nor to compare them efficiently. The primary contribution of this work is the design study of Vis-A-Ware, a visualization system to qualitatively and quantitatively evaluate, rank, and compare visibility data of candidate buildings with respect to a large number of viewpoints. Vis-A-Ware features a 3D spatial view of an urban scene and non-spatial views of data derived from visibility evaluations, which are tightly integrated by linked interaction. To enable a quantitative evaluation we developed four metrics in accordance with experts from urban planning. We illustrate the applicability of Vis-A-Ware on the basis of a use case scenario and present results from informal feedback sessions with domain experts from urban planning and development. This feedback suggests that Vis-A-Ware is a valuable tool for visibility analysis allowing analysts to answer complex questions more efficiently and objectively.

A 3-d laser scanning system and scan data processing method for themonitoring of tunnel deformations

Abstract. The paper presents the mobile multi-sensor system Orthos Plus for the monitoring and mapping of tunnel walls, a scan data processing method for the evaluation of 3-d tunnel wall displacements from subsequent wall scans and, finally, a virtual reality tool supporting the interpretation of data. The measuring system consists of a 3-d laser scanner, a motorised total station and a digital camera that are integrated on a light metal frame that is installed on a mobile platform. It has been designed to perform tunnel measurements most efficiently and to meet the special requirements of tunnels under construction. The evaluation of 3-d displacements is based on a 3-d matching algorithm that takes advantage of the particular conditions of tunnel (shotcrete) surfaces. The virtual reality tool allows viewing of data in a 3-d virtual reality tunnel model and their animation in time and space in order supports understanding in an optimal way. The measuring system Orthos Plus has been developed in the course of a national research project, the 3-d matching method in the frame of the Austrian Christian Doppler Laboratory Spatial Data from Laser Scanning and Remote Sensing and the VR tool in the Austrian COMET K1 Competence Center VRVis Center (www.vrvis.at).