Denis Kalkofen

Interactive Focus and Context Visualization for Augmented Reality

In this article we present interactive focus and context (F+C) visualizations for augmented reality (AR) applications. We demonstrate how F+C visualizations are used to affect the users perception of hidden objects by presenting contextual information in the area of augmentation. We carefully overlay synthetic data on top of the real world imagery by taking into account the information that is about to be occluded. Furthermore, we present operations to control the amount of augmented information. Additionally, we developed an interaction tool, based on the magic lens technique, which allows for interactive separation of focus from context. We integrated our work into a rendering framework developed on top of the Studierstube augmented reality system. We finally show examples to demonstrate how our work benefits AR.

Image-driven view management for augmented reality browsers

In this paper, we introduce a novel view management technique for placing labels in Augmented Reality systems. A common issue in many Augmented Reality applications is the absence of knowledge of the real environment, limiting the efficient representation and optimal layout of the digital information augmented onto the real world. To overcome this problem, we introduce an image-based approach, which combines a visual saliency algorithm with edge analysis to identify potentially important image regions and geometric constraints for placing labels. Our proposed solution also includes adaptive rendering techniques that allow a designer to control the appearance of depth cues. We describe the results obtained from a user study considering different scenarios, which we performed for validating our approach. Our technique will provide special benefits to Augmented Reality browsers that usually lack scene knowledge, but also to many other applications in the domain of Augmented Reality such as cultural heritage and maintenance applications.

Comprehensible Visualization for Augmented Reality

This article presents interactive visualizations to support the comprehension of spatial relationships between virtual and real world objects for augmented reality (AR) applications. To enhance the clarity of such relationships we discuss visualization techniques and their suitability for AR. We apply them on different AR applications with different goals, e.g. in X-Ray vision or in applications which draw a users attention to an object of interest. We demonstrate how Focus and Context (F+C) visualizations are used to affect the users perception of hidden or nearby objects by presenting contextual information in the area of augmentation. We discuss the organization and the possible sources of data for visualizations in augmented reality and present cascaded and multi level F+C visualizations to address complex, cluttered scenes that are inevitable in real environments. This article also shows filters and tools to interactively control the amount of augmentation. It compares the impact of real world context preserving to a pure virtual and uniform enhancement of these structures for augmentations of real world imagery. Finally this paper discusses the stylization of sparse object representations for AR to improve x-ray vision.

OmniKinect: real-time dense volumetric data acquisition and applications

Real-time three-dimensional acquisition of real-world scenes has many important applications in computer graphics, computer vision and human-computer interaction. Inexpensive depth sensors such as the Microsoft Kinect allow to leverage the development of such applications. However, this technology is still relatively recent, and no detailed studies on its scalability to dense and view-independent acquisition have been reported. This paper addresses the question of what can be done with a larger number of Kinects used simultaneously. We describe an interference-reducing physical setup, a calibration procedure and an extension to the KinectFusion algorithm, which allows to produce high quality volumetric reconstructions from multiple Kinects whilst overcoming systematic errors in the depth measurements. We also report on enhancing image based visual hull rendering by depth measurements, and compare the results to KinectFusion. Our system provides practical insight into achievable spatial and radial range and into bandwidth requirements for depth data acquisition. Finally, we present a number of practical applications of our system.

Image-based ghostings for single layer occlusions in augmented reality

In augmented reality displays, X-Ray visualization techniques make hidden objects visible through combining the physical view with an artificial rendering of the hidden information. An important step in X-Ray visualization is to decide which parts of the physical scene should be kept and which should be replaced by overlays. The combination should provide users with essential perceptual cues to understand the relationship of depth between hidden information and the physical scene. In this paper we present an approach that addresses this decision in unknown environments by analyzing camera images of the physical scene and using the extracted information for occlusion management. Pixels are grouped into perceptually coherent image regions and a set of parameters is determined for each region. The parameters change the X-Ray visualization for either preserving existing structures or generating synthetic structures. Finally, users can customize the overall opacity of foreground regions to adapt the visualization.

Interactive context-driven visualization tools for augmented reality

In this article we present an interaction tool, based on the Magic Lenses technique, that allows a 3D scene to be affected dynamically given contextual information, for example, to support information filtering. We show how elements of a scene graph are grouped by context in addition to hierarchically, and, how this enables us to locally modify their rendering styles. This research has two major contributions, the use of context sensitivity with 3D Magic Lenses in a scene graph and the implementation of multiple volumetric 3D Magic Lenses for Augmented Reality setups. We have developed our tool for the Studierstube framework which allows us doing rapid prototyping of Virtual and Augmented Reality applications. Some application directions are shown throughout the paper. We compare our work with other methods, highlight strengths and weaknesses and finally discuss research directions for our work.

Entourage: Visualizing Relationships between Biological Pathways using Contextual Subsets

Biological pathway maps are highly relevant tools for many tasks in molecular biology. They reduce the complexity of the overall biological network by partitioning it into smaller manageable parts. While this reduction of complexity is their biggest strength, it is, at the same time, their biggest weakness. By removing what is deemed not important for the primary function of the pathway, biologists lose the ability to follow and understand cross-talks between pathways. Considering these cross-talks is, however, critical in many analysis scenarios, such as judging effects of drugs. In this paper we introduce Entourage, a novel visualization technique that provides contextual information lost due to the artificial partitioning of the biological network, but at the same time limits the presented information to what is relevant to the analysts task. We use one pathway map as the focus of an analysis and allow a larger set of contextual pathways. For these context pathways we only show the contextual subsets, i.e., the parts of the graph that are relevant to a selection. Entourage suggests related pathways based on similarities and highlights parts of a pathway that are interesting in terms of mapped experimental data. We visualize interdependencies between pathways using stubs of visual links, which we found effective yet not obtrusive. By combining this approach with visualization of experimental data, we can provide domain experts with a highly valuable tool. We demonstrate the utility of Entourage with case studies conducted with a biochemist who researches the effects of drugs on pathways. We show that the technique is well suited to investigate interdependencies between pathways and to analyze, understand, and predict the effect that drugs have on different cell types.

Augmented Reality for Minimally Invasive Surgery: Overview and Some Recent Advances

Pablo Lamata1,2, Wajid Ali3, Alicia Cano1, Jordi Cornella3, Jerome Declerck2, Ole J. Elle3, Adinda Freudenthal4, Hugo Furtado5, Denis Kalkofen6, Edvard Naerum3, Eigil Samset3, Patricia Sanchez-Gonzalez1, Francisco M. Sanchez-Margallo7, Dieter Schmalstieg6, Mauro Sette8, Thomas Studeli4, Jos Vander Sloten8 and Enrique J. Gomez1 1Universidad Politecnica de Madrid, Spain 2Siemens, United Kingdom 3University of Oslo, Norway 4Delft University of Technology, Netherlands 5Medical Centre Ljubljana, Slovenia 6 Graz University of Technology, Austria 7Minimally Invasive Surgery Centre Jesus Uson, Spain 8University of Leuven, Belgium

Adaptive ghosted views for Augmented Reality

In Augmented Reality (AR), ghosted views allow a viewer to explore hidden structure within the real-world environment. A body of previous work has explored which features are suitable to support the structural interplay between occluding and occluded elements. However, the dynamics of AR environments pose serious challenges to the presentation of ghosted views. While a model of the real world may help determine distinctive structural features, changes in appearance or illumination detriment the composition of occluding and occluded structure. In this paper, we present an approach that considers the information value of the scene before and after generating the ghosted view. Hereby, a contrast adjustment of preserved occluding features is calculated, which adaptively varies their visual saliency within the ghosted view visualization. This allows us to not only preserve important features, but to also support their prominence after revealing occluded structure, thus achieving a positive effect on the perception of ghosted views.

Extended papers from NPAR 2010: Multi-perspective compact explosion diagrams

This article presents a system to automatically generate compact explosion diagrams. Inspired by handmade illustrations, our approach reduces the complexity of an explosion diagram by rendering an exploded view only for a suitable subset of the assemblies of an object. However, the exploded views are chosen so that they allow inferring the remaining unexploded assemblies of the entire 3D model. In particular, our approach demonstrates the assembly of a set of identical groups of parts by presenting an exploded view only for a single representative. In order to identify the representatives, our system automatically searches for recurring subassemblies. It selects representatives depending on a quality evaluation of their potential exploded view. Our system takes into account visibility information of both the exploded view of a potential representative as well as visibility information of the remaining unexploded assemblies. This allows rendering a balanced compact explosion diagram, consisting of a clear presentation of the exploded representatives as well as the unexploded remaining assemblies. Since representatives may interfere with one another, our system furthermore optimizes combinations of representatives. The optimization process also generates good views on the explosion diagram. Labels are added to the explosion diagram to increase the visibility of small or occluded parts. Throughout this article, we show a number of examples, which all have been rendered from unmodified 3D CAD data.