Felix Ritter

Illustrative visualization of 3D planning models for augmented reality in liver surgery

PurposeAugmented reality (AR) obtains increasing acceptance in the operating room. However, a meaningful augmentation of the surgical view with a 3D visualization of planning data which allows reliable comparisons of distances and spatial relations is still an open request.MethodsWe introduce methods for intraoperative visualization of 3D planning models which extend illustrative rendering and AR techniques. We aim to reduce visual complexity of 3D planning models and accentuate spatial relations between relevant objects. The main contribution of our work is an advanced silhouette algorithm for 3D planning models (distance-encoding silhouettes) combined with procedural textures (distance-encoding surfaces). In addition, we present a method for illustrative visualization of resection surfaces.ResultsThe developed algorithms have been embedded into a clinical prototype that has been evaluated in the operating room. To verify the expressiveness of our illustration methods, we performed a user study under controlled conditions. The study revealed a clear advantage in distance assessment with the proposed illustrative approach in comparison to classical rendering techniques.ConclusionThe presented illustration methods are beneficial for distance assessment in surgical AR. To increase the safety of interventions with the proposed approach, the reduction of inaccuracies in tracking and registration is a subject of our current research.

Real-Time Illustration of Vascular Structures

We present real-time vascular visualization methods, which extend on illustrative rendering techniques to particularly accentuate spatial depth and to improve the perceptive separation of important vascular properties such as branching level and supply area. The resulting visualization can and has already been used for direct projection on a patients organ in the operation theater where the varying absorption and reflection characteristics of the surface limit the use of color. The important contributions of our work are a GPU-based hatching algorithm for complex tubular structures that emphasizes shape and depth as well as GPU-accelerated shadow-like depth indicators, which enable reliable comparisons of depth distances in a static monoscopic 3D visualization. In addition, we verify the expressiveness of our illustration methods in a large, quantitative study with 160 subjects

Usability comparison of mouse-based interaction techniques for predictable 3d rotation

Due to the progress in computer graphics hardware high resolution 3d models may be explored at interactive frame rates, and facilities to explore them are a part of modern radiological workstations and therapy planning systems. Despite their advantages, 3d visualizations are only employed by a minority of potential users and even these employ 3d visualizations for a few selected tasks only. We hypothesize that this results from a lack of intuitive interaction techniques for 3d rotation. In this paper, we compare existing techniques with respect to design principles derived by clinical applications and present results of an empirical study. These results are relevant beyond clinical applications and strongly suggest that the presented design principles are crucial for comfortable and predictable interaction techniques for 3d rotation.

Illustrative shadows: integrating 3D and 2D information displays

Many exploration and manipulation tasks benefit from a coherent integration of multiple views onto complex information spaces. This paper proposes the concept of Illustrative Shadows for a tight integration of interactive 3D graphics and schematic depictions using the shadow metaphor. The shadow metaphor provides an intuitive visual link between 3D and 2D visualizations integrating the different displays into one combined information display. Users interactively explore spatial relations in realistic shaded virtual models while functional correlations and additional textual information are presented on additional projection layers using a semantic network approach. Manipulations of one visualization immediately influence the others, resulting in an in-formationally and perceptibly coherent presentation

Interactive visualization of multimodal volume data for neurosurgical tumor treatment

We present novel interactive methods for the visualization of multimodal volume data as used in neurosurgical therapy planning. These methods allow surgeons to explore multimodal volumes and focus on functional data and lesions. Computer graphics techniques are proposed to create expressive visualizations at interactive frame rates to reduce time‐consuming and complex interaction with the medical data. Contributions of our work are the distance‐based enhancements of functional data and lesions which allows the surgeon to perceive functional and anatomical structures at once and relate them directly to the intervention. In addition we propose methods for the visual exploration of the path to the structures of interest, to enhance anatomical landmarks, and to provide additional depth indicators. These techniques have been integrated in a visualization prototype that provides interaction capabilities for finding the optimal therapeutic strategy for the neurosurgeon.

Guided Exploration with Dynamic Potential Fields: the Cubical Path System

Exploring unknown models or scenes is a highly interactive and dynamic process. Systems for automatic presentation of models or scenes either require cinematographic rules, direct human interaction, framesets, or pre‐calculation of paths to a known goal. In this paper we present a system which can deal with rapidly changing user interest in objects of a scene or model as well as with dynamic models and changes of the camera position introduced interactively by the user or through cuts. We describe CubicalPath, a new potential field‐based camera control system that helps with the exploration of virtual environments.

CubicalPath-dynamic potential fields for guided exploration in virtual environments

Exploring unknown models or scenes is a highly interactive and dynamic process. Systems for automatic presentation of models or scenes either require cinematographic rules, direct human interaction, framesets or precalculation of paths to a known goal. We are looking for a system which can deal with rapidly changing user interest in objects of a scene or model as well as with dynamic models and changes of the camera position introduced interactively by the user or through cuts. In this paper we describe CubicalPath, a new potential field-based camera control system that helps with the exploration of virtual environments.

The Medical Exploration Toolkit: An Efficient Support for Visual Computing in Surgical Planning and Training

Application development is often guided by the usage of software libraries and toolkits. For medical applications, the toolkits currently available focus on image analysis and volume rendering. Advanced interactive visualizations and user interface issues are not adequately supported. Hence, we present a toolkit for application development in the field of medical intervention planning, training, and presentation-the MEDICALEXPLORATIONTOOLKIT (METK). The METK is based on the rapid prototyping platform MeVisLab and offers a large variety of facilities for an easy and efficient application development process. We present dedicated techniques for advanced medical visualizations, exploration, standardized documentation, and interface widgets for common tasks. These include, e.g., advanced animation facilities, viewpoint selection, several illustrative rendering techniques, and new techniques for object selection in 3D surface models. No extended programming skills are needed for application building, since a graphical programming approach can be used. The toolkit is freely available and well documented to facilitate the use and extension of the toolkit.

Workflow oriented software support for image guided radiofrequency ablation of focal liver malignancies

Image guided radiofrequency (RF) ablation has taken a significant part in the clinical routine as a minimally invasive method for the treatment of focal liver malignancies. Medical imaging is used in all parts of the clinical workflow of an RF ablation, incorporating treatment planning, interventional targeting and result assessment. This paper describes a software application, which has been designed to support the RF ablation workflow under consideration of the requirements of clinical routine, such as easy user interaction and a high degree of robust and fast automatic procedures, in order to keep the physician from spending too much time at the computer. The application therefore provides a collection of specialized image processing and visualization methods for treatment planning and result assessment. The algorithms are adapted to CT as well as to MR imaging. The planning support contains semi-automatic methods for the segmentation of liver tumors and the surrounding vascular system as well as an interactive virtual positioning of RF applicators and a concluding numerical estimation of the achievable heat distribution. The assessment of the ablation result is supported by the segmentation of the coagulative necrosis and an interactive registration of pre- and post-interventional image data for the comparison of tumor and necrosis segmentation masks. An automatic quantification of surface distances is performed to verify the embedding of the tumor area into the thermal lesion area. The visualization methods support representations in the commonly used orthogonal 2D view as well as in 3D scenes.

OPENNPAR: a system for developing, programming, and designing non-photorealistic animation and rendering

The notable amount and variation of current techniques in non-photorealistic rendering (NPR) indicates a level of maturity whereby the categorization of algorithms has become possible. We present a conceptual model for NPR, on which we base a modular system, OPENNPAR, which integrates NPR algorithms into distinct classes. Components in OPENNPAR are modularized and consequently reintegrated for various rendering purposes, allowing many kinds of NPR algorithms to be reproduced, including the integration of 2D and 3D methods. Additionally, the system provides support for a range of users (developers, programmers, designers) according to their respective levels of abstraction, thus being available in multiple contexts. Ultimately, OPENNPAR holds great potential as a tool in the development, augmentation, and creation of NPR effects.