Christian Krapichler

Virtual reality in medicine-computer graphics and interaction techniques

The paper describes several new visualization and interaction techniques that enable the use of virtual environments for routine medical purposes. A new volume-rendering method supports shaded and transparent visualization of medical image sequences in real-time with an interactive threshold definition. Based on these rendering algorithms two complementary segmentation approaches offer an intuitive assistance for a wide range of requirements in diagnosis and therapy planning. In addition, a hierarchical data representation for geometric surface descriptions guarantees an optimal use of available hardware resources and prevents inaccurate visualization. The combination of the presented techniques empowers the improved human-machine interface of virtual reality to support every interactive task in medical three-dimensional (3-D) image processing, from visualization of unsegmented data volumes up to the simulation of surgical procedures.

VR interaction techniques for medical imaging applications

Methods of virtual reality (VR) offer new ways of human-computer interaction. Medicine is predestined to benefit from this new technology in many ways. Virtual environments can support physicians in their work, alleviate communication between specialists from different fields or be established in educational and training applications. For the field of visualization and analysis of three-dimensional anatomical images (e.g. CT or MRI scans), an application is introduced which expedites recognition of spatial coherencies and the exploration and manipulation of the 3D data. To avoid long periods of learning and accustoming and to facilitate work in such an environment, a powerful human-oriented interface is required allowing interactions similar to the real world and utilization of our natural experiences. This paper shows the use of eye tracking parameters for a level-of-detail algorithm and the integration of a glove-based hand gesture recognition into the virtual environment as an essential component of the human-machine interface. Furthermore, virtual bronchoscopy and virtual angioscopy are presented as examples for the use of the virtual environment.

A human-machine interface for medical image analysis and visualization in virtual environments

Virtual worlds open new dimensions in human-machine and even human-human communication. Medicine is predestined to benefit from this new technology in many ways. For the field of visualization and analysis of tomography data, an application is introduced which expedites identification of spatial coherencies and exploration of pathological regions. To facilitate work in such an environment and to avoid long periods of accustoming, a human-oriented interface is required allowing physicians to interact as close to the real world as possible. Hand gesture recognition (with a data glove) and eye tracking (using biosignals) are essential parts to fulfil this demand. Their integration into the virtual environment as two components of the human-machine interface is presented.

Physicians in virtual environments — multimodal human–computer interaction

Abstract Modern tomography technologies like CT or MRI produce high-quality scans of the human anatomy. While conventional computer-aided image analysis falls back upon editing tomograms layer by layer, virtual environments offer enhanced visualization, image analysis and manipulation of the three-dimensional data sets. In this paper, the application of multimodal, user-oriented human–computer interaction is presented, facilitating and accelerating work with the tomographical data of individual patients. Hand gesture recognition is a major component of the interface, completed by speech understanding and further units like a 6-DOF mouse or acoustic feedback. Three-dimensional image segmentation, virtual bronchoscopy and virtual angioscopy are typical examples that illustrate the benefits of virtual environments for the realm of medicine.

Virtual reality and multimedia human-computer interaction in medicine

Tomography devices produce high-quality scans of the human anatomy. While conventional computer-aided image analysis falls back upon editing tomograms layer by layer, virtual environments offer enhanced visualization, image analysis and manipulation of the three-dimensional (3D) data. In this paper, the application of multimodal, user-oriented human-computer interaction is presented. Hand gesture recognition is a major component of the interface, completed by speech understanding and further units like a 6-DOF mouse or acoustic feedback. 3D image segmentation and virtual endoscopy are typical examples to demonstrate virtual environments in medicine.

Speech interaction in virtual reality

A system for the visualization of three-dimensional anatomical data, derived from magnetic resonance imaging (MRI) or computed tomography (CT), enables the physician to navigate through and interact with the patients 3D scans in a virtual environment. This paper presents the multimodal human-machine interaction focusing the speech input. For the concerned task, a speech understanding front-end using a special kind of semantic decoder was successfully adopted. Now, the navigation as well as certain parameters and functions can be directly accessed by spoken commands. Using the implemented interaction modalities, the speed and efficiency of the diagnosis could be considerably improved.

Virtual bronchoscopy based on spiral CT images

Purpose: To improve the diagnosis of pathologic modified airways, a visualization system has been developed and tested based on the techniques of digital image analysis, synthesis of spiral CT and the visualization by methods of virtual reality. Materials and Methods: 20 patients with pathologic modifications of the airways (tumors, obstructions) were examined with Spiral-CT. The three-dimensional shape of the airways and the lung tissue is defined by a semiautomatic volume growing method and a following geometric surface reconstruction. This is the basis of a multidimensional display system which visualizes volumes, surfaces and computation results simultaneously. To enable the intuitive and immersive inspection of the airways a virtual reality system, consisting of two graphic engines, a head mounted display system, data gloves and specialized software was integrated. Results: In 20 cases the extension of the pathologic modification of the airways could be visualized with the virtual bronchoscopy. The user interacts with and manipulates the 3D model of the airways in an intuitive and immersive way. In contrast to previously proposed virtual bronchoscopy systems the described method permits truly interactive navigation and detailed exploration of anatomic structures. The system enables a user oriented and fast inspection of the volumetric image data. Conclusion: To support radiological diagnosis with additional information in an easy to use and fast way a virtual bronchoscopy system was developed. It enables the immersive and intuitive interaction with 3D Spiral CTs by truly 3D navigation within the airway system. The complex anatomy of the central tracheobronchial system could be clearly visualized. Peripheral bronchi are displayed up to 5th degree.

Human-Machine Interface for a VR-based Medical Imaging Environment

Modern 3D scanning techniques like magnetic resonance imaging (MRI) or computed tomography (CT) produce high- quality images of the human anatomy. Virtual environments open new ways to display and to analyze those tomograms. Compared with todays inspection of 2D image sequences, physicians are empowered to recognize spatial coherencies and examine pathological regions more facile, diagnosis and therapy planning can be accelerated. For that purpose a powerful human-machine interface is required, which offers a variety of tools and features to enable both exploration and manipulation of the 3D data. Man-machine communication has to be intuitive and efficacious to avoid long accustoming times and to enhance familiarity with and acceptance of the interface. Hence, interaction capabilities in virtual worlds should be comparable to those in the real work to allow utilization of our natural experiences. In this paper the integration of hand gestures and visual focus, two important aspects in modern human-computer interaction, into a medical imaging environment is shown. With the presented human- machine interface, including virtual reality displaying and interaction techniques, radiologists can be supported in their work. Further, virtual environments can even alleviate communication between specialists from different fields or in educational and training applications.

Automatic segmentation of the bronchial tract from spiral-CT-data for virtual reality representation

Virtual bronchoscopy is one method using a virtual reality equipment to improve diagnostical information drawn from CT data. As a preprocessing step the bronchial tract has to be extracted from the data. Therefore a method has been developed, which allows automatic segmentation of the bronchial tract. In the preprocessing step a backbone to the bronchial tract is extracted, which allows using more sophisticated methods for segmentation. In the second step either minimal variance region growing or an Active Contour Model is used. The first one uses statistical information to extend the presegmented area according to the local greyvalue distribution. The second method automatically fits the predefined contour to the local gradient information.