Megumi Nakao

SpaceTag: an overlaid virtual system and its applications

SpaceTag is an object system on which each object called SpaceTag can be accessed only from limited locations and a limited time period. Its applications include entertainment systems, advertisement services, bulletin board systems, and personal communication systems. For one-way communication, they are broadcasted from the server; for two-way communication, users can cut and paste SpaceTags between their portable PCs and the real space. The SpaceTag system is a location-aware information system, as well as an augmented reality system because it attaches information to the real space. However, we categorize it as an overlaid virtual system, because it has no direct link to real objects. It can be realized as a public service without causing drastic change of this society, and without much cost.

Interaction model between elastic objects for haptic feedback considering collisions of soft tissue

The simulation of organ-organ interaction is indispensable for practical and advanced medical VR simulator such as open surgery and indirect palpation. This paper describes a method to represent real-time interaction between elastic objects for accurate force feedback in medical VR simulation. The proposed model defines boundary deformation of colliding elements based on temporary surface forces calculated by temporary deformation. The model produces accurate deformation and force feedback considering collisions of objects as well as prevents unrealistic overlap of objects. A prototype simulator of rectal palpation is constructed on general desktop PC with a haptic device, PHANToM. The system allows users to feel different stiffness of a rear elastic object located behind another elastic object. The results of experiments confirmed the method expresses organ-organ interaction in real-time and produces realistic and perceivable force feedback.

Transferring bioelasticity knowledge through haptic interaction

This study establishes a practical environment for transferring knowledge on bioelasticity between expert and trainee medical practitioners. Through haptic interaction with a deformable virtual anatomical model, experts set the models elasticity conditions by simulating a surgical procedure. Trainees experience the elasticity by attempting the same surgical manipulation

Physics-Based Interactive Volume Manipulation for Sharing Surgical Process

This paper presents a new set of techniques by which surgeons can interactively manipulate patient-specific volumetric models for sharing surgical process. To handle physical interaction between the surgical tools and organs, we propose a simple surface-constraint-based manipulation algorithm to consistently simulate common surgical manipulations such as grasping, holding and retraction. Our computation model is capable of simulating soft-tissue deformation and incision in real time. We also present visualization techniques in order to rapidly visualize time-varying, volumetric information on the deformed image. This paper demonstrates the success of the proposed methods in enabling the simulation of surgical processes, and the ways in which this simulation facilitates preoperative planning and rehearsal.

Haptic reproduction and interactive visualization of a beating heart for cardiovascular surgery simulation

This paper aims to achieve haptic reproduction and real-time visualization of a beating heart for cardiac surgery simulation. Unlike most forgoing approaches, the authors focus on time series datasets and propose a new framework for interactive simulation of active tissues. The framework handles both detection and response of collisions between a manipulator and a beating virtual heart. Physics-based force feedback of autonomous cardiac motion is also produced based on a stress-pressure model, which is adapted to elastic objects filled with fluid. Time series datasets of an adult man were applied to an integrated simulation system with a force feedback device. The system displays multi-dimensional representation of a beating heart and provides a basic training environment for surgical palpation. Finally, results of measurement and medical assessment confirm the achieved quality and performance of the presented framework.

Interactive bone drilling using a 2D pointing device to support Microendoscopic Discectomy planning.

PurposeTo support preoperative planning of bone drilling for Microendoscopic Discectomy, we present a set of interactive bone-drilling methods using a general 2D pointing device.MethodsUnlike the existing methods, our framework has the following features: (1) the user can directly cut away arbitrary 3D regions on the volumetrically rendered image, (2) in order to provide a simple interface to end-users, our algorithms make 3D drilling possible through only a general-purpose wheel mouse, (3) to reduce both over-drilling and unnatural drilling of an unintended region, we introduce a smart depth control to ensure the continuity of the cutting operation and (4) a GPU-based rendering scheme for high-quality shading of clipped boundaries.ResultsWe applied our techniques to some CT data of specific patients. Several experiments confirmed that the user was able to directly drill a 3D complex region on a volumetrically rendered lumber spine through simple mouse operation. Also, our rendering scheme clearly visualizes time-varying drilled surfaces at interactive rates. By comparing simulation results to actual postoperative CT images, we confirmed the user interactively simulates similar cutting to that carried out in real surgery.ConclusionWe concluded our techniques perform mouse-based, direct drilling of complex 3D regions with high-quality rendering of drilled boundaries and contribute to preoperative planning of Microendoscopic Discectomy.

Adaptive proxy geometry for direct volume manipulation

This paper introduces a new design to allow interactive, direct manipulation of volume data on volumetrically rendered images. We present an adaptive volume proxy mesh which serves not to define surfaces, but to encode the geometry and physical state of the volume. This system performs a modeling-free form of direct volume deformation by adaptively constructing the whole geometric structure on the background while keeping the structure transparent to the user. A complex, high-frequency structure is locally encoded into the mesh on-the-fly based on the users volume of interest. This paper also presents a scheme for volume shading/shadowing applied in combination with our framework for improving reality in the visualization of time-varying deformable objects. We demonstrate examples of geometry-encoded direct volume manipulation and application to volume exploration and surgical simulation.

Physics-Based Simulation of Surgical Fields for Preoperative Strategic Planning

Although careful planning of surgical approach is a key for success of surgery, conventional planning and simulation tools cannot support detailed discussion. This issue is derived from the difficulty of estimating complex physical behavior of soft tissues provided by a series of surgical procedures like cutting and deformation. This paper proposes an adaptive physics-based framework that simulates both interactive cutting and accurate deformation on virtual bodies, and performs preoperative planning for supporting strategic discussion. We focus on limited use of the two models: A particle-based model and an FEM-based model considering required quality and performance in different situations. FEM-based deformation of incision accurately produces estimated surgical fields. Based on the framework, a strategic planning system was developed for supporting decision of surgical approach using 3D representation of the surgical fields. We applied clinical CT dataset of an aortic aneurysm case to the system. Some experiments and usability tests confirmed that the system contributes to grasping 3D shape and location of the target organs and performs detailed discussion on patient-specific surgical approaches.

Combining Volumetric Soft Tissue Cuts for Interventional Surgery Simulation

This paper proposes a framework to simulate soft tissue cuts for interventional surgery simulation. A strained status of soft tissues is modeled as internal tension between adjacent vertices in a particle based model. Both remodeling particle systems and an adaptive scheme in tetrahedral subdivision provide volumetric and smooth cuts on large virtual objects. 3D MRI datasets are applied to a developed system with a force feedback device. Measurement of the calculation time and visualization of simulation quality confirms that the framework contributes to surgical planning and training with tissue cutting.

Volumetric Fibular Transfer Planning With Shape-Based Indicators in Mandibular Reconstruction

In preoperative planning for mandibular reconstructive surgery, it is necessary to determine the osteotomy lines for fibular shaping and the proper placement of fibular segments in the mandible. Although virtual surgical planning has been utilized in preoperative decision making, current software designs require manual operation and a trial-and-error process to refine the reconstruction plan. We have developed volumetric fibular transfer simulation software that can quickly design a preoperative plan based on direct volume manipulation and quantitative comparison with the patients original mandible. We propose three quantitative shape indicators-volume ratio, contour error, and maximum projection-for symmetrical lesions of the mandible, and have implemented a parallel computation algorithm for the semiautomatic placement of fibular segments. Using this virtual planning software, we conducted a retrospective study of the computed tomography data from nine patients. We found that combining direct volume manipulation with real-time local search of placement improved the applicability of the planning system to optimize mandibular reconstruction.