Katsuhiko Onishi

Interactive modeling of trees by using growth simulation

We propose a real-time interactive system that enables users to generate, manipulate and edit the shape model of a tree based on growth simulation by directly indicating its global and spatial information. For this purpose, three-dimensional (3D) spatial information is introduced to the well-known L-system as an attribute of the growth simulation. Moreover, we propose an efficient data structure of L-strings in order to speed up the process.

Virtual Liver Surgical Simulator by Using Z-Buffer for Object Deformation

Virtual surgical simulator which is using computer graphics is much popular system than before. It is generally used in the medical areas, such as medical hospital or medical university. The simulator uses virtual organ models like liver, brain and so on. These models are usually based on the scanning data from patients and are used as volume models. Fortunately, the volume model is familiar with its cutting or deforming operation in a surgical system. For this reason, there are many kinds of surgical simulation or navigation systems using the volume model. However, visual reality of the volume model is not sufficient for human being including doctors. This means that the doctors cannot identify shape or location of a target organ from volume objects. In order to overcome this, we should use the translating method, such as marching cubes method and so on, for getting precisely polygon models which is included normal vectors of volume object. However, the method is quite time consuming and consequently the doctors cannot operate the virtual model in real-time.

Depth Camera Calibration and Knife Tip Position Estimation for Liver Surgery Support System

We have developed a liver surgery support system that uses two depth cameras and measures positional relationships between a surgical knife and a liver in real time. In this report, the overview of our system, the method for depth camera calibration, the estimation for knife tip positioning, and some experimental results are described.

Modeling of trees with interactive l-system and 3d gestures

We propose a modeling system that enables users to create tree models with 3D gesture input and Interactive L-system. It generates tree models by using growth simulation based on the trunk or silhouette shapes of trees given by user gestures. The Interactive L-system is one of the growth simulation algorithm, having spatial information of tree models, and allows users to generate, manipulate, and edit the shape of tree models by user’s direct input interactively. The system carefully addresses the fragile balance and tradeoff between the freedom of user interaction and the autonomy of tree growth. Users intuitively and easily create tree models that have the exact features of branching structures or the silhouette shape of trees according to user intentions and imagination.

Tracking a Real Liver Using a Virtual Liver and an Experimental Evaluation with Kinect v2

In this study, we propose a smart transcription algorithm for translation and/or rotation motions. This algorithm has two phases: calculating the differences between real and virtual 2D depth images, and searching the motion space defined by three translation and three rotation degrees of freedom based on the depth differences. One depth image is captured for a real liver using a Kinect v2 depth camera and another depth image is obtained for a virtual liver (a polyhedron in stereo-lithography (STL) format by z-buffering with a graphics processing unit). The STL data are converted from Digital Imaging and Communication in Medicine (DICOM) data, where the DICOM data are captured from a patient’s liver using magnetic resonance imaging and/or a computed tomography scanner. In this study, we evaluated the motion precision of our proposed algorithm based on several experiments based using a Kinect v2 depth camera.

Haptic AR Dental Simulator Using Z-buffer for Object Deformation

Dental surgical simulator could be one of the efficient tools to learning and practicing dental surgical skills. To these simulators, the visual and tactile feedback is desirable to be processed in real time. And, in the dental operation, the hand position during operations is one of the skills to learn and practice. Therefore, we develop the dental surgical simulator which use virtual tooth surface model for processing real time rendering. And we develop a display system which allow users to training dental operation by a right hand position.. The tooth model is deformed by cutting and drilling operation using haptic device. And the display is set close to users hand position and shows combined image with virtual tooth model as a surgical target and a real tooth model as other parts of the patient dental model. The system uses a collision detection and deformation method by using Z-buffer for virtual objects. This method enables users to view the complex shape of virtual tooth model by the surgical operation tasks and practicing dental surgical tasks. We developed prototype system and confirmed about the capability of our system.

Development of a Surgical Knife Attachment with Proximity Indicators

To prevent liver surgical accidents, we have been developing a surgery navigation system with two depth cameras with different characteristics. In this paper, we describe our developed surgical knife attachment which indicates the proximity of the blood vessels in the liver by light and sound. The method to calibrate the tip position of the knife in our camera system is also mentioned. Using this attachment, some experiments to navigate operator were conducted and the results indicated that the navigation worked correctly within the target range.

Depth Image Matching Algorithm for Deforming and Cutting a Virtual Liver via Its Real Liver Image Captured Using Kinect v2

In this paper, we propose a smart deforming and/or cutting transcription algorithm for rheology objects such as human livers. Moreover, evaluation of performance and shape precision under the proposed algorithm are experimentally verified by deforming a real clay liver and/or cutting a gel block prepared at human body temperature. First, we capture the image of the liver of a patient by digital imaging and communication in medicine (DICOM) generated by magnetic resonance imaging (MRI) and/or computed tomography (CT) scanner. Then, the DICOM data is segmented and converted into four types of stereo-lithography (STL) polyhedra, which correspond to the whole liver and three blood vessels. Second, we easily overlap the virtual and real liver images in our mixed reality (MR) surgical navigation system using our initial position/orientation/shape adjustment system that uses color images to differentiate between real and virtual depth images. After overlapping, as long as the real liver is deformed and/or cut by a human (doctor), the liver is constantly captured by Kinect v2. Subsequently, by using the real depth image captured in real time, many vertices around the virtual polyhedral liver in STL format are pushed/pulled by viscoelastic elements called the Kelvin–Voigt materials located on the vertices. Finally, after determining the displacements of the vertices, we obtain an adequately shaped STL. The vertex position required for fixing the shape is calculated using the Runge–Kutta method.

A new 2D depth-depth matching algorithm whose translation and rotation freedoms are separated

In this paper, we revise a previous 2D depth-depth-matching algorithm in order to copy motions fast from a real liver to a virtual liver in a surgical navigation. The real liver is always captured by 3D depth camera, and the virtual liver is represented by a polyhedron with STL format via DICOM captured by MRI/CT. In our algorithm, we firstly compare a 2D depth image in a real world and the Z-buffer of STL in a virtual world, and by using the difference of two depth images, we secondly search the best movement of a virtual liver from a huge number of possibilities with 3 translation and 3 rotation degrees-of-freedom. In this paper, we firstly divide translation and rotation D.O.F, and individually select the most adequate 3 DOF sets of a virtual liver following its real liver. Based on the division, we can find a sequence of following motions more precise and faster than our previous 2D depth-depth-matching algorithms.

AR Dental Surgical Simulator Using Haptic Feedback

We describe about our dental surgical simulator which enable users to simulate dental surgical operation. Our simulator which enables the user to learn dental surgical methods through actual hand and body postures. The proposed system uses a display showing a virtual tooth model and real teeth and gums that are positioned close to the hands of the user, which allows the user to directly manipulate objects with haptic feedback. As a preliminary evaluation, in display system, we measured the deviation between real object image and virtual object image at user’s view positions. And we confirmed the capability and the limitation of our system.