Shigeki Nankaku

Tracking a Real Liver Using a Virtual Liver and an Experimental Evaluation with Kinect v2

In this study, we propose a smart transcription algorithm for translation and/or rotation motions. This algorithm has two phases: calculating the differences between real and virtual 2D depth images, and searching the motion space defined by three translation and three rotation degrees of freedom based on the depth differences. One depth image is captured for a real liver using a Kinect v2 depth camera and another depth image is obtained for a virtual liver (a polyhedron in stereo-lithography (STL) format by z-buffering with a graphics processing unit). The STL data are converted from Digital Imaging and Communication in Medicine (DICOM) data, where the DICOM data are captured from a patient’s liver using magnetic resonance imaging and/or a computed tomography scanner. In this study, we evaluated the motion precision of our proposed algorithm based on several experiments based using a Kinect v2 depth camera.

Depth Image Matching Algorithm for Deforming and Cutting a Virtual Liver via Its Real Liver Image Captured Using Kinect v2

In this paper, we propose a smart deforming and/or cutting transcription algorithm for rheology objects such as human livers. Moreover, evaluation of performance and shape precision under the proposed algorithm are experimentally verified by deforming a real clay liver and/or cutting a gel block prepared at human body temperature. First, we capture the image of the liver of a patient by digital imaging and communication in medicine (DICOM) generated by magnetic resonance imaging (MRI) and/or computed tomography (CT) scanner. Then, the DICOM data is segmented and converted into four types of stereo-lithography (STL) polyhedra, which correspond to the whole liver and three blood vessels. Second, we easily overlap the virtual and real liver images in our mixed reality (MR) surgical navigation system using our initial position/orientation/shape adjustment system that uses color images to differentiate between real and virtual depth images. After overlapping, as long as the real liver is deformed and/or cut by a human (doctor), the liver is constantly captured by Kinect v2. Subsequently, by using the real depth image captured in real time, many vertices around the virtual polyhedral liver in STL format are pushed/pulled by viscoelastic elements called the Kelvin–Voigt materials located on the vertices. Finally, after determining the displacements of the vertices, we obtain an adequately shaped STL. The vertex position required for fixing the shape is calculated using the Runge–Kutta method.

A new 2D depth-depth matching algorithm whose translation and rotation freedoms are separated

In this paper, we revise a previous 2D depth-depth-matching algorithm in order to copy motions fast from a real liver to a virtual liver in a surgical navigation. The real liver is always captured by 3D depth camera, and the virtual liver is represented by a polyhedron with STL format via DICOM captured by MRI/CT. In our algorithm, we firstly compare a 2D depth image in a real world and the Z-buffer of STL in a virtual world, and by using the difference of two depth images, we secondly search the best movement of a virtual liver from a huge number of possibilities with 3 translation and 3 rotation degrees-of-freedom. In this paper, we firstly divide translation and rotation D.O.F, and individually select the most adequate 3 DOF sets of a virtual liver following its real liver. Based on the division, we can find a sequence of following motions more precise and faster than our previous 2D depth-depth-matching algorithms.