Jungsik Kim

A Haptic Interaction Method Using Visual Information and Physically Based Modeling

Haptic feedback can be used to sense a physical environment at a remote site in order to overcome spatial or scale barriers in telemanipulation. The aim of this paper is to develop a haptic interaction method for a deformable object manipulation system by means of image processing and physically based modeling techniques. The interaction forces between the instrument driven by a haptic device and a deformable object are inferred in real time based on the visual information from a slave environment without force sensor. A physically based model of the deformable object is constructed by integrating the geometric information from vision, a priori knowledge of the object mechanical properties, and a predefined coordinate system of the slave environment. The forces are then derived from the model, while a boundary condition is updated based on the images (a tool-tip position tracking). In order to demonstrate the applicability and effectiveness of the proposed algorithm, macro- and microscale experimental systems were built and equipped with a telemanipulation system and a commercial haptic display. The proposed method was verified using silicone (macroscale) and zebrafish embryos (microscale).

Graphic and haptic modelling of the oesophagus for VR-based medical simulation.

Medical simulators with vision and haptic feedback have been applied to many medical procedures in recent years, due to their safe and repetitive nature for training. Among the many technical components of the simulators, realistic and interactive organ modelling stands out as a key issue for judging the fidelity of the simulation. This paper describes the modelling of an oesophagus for a real‐time laparoscopic surgical simulator.

Viscoelastic Characterization of Mouse Zona Pellucida

The viscoelastic properties of the zona pellucida (ZP), which is the extracellular coat surrounding an oocyte/embryo, are evaluated in this study. Previous studies demonstrate that ZP mechanical properties change during oocyte maturation, fertilization, and early embryo development, but linear pure elastic models currently being used do not satisfy the time-dependent mechanical behavior of the ZP. In this paper, nonlinear viscoelastic characterization was performed using the Hunt-Crossley model and the newly developed vision-based nanoforce estimation method. The results show that viscoelasticity is a physical property of the ZP that exhibits hysteresis. The stiffness and viscosity parameters simultaneously increase following fertilization, causing the stiffness and viscosity of the embryo ZP (ten samples) to be 2.57-fold and 4.44-fold greater, respectively, than that of the oocyte ZP (eleven samples). This behavior well describes the noncovalently cross-linked filamentous structure of the ZP, supporting zona hardening during fertilization as a mechanically relevant event.

Evaluation of Telerobotic Shared Control Strategy for Efficient Single-Cell Manipulation

Microinjection is a method for the delivery of exogenous materials into cells and is widely used in biomedical research areas such as transgenics and genomics. However, this direct injection is a time-consuming and laborious task, resulting in low throughput and poor reproducibility. Here, we describe a telerobotic shared control framework for microinjection, in which a micromanipulator is controlled by the shared motion commands of both the human operator and the autonomous controller. To determine the weightings between the operator and the controller, we proposed a quantitative evaluation method using a model of speed/accuracy trade-offs in human movement. The results showed that a 40%-60% weighting on the human operator (or the controller) produced the best performance for both speed and accuracy of guiding and targeting task in microinjection suggesting that some level of both automation and human involvement is important for microinjection tasks.

Real-time haptic rendering of slowly deformable bodies based on two dimensional visual information for telemanipulation

This paper presents a real-time haptic rendering technique for deformable objects based on visual information about the intervention between a tool and a real object in a remote place. A user can feel the artificial reaction force through a haptic device in real-time when a slave system exerts manipulation tasks on a deformable object. Models of the deformable object and the manipulator are created from the captured image obtained with a CCD camera and the objects are recognized using image processing techniques. The force at a rate of 1 kHz for stable haptic interaction is deduced by extrapolating forces at a low update rate. The rendering algorithm developed was tested and validated on a test platform consisting of a one-dimensional indentation device and an off-the shelf force feedback device.

Haptic Feedback Based on Physically Based Modeling for Cellular Manipulation Systems

This paper focuses on developing a haptic rendering technique for cellular manipulation using image processing techniques and physically based models. The interaction forces between a micropipette and cellular tissues are predicted based on biomechanical models of cells, which consist of a boundary element model and a prior knowledge of the cells mechanical properties. These models are used to allow users to feel amplified reaction forces during cell injection tasks through a haptic device in real time. The experimental system, equipped with a micro-injection system and a commercial haptic display, was developed and tested using zebrafish embryos. The proposed haptic rendering algorithm could be used to improve success rates of cellular manipulation tasks.

A physically-based haptic rendering for telemanipulation with visual information: Macro and micro applications

This paper presents a haptic rendering technique for a telemanipulation system of deformable objects using image processing techniques and physically based modeling. The interaction forces between an instrument driven by a haptic device and a deformable object are inferred in real time based on a continuum mechanics model of the object, which consists of a boundary element model and a prior knowledge of the objectpsilas mechanical properties. These models allow users to feel reaction forces during manipulation tasks through a haptic device. Macro- and micro-scale experimental systems, equipped with a telemanipulation system and a commercial haptic display, were developed and tested using silicone (macro-scale) and zebrafish embryos (micro-scale). The developed algorithm can be used with a needle operation robot or a cellular injection system.

Gallbladder Removal Simulation for Laparoscopic Surgery Training: A Hybrid Modeling Method

Laparoscopic surgery has many advantages, but it is difficult for a surgeon to achieve the necessary surgical skills. Recently, virtual training simulations have been gaining interest because they can provide a safe and efficient learning environment for medical students and novice surgeons. In this paper, we present a hybrid modeling method for simulating gallbladder removal that uses both the boundary element method (BEM) and the finite element method (FEM). Each modeling method is applied according to the deformable properties of human organs: BEM for the liver and FEM for the gallbladder. Connective tissues between the liver and the gallbladder are also included in the surgical simulation. Deformations in the liver and the gallbladder models are transferred via connective tissue springs using a mass-spring method. Special effects and techniques are developed to achieve realistic simulations, and the software is integrated into a custom-designed haptic interface device. Various computer graphical techniques are also applied in the virtual gallbladder removal laparoscopic surgery training. The detailed techniques and the results of the simulations are described in this paper.

Inclusion detection with haptic-palpation system for medical telediagnosis

Detection of abnormalities in subcutaneous tissue is an important issue in medical diagnosis. Surgeons palpate this type of tissue to perceive pathologies through haptic sensation. This paper presents the framework of a real-time haptic-palpation system with inclusion detection to manipulate biological soft tissues and perceive their structures and material properties non-invasively. A user with a haptic device guides a robotic manipulator to perform palpation tasks and can detect the location of inclusions inside a soft medium (silicone-molded tissue phantom). For the objective measurement of the depth of the inclusion, a finite element (FE) model was developed to predict the reaction forces for various inclusion depths. The methods presented in this paper can be applied to the early detection of prostate cancer or breast cancer in telediagnosis.

ABS Sliding Mode Control considering Optimum Road Friction Force of Tyre

This paper presents the sliding mode control methods for anti-lock brake system (ABS) with the friction force observer. Using a simplified quarter car model, the sliding mode controller for ABS is designed to track the desired wheel slip ratio. Here, new method to find the desired wheel slip ratio which produces the maximum friction force between road and tire is suggested. The desired wheel slip ratio is varying according road and tire conditions to produce maximum friction force. In order to find optimum desired wheel slip ratio, the sliding mode observer for friction force is used. The proposed sliding mode controller with observer is evaluated in simulation, and the control design is shown to have high performance on roads with constant and varying adhesion coefficients.