Hyun K. Kim

Haptics in minimally invasive surgical simulation and training

Haptics is a valuable tool in minimally invasive surgical simulation and training. We discuss important aspects of haptics in MISST, such as haptic rendering and haptic recording and playback. Minimally invasive surgery has revolutionized many surgical procedures over the last few decades. MIS is performed using a small video camera, a video display, and a few customized surgical tools. In procedures such as gall bladder removal (laparoscopic cholesystectomy), surgeons insert a camera and long slender tools into the abdomen through small skin incisions to explore the internal cavity and manipulate organs from outside the body as they view their actions on a video display. Because the development of minimally invasive techniques has reduced the sense of touch compared to open surgery, surgeons must rely more on the feeling of net forces resulting from tool-tissue interactions and need more training to successfully operate on patients.

Two-photon deep tissue ex vivo imaging of mouse dermal and subcutaneous structures

The non-invasive determination of deep tissue three dimensional structure and biochemistry is the ultimate goal of optical biopsy. Two-photon microscopy has been shown to be a particularly promising approach. The use of infrared radiation in two-photon microscopy is critical for deep tissue imaging since tissue absorption and scattering coefficients for infrared light are much lower than for shorter wavelengths. Equally important, tissue photodamage is localized to the focal region where fluorescence excitation occurs. This report demonstrates that, by means of high resolution two-photon microscopy, skin and subcutaneous tissue structures can be imaged utilizing their endogenous fluorescence. From a freshly prepared tissue punch of a mouse ear, we were able to 3D resolve both the living and cornified keratinocytes in the epidermis, the collagen/elastin fibers in the dermal layer and the cartilage in the subcutaneous layer. The ability to non-invasively acquire 3D structures of these tissue components may find application in areas such as non-invasive diagnosis of skin cancer and the study of wound healing processes.

Transatlantic touch: a study of haptic collaboration over long distance

The extent to which the addition of haptic communication between human users in a shared virtual environment (SVE) contributes to the shared experience of the users has not received much attention in the literature. In this paper we describe a demonstration of and an experimental study on haptic interaction between two users over a network of significant physical distance and a number of network hops. A number of techniques to mitigate instability of the haptic interactions induced by network latency are presented. An experiment to evaluate the use of haptics in a collaborative situation mediated by a networked virtual environment is examined. The experimental subjects were to cooperate in lifting a virtual box together under one of four conditions in a between-groups design. Questionnaires were used to report the ease with which they could perform the task and the subjective levels of presence and copresence experienced. This extends earlier work by the authors to consider the possibility of haptic collaboration under real network conditions with a number of improvements. Using the technology described in this paper, transatlantic touch was successfully demonstrated between the Touch Lab at Massachusetts Institute of Technology, USA and Virtual Environments and Computer Graphics (VECG) lab at University College London (UCL), UK in 2002. It was also presented at the Internet II demonstration meeting in 2002 between University of Southern California and the Massachusetts Institute of Technology.

Continuous shared control for stabilizing reaching and grasping with brain-machine interfaces

Research on brain-machine interfaces (BMIs) is directed toward enabling paralyzed individuals to manipulate their environment through slave robots. Even for able-bodied individuals, using a robot to reach and grasp objects in unstructured environments can be a difficult telemanipulation task. Controlling the slave directly with neural signals instead of a hand-master adds further challenges, such as uncertainty about the intended trajectory coupled with a low update rate for the command signal. To address these challenges, a continuous shared control (CSC) paradigm is introduced for BMI where robot sensors produce reflex-like reactions to augment brain-controlled trajectories. To test the merits of this approach, CSC was implemented on a 3-degree-of-freedom robot with a gripper bearing three co-located range sensors. The robot was commanded to follow eighty-three reach-and-grasp trajectories estimated previously from the outputs of a population of neurons recorded from the brain of a monkey. Five different levels of sensor-based reflexes were tested. Weighting brain commands 70% and sensor commands 30% produced the best task performance, better than brain signals alone by more than seven-fold. Such a marked performance improvement in this test case suggests that some level of machine autonomy will be an important component of successful BMI systems in general.

Biped humanoid robot Mahru III

This paper presents a new biped humanoid robot Mahru III (150 cm, 62 Kg) developed by Samsung Electronics Co., Ltd in 2007. The ultimate goal of this project is to design biped humanoid robots connected to a wireless network environment so that they can be used in our daily life. As the first step, Mahru III was built as a pilot platform for future household companion robots. In this paper we focus on the design of the mechanical system, electrical/electronic parts, and the control algorithm with some experimental results. We designed the mechanical system to achieve lightweight, high stiffness and high energy efficiency through CAE(computer aided engineering) analysis and DOE(design of experiments)-based optimization. We also built an extensible electrical/electronic system to provide for future network-based applications. Walking control algorithm is realized to make stable walking on an uneven floor possible. The experiments of the first phase shows that our humanoid can walk at 1.3 km/h and overcome 1 cm protrusions successfully.

The Role of Simulation Fidelity in Laparoscopic Surgical Training

Although there have been significant advances in the development of virtual reality based surgical simulations, there still remain fundamental questions concerning the fidelity required for effective surgical training. A dual station experimental platform was built for the purpose of investigating these fidelity requirements. Analogous laparoscopic surgical tasks were implemented in a virtual and a real station, with the virtual station modeling the real environment to various degrees of fidelity. After measuring subjects’ initial performance in the real station, different groups of subjects were trained on the virtual station under a variety of conditions and tested finally at the real station. Experiments involved bimanual pushing and cutting tasks on a nonlinear elastic object. The results showed that force feedback results in a significantly improved training transfer compared to training without force feedback. The training effectiveness of a linear approximation model was comparable to the effectiveness of a more accurate nonlinear model.

Developments in brain-machine interfaces from the perspective of robotics.

Many patients suffer from the loss of motor skills, resulting from traumatic brain and spinal cord injuries, stroke, and many other disabling conditions. Thanks to technological advances in measuring and decoding the electrical activity of cortical neurons, brain-machine interfaces (BMI) have become a promising technology that can aid paralyzed individuals. In recent studies on BMI, robotic manipulators have demonstrated their potential as neuroprostheses. Restoring motor skills through robot manipulators controlled by brain signals may improve the quality of life of people with disability. This article reviews current robotic technologies that are relevant to BMI and suggests strategies that could improve the effectiveness of a brain-operated neuroprosthesis through robotics.

Estimation of Multijoint Stiffness Using Electromyogram and Artificial Neural Network

The human arm exhibits outstanding manipulability in executing various tasks by taking advantage of its intrinsic compliance, force sensation, and tactile contact clues. By examining human strategy in controlling arm impedance, we may be able to understand underlying human motor control and develop control methods for dexterous robotic manipulation. This paper presents a novel method for estimating multijoint stiffness by using electromyogram (EMG) and an artificial neural network model. The artificial network model developed in this paper relates EMG data and joint motion data to joint stiffness. With the proposed method, the multijoint stiffness of the arm was estimated without complex calculation or specialized apparatus. The feasibility of the proposed method was confirmed through experimental and simulation results.

Human haptic perception is interrupted by explorative stops of milliseconds

Introduction: The explorative scanning movements of the hands have been compared to those of the eyes. The visual process is known to be composed of alternating phases of saccadic eye movements and fixation pauses. Descriptive results suggest that during the haptic exploration of objects short movement pauses occur as well. The goal of the present study was to detect these “explorative stops” (ES) during one-handed and two-handed haptic explorations of various objects and patterns, and to measure their duration. Additionally, the associations between the following variables were analyzed: (a) between mean exploration time and duration of ES, (b) between certain stimulus features and ES frequency, and (c) the duration of ES during the course of exploration. Methods: Five different Experiments were used. The first two Experiments were classical recognition tasks of unknown haptic stimuli (A) and of common objects (B). In Experiment C space-position information of angle legs had to be perceived and reproduced. For Experiments D and E the PHANToM haptic device was used for the exploration of virtual (D) and real (E) sunken reliefs. Results: In each Experiment we observed explorative stops of different average durations. For Experiment A: 329.50 ms, Experiment B: 67.47 ms, Experiment C: 189.92 ms, Experiment D: 186.17 ms and Experiment E: 140.02 ms. Significant correlations were observed between exploration time and the duration of the ES. Also, ES occurred more frequently, but not exclusively, at defined stimulus features like corners, curves and the endpoints of lines. However, explorative stops do not occur every time a stimulus feature is explored. Conclusions: We assume that ES are a general aspect of human haptic exploration processes. We have tried to interpret the occurrence and duration of ES with respect to the Hypotheses-Rebuild-Model and the Limited Capacity Control System theory.

Biologically Inspired Energy Efficient Walking for Biped Robots

Prior methods of bipedal humanoid walking through high gain position control is energy intensive. This paper introduces a biologically inspired method for robot locomotion that improves energy efficiency. Neural oscillators were used as the central pattern generator of the gait pattern and were able to entrain the natural frequency for robot locomotion. The impedance of the legs was modulated so that the legs can swing more freely during swing phase and control position accurately during support phase. Dynamic simulations of humanoid robot walking with the proposed method resulted in energy efficiency improvement of 40.5%. Also, the modulation of impedance resulted in a more stable contact with the ground during landing of the foot.