Cagatay Basdogan

Haptics in virtual environments: Taxonomy, research status, and challenges

Abstract Haptic displays are emerging as effective interaction aids for improving the realism of virtual worlds. Being able to touch, feel, and manipulate objects in virtual environments has a large number of exciting applications. The underlying technology, both in terms of electromechanical hardware and computer software, is becoming mature and has opened up novel and interesting research areas. In this paper, we clarify the terminology of human and machine haptics and provide a brief overview of the progress recently achieved in these fields, based on our investigations as well as other studies. We describe the major advances in a new discipline, Computer Haptics (analogous to computer graphics), that is concerned with the techniques and processes associated with generating and displaying haptic stimuli to the human user. We also summarize the issues and some of our results in integrating haptics into multimodal and distributed virtual environments, and speculate on the challenges for the future.

An experimental study on the role of touch in shared virtual environments

Investigating virtual environments has become an increasingly interesting research topic for engineers, computer and cognitive scientists, and psychologists. Although there have been several recent studies focused on the development of multimodal virtual environments (VEs) to study human-machine interactions, less attention has been paid to human-human and human-machine interactions in shared virtual environments (SVEs), and to our knowledge, no attention paid at all to what extent the addition of haptic communication between people would contribute to the shared experience. We have developed a multimodal shared virtual environment and performed a set of experiments with human subjects to study the role of haptic feedback in collaborative tasks and whether haptic communication through force feedback can facilitate a sense of being and collaborating with a remote partner. The study concerns a scenario where two participants at remote sites must cooperate to perform a joint task in an SVE. The goals of the study are (1) to assess the impact of force feedback on task performance, (2) to better understand the role of haptic communication in human-human interactions, (3) to study the impact of touch on the subjective sense of collaborating with a human as reported by the participants based on what they could see and feel, and (4) to investigate if gender, personality, or emotional experiences of users can affect haptic communication in SVEs. The outcomes of this research can have a powerful impact on the development of next-generation human-computer interfaces and network protocols that integrate touch and force feedback technology into the internet, development of protocols and techniques for collaborative teleoperation such as hazardous material removal, space station.

Haptics in minimally invasive surgical simulation and training

Haptics is a valuable tool in minimally invasive surgical simulation and training. We discuss important aspects of haptics in MISST, such as haptic rendering and haptic recording and playback. Minimally invasive surgery has revolutionized many surgical procedures over the last few decades. MIS is performed using a small video camera, a video display, and a few customized surgical tools. In procedures such as gall bladder removal (laparoscopic cholesystectomy), surgeons insert a camera and long slender tools into the abdomen through small skin incisions to explore the internal cavity and manipulate organs from outside the body as they view their actions on a video display. Because the development of minimally invasive techniques has reduced the sense of touch compared to open surgery, surgeons must rely more on the feeling of net forces resulting from tool-tissue interactions and need more training to successfully operate on patients.

Virtual environments for medical training: graphical and haptic simulation of laparoscopic common bile duct exploration

We develop a computer-based training system to simulate laparoscopic procedures in virtual environments for medical training. The major hardware components of our system include a computer monitor to display visual interactions between 3D virtual models of organs and instruments together with a pair of force feedback devices interfaced with laparoscopic instruments to simulate haptic interactions. We simulate a surgical procedure that involves inserting a catheter into the cystic duct using a pair of laparoscopic forceps. This procedure is performed during laparoscopic cholecystectomy to search for gallstones in the common bile duct. Using the proposed system, the user can be trained to grasp and insert a flexible and freely moving catheter into the deformable cystic duct in virtual environments. The associated deformations are displayed on the computer screen and the reaction forces are fed back to the user through the force feedback devices. A hybrid modeling approach was developed to simulate the real-time visual and haptic interactions that take place between the forceps and the catheter, as well as the duct; and between the catheter and the duct.

Efficient Point-Based Rendering Techniques for Haptic Display of Virtual Objects

Computer haptics, an emerging field of research that is analogous to computer graphics, is concerned with the generation and rendering of haptic virtual objects. In this paper, we propose an efficient haptic rendering method for displaying the feel of 3-D polyhedral objects in virtual environments (VEs). Using this method and a haptic interface device, the users can manually explore and feel the shape and surface details of virtual objects. The main component of our rendering method is the neighborhood watch algorithm that takes advantage of precomputed connectivity information for detecting collisions between the end effector of a force-reflecting robot and polyhedral objects in VEs. We use a hierarchical database, multithreading techniques, and efficient search procedures to reduce the computational time such that the haptic servo rate after the first contact is essentially independent of the number of polygons that represent the object. We also propose efficient methods for displaying surface properties of objects such as haptic texture and friction. Our haptic-texturing techniques and friction model can add surface details onto convex or concave 3-D polygonal surfaces. These haptic-rendering techniques can be extended to display dynamics of rigid and deformable objects.

A robotic indenter for minimally invasive measurement and characterization of soft tissue response

The lack of experimental data in current literature on material properties of soft tissues in living condition has been a significant obstacle in the development of realistic soft tissue models for virtual reality based surgical simulators used in medical training. A robotic indenter was developed for minimally invasive measurement of soft tissue properties in abdominal region during a laparoscopic surgery. Using the robotic indenter, force versus displacement and force versus time responses of pig liver under static and dynamic loading conditions were successfully measured to characterize its material properties in three consecutive steps. First, the effective elastic modulus of pig liver was estimated as 10-15 kPa from the force versus displacement data of static indentations based on the small deformation assumption. Then, the stress relaxation function, relating the variation of stress with respect to time, was determined from the force versus time response data via curve fitting. Finally, an inverse finite element solution was developed using ANSYS finite element package to estimate the optimum values of viscoelastic and nonlinear hyperelastic material properties of pig liver through iterations. The initial estimates of the material properties for the iterations were extracted from the experimental data for faster convergence of the solutions.

On the Measurement of Dynamic Stability of Human Locomotion

The main focus of the present investigation is the development of quantitative measures to assess the dynamic stability of human locomotion. The analytical methodology is based on Floquet theory, which was developed to investigate the stability of nonlinear oscillators. Here the basic approach is modified such that it accommodates the study of the complex dynamics of human locomotion and differences among various individuals. A quantitative stability index has been developed to characterize the ability of humans to maintain steady gait patterns. Floquet multipliers of twenty normal subjects were computed from the kinematic data at Poincaré sections taken at four instants of the gait cycle, namely heel strike, foot flat, heel off, and toe off. Then, an averaged stability index was computed for each subject. Statistical analysis was performed to demonstrate the utility of the stability indices as quantitative measures of dynamic stability of gait for the subject population tested during the present study.

VR-Based Simulators for Training in Minimally Invasive Surgery

Simulation-based training using VR techniques is a promising alternative to traditional training in minimally invasive surgery (MIS). Simulators let the trainee touch, feel, and manipulate virtual tissues and organs through the same surgical tool handles used in actual MIS while viewing images of tool-tissue interactions on a monitor as in real laparoscopic procedures

The role of roles: Physical cooperation between humans and robots

Since the strict separation of working spaces of humans and robots has experienced a softening due to recent robotics research achievements, close interaction of humans and robots comes rapidly into reach. In this context, physical human–robot interaction raises a number of questions regarding a desired intuitive robot behavior. The continuous bilateral information and energy exchange requires an appropriate continuous robot feedback. Investigating a cooperative manipulation task, the desired behavior is a combination of an urge to fulfill the task, a smooth instant reactive behavior to human force inputs and an assignment of the task effort to the cooperating agents. In this paper, a formal analysis of human–robot cooperative load transport is presented. Three different possibilities for the assignment of task effort are proposed. Two proposed dynamic role exchange mechanisms adjust the robot’s urge to complete the task based on the human feedback. For comparison, a static role allocation strategy not relying on the human agreement feedback is investigated as well. All three role allocation mechanisms are evaluated in a user study that involves large-scale kinesthetic interaction and full-body human motion. Results show tradeoffs between subjective and objective performance measures stating a clear objective advantage of the proposed dynamic role allocation scheme.

Kinematics and Dynamic Stability of the Locomotion of Post-Polio Patients

The study reported in this article was conducted to propose a set graphical and analytical tools and assess their clinical utility by analyzing gait kinematics and dynamics of polio survivors. Phase-plane portraits and first return maps were used as graphical tools to detect abnormal patterns in the sagittal kinematics of post-polio gait. Two new scalar measures were introduced to assess the bilateral kinematic symmetry and dynamic stability of human locomotion. Nine healthy subjects and seventeen post-polio patients were involved in the project. Significant increases in the knee extension and ankle plantar flexion of post-polio patients were observed during the weight acceptance phases of their gait. Polio patients also exhibited highly noticeable excessive hip flexion during the swing phase of their ambulation. Using the proposed symmetry measure, we concluded that post-polio patients walked less symmetrically than normals. Our conclusion, however, was based on the bilateral symmetry in the sagittal plane only. Finally, we observed that post-polio patients walked significantly less stably than normals. In addition, weaknesses in lower extremity muscles of polio patients were found to be an important factor that affected stable ambulation.