François Malric

A New Hand-Measurement Method to Simplify Calibration in CyberGlove-Based Virtual Rehabilitation

We have previously developed a prototype virtual-reality-enhanced rehabilitation system using the CyberForce system to assist patients who have suffered from upper extremity stroke to practice some daily life exercises. However, full calibration of the system for each patient is currently not only tedious and time consuming but also impractical in the case of severely disabled hands. In this paper, we propose a practical and easy-to-perform hand-measurement method to calibrate the CyberGlove using artificial neural networks (NNs). The NNs are trained with the hand-segment sizes as input and the manually collected calibrated data as output. The only external device needed is a 2-D digital camera to take the picture of the subjects hand against a chessboard for the hand-segment-size measurement. Subjective evaluation results for various common hand postures show the effectiveness of the proposed method.

Haptic-Enabled Telementoring Surgery Simulation

Medical surgery involves a high degree of skill and experience, making the learning curve for medical trainees quite long. For instance, in eye cataract surgery, despite it only taking around seven minutes for a well-trained surgeon to perform and having a success rate of 99 percent, medical residents need months to become proficient in this procedure to avoid its typical complications. Medical trainees traditionally have acquired surgical skills through apprenticeships in which trainees observe senior surgeons, then perform under guidance until they achieve mastery. Training often makes use of cadavers or laboratory animals, but this type of training is becoming increasingly difficult to do in many countries due to ethical reasons. An effective alternative is medical simulation, which can enhance understanding, improve performance, and assess competence; in preoperative settings, it assists surgeons in remaining at a high technical skill level. Surgical simulation can provide high-fidelity training that increases the diffusion of innovative and less- invasive procedures while decreasing the surgeons learning curve.

Immersive Haptic Eye Tele-Surgery Training Simulation

As medical surgery is usually a complex and complicated procedure, medical trainees must acquire specific skills and plenty of experience before attempting to perform such a procedure in a real-world setting. The goal of surgical simulation is to provide high-fidelity training to increase the diffusion of innovative and less-invasive procedures while decreasing the surgeons learning curve. The paper presents an immersive hapto-virtual training simulation for eye cataract surgery. The system architecture and technology and design techniques are addressed. Finally the paper concludes with future work to improve the application.

A haptic virtual environment for industrial training

This paper presents a haptic virtual environment application for industrial training. A choice of haptic devices. (PHANTOM and CyberForce/CyberGrasp) is used to perform a training exercise with the possibility of sharing virtual scene states with remote users to allow collaboration and remote observation. Furthermore, the application employs video textured avatars to improve face-to-face communication, a novel area of interest management mechanism to reduce network bandwidth usage, standard encoding of state updates for compatibility and is used in conjunction with various projection technologies such as a head mounted display, CAVE configuration or regular PC monitor. The paper serves to underline the various components needed for the development of a haptic virtual environment and our experiences in doing so.

A distributed, collaborative and haptic-enabled eye cataract surgery application with a user interface on desktop, stereo desktop and immersive displays

In this paper, we discuss the technologies and approaches utilized in developing a cataract eye surgery simulation that will be used for training novice surgeons. The three different techniques described in this paper, are all hapto-visual techniques that resulted in three different implementations of the application: 2D simulation, 3D immersive simulation, and completely immersive simulation. The paper begins with an introduction and a general overview of the medical procedure of the cataract surgery. Then an overview of the eye and surgical tools modeling is given. Following that, the architecture and technology of each of the three design techniques is given. Finally the paper concludes with future work to improve the application

Artificial neural networks for real-time optical hand posture recognition using a color-coded glove

Optical pose recognition of the hand is an extremely attractive method for user-computer interaction in many applications. The image of a hand in the frame of a video camera is processed and the pose it is making, its current finger configuration, is detected. Often combined with position tracking, it allows for a very natural way of giving commands. Furthermore, it alleviates the use of sometimes cumbersome pieces of hardware. Within immersive virtual reality systems, the liberty of movement of the commanding hand requires extra considerations not normally dealt with by typical optical hand posture recognition interfaces for desktop system applications. This research proposes an artificial neural network approach to the recognition of hand postures. The optical capture inside an immersive virtual reality workspace and the extraction of features of this hand are facilitated by the use of a specially coded color glove.

Uniform Hardness Perception in 6-DOF Haptic Rendering

In this paper, we study the influence of a haptic interfaces effective mass and viscous damping on the hardness perception of simulated virtual objects in six-degree-of-freedom (6-DOF) haptic rendering. It is found that induced forces from these physical parameters affect the perceived hardness nonuniformly as users tap virtual objects with different parts of the tool. Fundamentally, the distinct hardness perception is caused by the discrepancy between the users holding point and the active end of the device. A stick-on-ball application was used for comparisons of three proposed compensation methods. A total of six stiffness intensities in the range of 0.1-0.6 N/m at a step of 0.1 were randomly presented to a total of ten test subjects. Experimental results show both the need for compensation and the effectiveness of the proposed methods.

Compensating device inertia for 6-DOF haptic rendering

In this paper, the importance of the users primary holding pivot point on the end effector of a haptic interface is discussed. Both theoretical analysis and experimental results demonstrate that this holding pivot point is critical for the correct perception of the haptic properties assigned to the virtual objects. We also study the physical inertia effects of the end effector on the non-uniform stiffness perception of simulated virtual objects. To the best of our knowledge, no such combined consideration of holding pivot point and device structure related inertia has so far been made for works in 6-DOF haptic rendering. We have instrumented the end effector of a haptic interface with a membrane potentiometer to measure the users primary holding pivot point in real-time. Accordingly, a preliminary adaptive feedback method is developed to render the appropriate forces/torques to compensate for the effects of the end effectors inertia on the haptic stiffness perception.

Practical calibation for upper extremety patients in haptic rehabilitation

As haptic devices become more affordable and economical, research in Haptic-based virtual rehabilitation systems is gaining more interest. A prototype haptic virtual rehabilitation system was developed in our lab using the CyberGrasp system with the goal of assisting patients who suffered from stroke leading to temporary upper extremity disability. However, calibration of the system for each user is currently tedious and time-consuming. In this paper, we propose a practical method to calibrate the CyberGlove using a Neural Network. The neural network is trained with the hand segment size and the manually collected calibrated data of 12 subjects. The hand segment sizes are extracted from a 2-D hand image. The preliminary results are very encouraging and show its effectiveness for the sensors on the index, the middle, the ring, the pinky fingers.

A 3D Annotation Interface Using the DIVINE Visual Display

While systems such as CAVEs have been experimented with and used for a number of years, their deployment has been slow mainly due to their expense and space requirements. As such, researchers have been moving towards smaller and cheaper immersive systems. In this paper, we introduce an immersive interface for manipulating 3D objects using the DIVINE system