Mehdi Ammi

Multisensory VR interaction for protein-docking in the CoRSAIRe project

Proteins take on their function in the cell by interacting with other proteins or biomolecular complexes. To study this process, computational methods, collectively named protein docking, are used to predict the position and orientation of a protein ligand when it is bound to a protein receptor or enzyme, taking into account chemical or physical criteria. This process is intensively studied to discover new biological functions for proteins and to better understand how these macromolecules take on these functions at the molecular scale. Pharmaceutical research also employs docking techniques for a variety of purposes, most notably in the virtual screening of large databases of available chemicals to select likely molecular candidates for drug design. The basic hypothesis of our work is that Virtual Reality (VR) and multimodal interaction can increase efficiency in reaching and analysing docking solutions, in addition to fully a computational docking approach. To this end, we conducted an ergonomic analysis of the protein–protein current docking task as it is carried out today. Using these results, we designed an immersive and multimodal application where VR devices, such as the three-dimensional mouse and haptic devices, are used to interactively manipulate two proteins to explore possible docking solutions. During this exploration, visual, audio, and haptic feedbacks are combined to render and evaluate chemical or physical properties of the current docking configuration.

Evaluation of 3D Pseudo-Haptic Rendering using Vision for Cell Micromanipulation

This paper presents a new three-dimensional biomicromanipulation system for biological objects such as embryos, cells or oocytes. As the cell is very small, kept in the liquid, and observed through a microscope, the two-dimensional visual feedback makes difficult accurate manipulation in the 3-D world. To improve the manipulation work, we proposed an augmented human-machine interface. A 3-D visual information is provided to the operator through a 3-D reconstruction method using vision-based tracking deformations of the cell embryo. In order to stable injection tasks, the operator needs force feedback and haptic assistance during penetrating the cell envelop, the chorion. The proposed human-machine users interface allows real-time realistic visual and haptic control strategies for constrained motion in image coordinates. Virtual haptic rendering allows to constrain the path of insertion and removal in the 3-D scene or to avoid cell destruction by controlling adequately position, velocity and force parameters

Robotic Assisted Micromanipulation System using Virtual Fixtures and Metaphors

This paper describes the use of virtual fixtures and metaphors of assistance for robotic-assisted micromanipulation system in order to prevent the influence of microphysics on path planning and handling tasks. The system is based on a multimodal telemanipulation system using haptic/visual/sound interfaces for observation of microobjects under an optical microscope. Feasible haptically-generated paths based on potentials fields reaction forces and shock absorbers are described for efficient and safe pushing-based or adhesion-based micromanipulation. Then, metaphors with human sensory substitution are proposed in order to improve the perception of data or events. Finally, an experimental investigation carried out by nine trainees proves that the system guides efficiently and safely the operators gesture. Moreover, user performance on a given task can increase as much as 52% in typical micromanipulation tasks.

Automatic Camera-Based Microscope Calibration for a Telemicromanipulation System Using a Virtual Pattern

In the context of virtualized-reality-based telemicromanipulation, this paper presents a visual calibration technique for an optical microscope coupled to a charge-coupled device (CCD) camera. The accuracy and flexibility of the proposed automatic virtual calibration method, based on parallel single-plane properties, are outlined. In contrast to standard approaches, a 3-D virtual calibration pattern is constructed using the micromanipulator tip with subpixel-order localization in the image frame. The proposed procedure leads to a linear system whose solution provides directly both the intrinsic and extrinsic parameters of the geometrical model. Computer simulations and real data have been used to test the proposed technique, and promising results have been obtained. Based on the proposed calibration techniques, a 3-D virtual microenvironment of the workspace is reconstructed through the real-time imaging of two perpendicular optical microscopes. Our method provides a flexible, easy-to-use technical alternative to the classical techniques used in micromanipulation systems.

Biological cell injection visual and haptic interface

This paper presents a new three-dimensional (3-D) biomicromanipulation system for biological objects such as embryos, cells or oocytes. As the cell is very small, kept in liquid and observed through a microscope, 2-D visual feedback makes accurate manipulation in the 3-D world difficult. To improve the manipulation work, we proposed an intelligent human–machine interface. The 3-D visual information is provided to the operator through a 3-D reconstruction method using vision-based tracking deformations of the cell embryo. In order to perform stable microinjection tasks, the operator needs force feedback and haptic assistance during penetration of the cell envelop — the chorion. Thus, realistic haptic rendering techniques have been implemented to validate stable insertion of a micropipette in a living cell. The proposed human–machine users interface allows real-time realistic visual and haptic control strategies for constrained motion in image coordinates, virtual haptic rendering to constrain the path of insertion and removal in the 3-D scene or to avoid cell destruction by adequately controlling position, velocity and force parameters. Experiments showed that the virtualized reality interface acts as a tool for total guidance and assistance during microinjection tasks.

Psychophysical study of air jet based tactile stimulation

Haptic rendering technologies are becoming a strategic component of the new Human-Machines Interfaces. Many existing devices generally operate with intrusive mechanical structures that limit rendering and transparency of the haptic interaction. In the context of the design of a novel non-contact haptic device that exploit the air jet tactile stimulation with long distance, we carried out psychophysical experiments in order to characterize the human perception of the air jet tactile stimulation. This study focuses on the stimulation of the users hand palm. Fluid mechanics suggests that perceived parameters of the air jet stimulation can be controlled by the air jet flow rate, the nozzle geometry, and the distance between the nozzle and the hand palm. In this paper we investigate how these control variables affect the perceived force on the hand palm. Two psychophysical studies were carried out. The first experiment investigated three distances from the air nozzle to determine the absolute threshold of the air flow rate on the hand palm. Results reveal that there is a linear relationship between the perceived absolute threshold and the distance from the hand palm to the nozzle. The second psychophysical experiment was made to determinate the just noticeable difference of the air flow rate according to three referential stimuli. Then we estimated the Weber fraction. These two results are important for the display control and will be used further for the design of the future noncontact haptic device.

Haptical exploration of an unsteady flow

This paper investigates a human-centered approach for the exploration of large data sets resulting from CFD (Computational Fluid Dynamics) simulations. In a VR (Virtual Reality) immersive environment we propose to couple haptic feedbacks with visual rendering. The proposed solution aims at identifying critical points of an unsteady fluid flow without resorting to any topology analysis such as eigen analysis of the vector field Jacobian. In a first step, while exploring the data volume, a vibration alerts the user on the presence of critical points in the local volume surrounding the probe position. In a second step, critical point properties are emphasized by means of a visuo-haptic feedback. A set of experiments confirms the effectiveness of the proposed approach. Thanks to the combination of haptic feedbacks with the visual materialization of critical points and streamlines, users were able to easily localize, analyze and focus on specific critical points, selected by experts as representative of the flow topology.

Virtualized reality interface for tele-micromanipulation

Operators suffer much difficulty in manipulating micro/nano-sized objects without the assistance of human-machine interfaces due to scaling effects. We developed an immersive telemanipulation system using haptic/visual/sound interfaces for observation of micro-objects under an optical microscope. As the image of the microscope is two-dimensional, so it is hard to observe the workspace in the 3-D space. To improve the real-time observation and manipulation, we proposed real-time 3-D reconstruction of the microworld using image processing and virtualized reality techniques. Then, feasible haptically-generated paths based on potentials fields reaction forces are selected for efficient pushing-based manipulation without collisions. The proposed system guides the operators gesture fully immerged in the virtual workspace.

Improvement of the recognition of facial expressions with haptic feedback

Affective computing is among emerging fields for the design of new Human-Computer Interfaces. The facial expression was considered as the main way to communicate emotions. However, this approach presents some limitations for the recognition of complex emotions. Recent researches highlight new approaches to communicate effectively some emotions. The use of haptic feedback seems to be a promising solution to improve the level of recognition. In fact, haptic feedback, introduces different physical dimensions which complete the visual features of emotions. This paper proposes the study of the complementary between facial expressions and haptic feedback to recognize different class of emotions. Results show that the proposed haptic feedback improves at different levels the recognition of some emotions.

Haptically generated paths of an AFM-based nanomanipulator using potential fields

This paper proposes a practical planning system for 2D assembly tasks at the micro/nano scale. To improve the real-time observation and manipulation, we proposed real-time 3-D reconstruction of the microworld using image processing and virtualized reality techniques. Then, a planner covers a whole range of problems in object assignment, obstacle detection and avoidance and path trajectory. We describe algorithms based on optimization theory and Voronoi graph construction. As automatic motion planners fail due to the difficulty of discovering critical configurations due to microdomain interactions, cooperation with the operator skills is able to solve a motion planning query. The feasible haptically-generated paths based on potentials fields reaction forces are selected for efficient pushing-based nanomanipulation without collisions.