Patrick Bourdot

Advances in Human-Protein Interaction - Interactive and Immersive Molecular Simulations

Molecular simulations allow researchers to obtain complementary data with respect to experimental studies and to overcome some of their limitations. Current experimental techniques do not allow to observe the full dynamics of a protein at atomic detail. In return, experiments provide the structures, i.e. the spatial atomic positions, for numerous biomolecular systems, which are often used as starting point for simulation studies. In order to predict, to explain and to understand experimental results, researchers have developed a variety of biomolecular representations and algorithms. They allow to simulate the dynamic behavior of macromolecules at different scales, ranging from detailed models using quantum mechanics or classical molecular mechanics to more approximate representations. These simulations are often controlled a priori by complex and empirical settings. Most researchers visualise the result of their simulation once the computation is finished. Such post-simulation analysis often makes use of specific molecular user interfaces, by reading and visualising the molecular 3D configuration at each step of the simulation. This approach makes it difficult to interact with a simulation in progress. When a problem occurs, or when the researcher does not achieve to observe the predicted behavior, the simulation must be restarted with other settings or constraints. This can result in the waste of an important number of compute cycles, as some simulations last for a long time: several days to weeks may be required to reproduce a short timespan, a few nanoseconds, of molecular reality. Moreover, several biomolecular processes, like folding or large conformational changes of proteins, occur on even longer timescales that are inaccessible to current simulation techniques. It can thus be necessary to impose empirical constraints in order to accelerate a simulation and to reproduce

Multisensory VR interaction for protein-docking in the CoRSAIRe project

Proteins take on their function in the cell by interacting with other proteins or biomolecular complexes. To study this process, computational methods, collectively named protein docking, are used to predict the position and orientation of a protein ligand when it is bound to a protein receptor or enzyme, taking into account chemical or physical criteria. This process is intensively studied to discover new biological functions for proteins and to better understand how these macromolecules take on these functions at the molecular scale. Pharmaceutical research also employs docking techniques for a variety of purposes, most notably in the virtual screening of large databases of available chemicals to select likely molecular candidates for drug design. The basic hypothesis of our work is that Virtual Reality (VR) and multimodal interaction can increase efficiency in reaching and analysing docking solutions, in addition to fully a computational docking approach. To this end, we conducted an ergonomic analysis of the protein–protein current docking task as it is carried out today. Using these results, we designed an immersive and multimodal application where VR devices, such as the three-dimensional mouse and haptic devices, are used to interactively manipulate two proteins to explore possible docking solutions. During this exploration, visual, audio, and haptic feedbacks are combined to render and evaluate chemical or physical properties of the current docking configuration.

Audio haptic feedbacks for an acquisition task in a multi-target context

This paper presents the use of audio and haptic feedbacks to reduce the load of the visual channel in interaction tasks within virtual environments. An examination is made regarding the exploitation of audio and/or haptic cues for the acquisition of a desired target in an environment containing multiple and obscured distractors. This study compares different ways of identifying and locating a specified target among others by the mean of either audio, haptic, or both feedbacks rendered simultaneously. The analysis of results and subjective user comments indicate that active haptic and combined audio/haptic conditions offer better results when compared to the audio only condition. Moreover, that the association of haptic and audio feedback presents a real potential for the completion of the task.

A framework to manage multimodal fusion of events for advanced interactions within virtual environments

This paper describes the EVI3d framework, a distributed architecture developed to enhance interactions within Virtual Environments (VE). This framework manages many multi-sensorial devices such as trackers, data gloves, and speech or gesture recognition systems as well as haptic devices. The structure of this architecture allows a complete dispatching of device services and their clients on as many machines as required. With the dated events provided by its time synchronization system, it becomes possible to design a specific module to manage multimodal fusion processes. To this end, we describe how the EVI3d framework manages not only low-level events but also abstract modalities. Moreover, the data flow service of the EVI3d framework solves the problem of sharing the virtual scene between modality modules.

Adaptive tessellation of connected primitives for interactive walkthroughs in complex industrial virtual environments

Geometrical primitives used in virtual environments are converted to an important amount triangles at rendering time. The meshes of the resulting simplifications usually introduce discontinuities between neighboring object. We extend a simple adaptive tessellation method which adapts the amount of triangles to the viewing conditions with connection information to ensure that the meshes of connected primitives remain continuous. An ergonomical study has validated this approach for applications using virtual environments.

Non-visual identification, localization, and selection of entities of interest in a 3D environment

This paper addresses the use of audio and haptics as a mean to reduce the load of the visual channel in interaction tasks within virtual environments. An examination is made regarding the exploitation of audio and/or haptic interactions for the acquisition of a target of interest in an environment containing multiple and obscured distractors. A first study compares means for identifying and locating a specified target among others employing either audio, haptic, or both sensori-motor channels activated simultaneously. Following an analysis of the results and subject comments, an improved multimodal approach is proposed and evaluated in a second study, combining advantages offered by each sensory channel. Results confirm the efficiency and effectiveness of the proposed multimodal approach.

Polyvalent display framework to control virtual navigations by 6DOF tracking

Many applications using Virtual Reality (VR) require free hand navigation control. This forbids the use of devices such as Joystick, 3D mouse, ... That is generally because hand-based gesture interaction must be dedicated to specific tasks: sculpting or grasping the objects, which compose a virtual scene. With this aim, we present a polyvalent display approach, which allows the system to determine the desired focus and the moving intention of the user for virtual navigations. After we point out the limits of the classical solutions used to control virtual navigations, we describe the conceptual and mathematical implementation of our approach. Some of our considerations depend on physiological and ergonomic constraints. We conclude on the first applications of our fully head tracking navigation controller.

Levels of detail & polygonal simplification

This paper covers the techniques of Polygonal Simplification in order to produce Levels of Detail (LODs). The problem of creating LODs is a complex one: how can simpler versions of a model be created? How can the approximation error be measured? How can the visual degradation be estimated? Can all this be done automatically? After exposing the basic aims and principles of polygonal simplification, we compare recent algorithms and state their various qualities and weaknesses.

Haptical exploration of an unsteady flow

This paper investigates a human-centered approach for the exploration of large data sets resulting from CFD (Computational Fluid Dynamics) simulations. In a VR (Virtual Reality) immersive environment we propose to couple haptic feedbacks with visual rendering. The proposed solution aims at identifying critical points of an unsteady fluid flow without resorting to any topology analysis such as eigen analysis of the vector field Jacobian. In a first step, while exploring the data volume, a vibration alerts the user on the presence of critical points in the local volume surrounding the probe position. In a second step, critical point properties are emphasized by means of a visuo-haptic feedback. A set of experiments confirms the effectiveness of the proposed approach. Thanks to the combination of haptic feedbacks with the visual materialization of critical points and streamlines, users were able to easily localize, analyze and focus on specific critical points, selected by experts as representative of the flow topology.

6DoF navigation in virtual worlds: comparison of joystick-based and head-controlled paradigms

6DoF navigation in a virtual world can usually be implemented by two types of navigation techniques: joystick-based input devices and steering metaphors based on movements of the users body, e.g. head-controlled paradigms. These two different types of 6DoF navigation techniques provide users with the same level of control, but the latter introduces the users physical movements in the navigation, which we believe will improve the navigation experience in immersive virtual environments. In this paper, we compare these two types of 6DoF navigation techniques in an immersive context, through an experiment using both objective and subjective measurements to assess user performance, the occurrence of cybersickness symptoms and the level of presence, when using either of these navigation paradigms.