Cédric Dumas

The MaggLite post-WIMP toolkit: draw it, connect it and run it

This article presents MaggLite, a toolkit and sketch-based interface builder allowing fast and interactive design of post-WIMP user interfaces. MaggLite improves design of advanced UIs thanks to its novel <i>mixed-graph</i> architecture that dynamically combines scene-graphs with interaction-graphs. <i>Scene-graphs</i> provide mechanisms to describe and produce rich graphical effects, whereas <i>interaction-graphs</i> allow expressive and fine-grained description of advanced interaction techniques and behaviors such as multiple pointers management, toolglasses, bimanual interaction, gesture, and speech recognition. Both graphs can be built interactively by sketching the UI and specifying the interaction using a dataflow visual language. Communication between the two graphs is managed at runtime by components we call <i>Interaction Access Points</i>. While developers can extend the toolkit by refining built-in generic mechanisms, UI designers can quickly and interactively design, prototype and test advanced user interfaces by applying the MaggLite principle: draw it, connect it and run it.

Influences of haptic communication on a shared manual task

Abstract With the advent of new haptic feedback devices, researchers are giving serious consideration to the incorporation of haptic communication in collaborative virtual environments. For instance, haptic interactions based tools can be used for medical and related education whereby students can train in minimal invasive surgery using virtual reality before approaching human subjects. To design virtual environments that support haptic communication, a deeper understanding of humans′ haptic interactions is required. In this paper, human′s haptic collaboration is investigated. A collaborative virtual environment was designed to support performing a shared manual task. To evaluate this system, 60 medical students participated to an experimental study. Participants were asked to perform in dyads a needle insertion task after a training period. Results show that compared to conventional training methods, a visual-haptic training improves user′s collaborative performance. In addition, we found that haptic interaction influences the partners′ verbal communication when sharing haptic information. This indicates that the haptic communication training changes the nature of the users′ mental representations. Finally, we found that haptic interactions increased the sense of copresence in the virtual environment: haptic communication facilitates users′ collaboration in a shared manual task within a shared virtual environment. Design implications for including haptic communication in virtual environments are outlined.

Haptic Communication to Support Biopsy Procedures Learning in Virtual Environments

In interventional radiology, physicians require high haptic sensitivity and fine motor skills development because of the limited real-time visual feedback of the surgical site. The transfer of this type of surgical skill to novices is a challenging issue. This paper presents a study on the design of a biopsy procedure learning system. Our methodology, based on a task-centered design approach, aims to bring out new design rules for virtual learning environments. A new collaborative haptic training paradigm is introduced to support human-haptic interaction in a virtual environment. The interaction paradigm supports haptic communication between two distant users to teach a surgical skill. In order to evaluate this paradigm, a user experiment was conducted. Sixty volunteer medical students participated in the study to assess the influence of the teaching method on their performance in a biopsy procedure task. The results show that to transfer the skills, the combination of haptic communication with verbal and visual communications improves the novices’ performance compared to conventional teaching methods. Furthermore, the results show that, depending on the teaching method, participants developed different needle insertion profiles. We conclude that our interaction paradigm facilitates expert-novice haptic communication and improves skills transfer; and new skills acquisition depends on the availability of different communication channels between experts and novices. Our findings indicate that the traditional fellowship methods in surgery should evolve to an off-patient collaborative environment that will continue to support visual and verbal communication, but also haptic communication, in order to achieve a better and more complete skills training.

Influence of contextual objects on spatial interactions and viewpoints sharing in virtual environments

Collaborative virtual environments (CVEs) are 3D spaces in which users share virtual objects, communicate, and work together. To collaborate efficiently, users must develop a common representation of their shared virtual space. In this work, we investigated spatial communication in virtual environments. In order to perform an object co-manipulation task, the users must be able to communicate and exchange spatial information, such as object position, in a virtual environment. We conducted an experiment in which we manipulated the contents of the shared virtual space to understand how users verbally construct a common spatial representation of their environment. Forty-four students participated in the experiment to assess the influence of contextual objects on spatial communication and sharing of viewpoints. The participants were asked to perform in dyads an object co-manipulation task. The results show that the presence of a contextual object such as fixed and lateralized visual landmarks in the virtual environment positively influences the way male operators collaborate to perform this task. These results allow us to provide some design recommendations for CVEs for object manipulation tasks.

Development of a novel image-based program to teach narrow-band imaging

Objectives: Narrow-band imaging (NBI) is a widely available endoscopic imaging technology; however, uptake of the technique could be improved. Teaching new imaging techniques and assessing trainees’ performance can be a challenging exercise during a 1-day workshop. To support NBI training, we developed an online training tool (Medimq) to help experts train novices in NBI bronchoscopy that could assess trainees’ performance and provide feedback before the close of the 1-day course. The present study determines whether trainees’ capacity to identify relevant pathology increases with the proposed interactive testing method. Methods: Two groups of 20 and 18 bronchoscopists have attended an NBI course where they did a pretest and post-test before and after the main lecture, and a follow-up test 4 weeks later to measure retention of knowledge. We measured their ability to mark normal and abnormal ‘biopsy size’ areas on bronchoscopic NBI images for biopsy. These markings were compared with areas marked by experts on the same images. Results: The first group results were used to pilot the test. After modifications, the results of the improved test for group 2 showed trainees improved by 32% (total class average normalized gain) in detecting normal or abnormal areas. On follow-up testing, Group 2 improved by 23%. Conclusions: The overall class average normalized gain of 32% shows our test can be used to improve trainees’ competency in analyzing NBI Images. The testing method (and tool) can be used to measure the follow up 4 weeks later. Better follow-up test results would be expected with more frequent practice by trainees after the course.

Flexibilité et modularité pour la conception d'interactions: le modèle d'architecture logicielle des graphes combinés

This paper presents the Mixed Graphs software architecture of the MAGGLITE post-WIMP toolkit. This model extends and refines the scene graph architecture by describing interactions with another structure: interaction graphs. Graphs are dynamically combined at runtime, thanks to specific communication components named Interaction Access Points. This adaptable and modular approach for describing interactive graphics is gainful for prototyping, implementing and using alternative input devices and/or interaction techniques.

Svalabard: une table à dessin virtuelle pour la modélisation 3D

This paper introduces a perspective sketching interface for creative 3D modeling: Svalabard. To close the gap between designers abilities and 3D modeling tools, our proposal is a freehand sketching tool, metaphor of a drawing desk. Interactions and tools take advantage of designers work habits (multilayer drawing, association of interactions with input devices). Svalabard includes background drawing filters, developed from a user study in the architectural domain. Those filters detect the current drawing context in real-time to adapt the interface and clean the sketch. Finally, the result of the filters is a structured representation of drawn 3D volumes.

SyTroN: a virtual classroom for collaborative and distant e-learning systemby teleoperating real devices

Distant E-learning is a main issue nowadays, and it is strongly motivated by social and economical considerations. The increased people mobility and the reduction of educational costs push to develop ad hoc solutions enabling to access to knowledge regardless to geographical situation and economical capabilities. These parameters should not be limits for good training: learning materials pertinence and efficiency have to remain the core of educational activities. In this paper we address the problem through SyTroN: a tele-learning system. This system combines virtual reality and teleoperation techniques to offer an open platform with two main objectives. The first one is to propose intuitive virtual classrooms/desks, including a real teacher supervision and supporting collaborative and individual distant learning. The second goal is to place learners in real conditions with remote connections to real devices allowing distant experimentations. Both goals participate to increase learning impacts and to reduce costs, that is, sharing costly real devices from anywhere at any time. After 5 years of development, our work has been validated by an extensive use at a high engineering school. In situ tests and learning impact studies have been done. They show some advantages and some drawbacks of our global solution.

Vers des modeleurs 3D créatifs: étude de dessins d'architectes

This paper describes an experiment we conduct to provide designers with an advanced 3D modelling Human-Computer Interaction. The main project - called GINA [4], a French acronym for Interactive and Natural Geometry-expect to introduce new intuitive and creative 3D modeling tools, relying on 2D perspective drawing reconstruction. This domain study will help us to deduce implications for Human-Computer Interaction in such a system.

Haptic feedback tuning in colonoscopy simulation

According to medical experts, haptic realism is difficult to achieve, and even more difficult to have inter-expert agreement on the haptic feedback of one simulation. However haptic feedback is important in medical training, and allows educators to share the forces felt during a procedure if they know and trust what a particular virtual simulator will provide to the trainee. A new approach is proposed to refine bio-mechanical models with experts input, to closely match the forces felt during a simulated procedure with an expert trainers expectations. By allowing experts to tune a training scenarios haptic feedback as they trial a newly developed case, the experts can replicate their haptic perception and match their expectations with the simulation.