David Navarre

A formal description of multimodal interaction techniques for immersive virtual reality applications

Nowadays, designers of Virtual Reality (VR) applications are faced with the choice of a large number of different input and output devices leading to a growing number of interaction techniques. Usually VR interaction techniques are described informally, based on the actions users can perform within the VR environment. At implementation time, such informal descriptions (made at design time) yield to ambiguous interpretations by the developers. In addition, informal descriptions make it difficult to foresee the impact throughout the application of a modification of the interaction techniques. This paper discusses the advantages of using a formal description technique (called ICO) to model interaction techniques and dialogues for VR applications. This notation is presented via a case study featuring an immersive VR application. The case study is then used to show, through analysis of models, how the formal notation can help to ensure the usability, reliability and efficiency of virtual reality systems.

Model-based engineering of widgets, user applications and servers compliant with ARINC 661 specification

The purpose of ARINC 661 specification [1] is to define interfaces to a Cockpit Display System (CDS) used in any types of aircraft installations. ARINC 661 provides precise information for communication protocol between application (called User Applications) and user interface components (called widgets) as well as precise information about the widgets themselves. However, in ARINC 661, no information is given about the behaviour of these widgets and about the behaviour of an application made up of a set of such widgets. This paper presents the results of the application of a formal description technique to the various elements of ARINC 661 specification within an industrial project. This formal description technique called Interactive Cooperative Objects defines in a precise and non-ambiguous way all the elements of ARINC 661 specification. The application of the formal description techniques is shown on an interactive application called MPIA (Multi Purpose Interactive Application). Within this application, we present how ICO are used for describing interactive widgets, User Applications and User Interface servers (in charge of interaction techniques). The emphasis is put on the model-based management of the feel of the applications allowing rapid prototyping of the external presentation and the interaction techniques. Lastly, we present the CASE (Computer Aided Software Engineering) tool supporting the formal description technique and its new extensions in order to deal with large scale applications as the ones targeted at by ARINC 661 specification.

High-Fidelity Prototyping of Interactive Systems Can Be Formal Too

The design of safety critical systems calls for advanced software engineering models, methods and tools in order to meet the safety requirements that will avoid putting human life at stake. When the safety critical system encompasses a substantial interactive component, the same level of confidence is required towards the human-computer interface. Conventional empirical or semi-formal techniques, although very fruitful, do not provide sufficient insight on the reliability of the human-system cooperation, and offer no easy way to, for example, quantitatively compare two design options. The aim of this paper is to present a method, with supporting tools and techniques, for engineering the design and development of usable user interfaces for safety-critical applications. More precisely we present the Petshop environment which is a Petri net based tool for the design specification, prototyping and validation of interactive software. In this environment models of the interactive application can be interactively modified and executed. This is used to support prototyping phases (when the models and the interactive application evolve significantly to meet late user requirements for instance) as well as in the operation phase (after the system is deployed). The use of the description technique (the ICO formalism) supported by PetShop is presented on a multimodal ground segment application for satellite control and more precisely how prototyping can be performed at the various levels of the architecture of interactive systems.

An approach integrating two complementary model-based environments for the construction of multimodal interactive applications

This paper presents a tool suite for the engineering of multimodal Post-WIMP Interactive Systems. The work presented here extends previous work done on design, prototyping, specification and verification of interactive systems and integrates two previously unrelated approaches. The first element of this integration is ICoM (a data-flow model dedicated to low-level input modelling) and its environment ICon which allows for editing and simulating ICoM models. The other element is ICOs (a formal description technique mainly dedicated to dialogue modelling) and its environment PetShop, which allows for editing, simulating and verifying ICOs models. This paper shows how these two approaches have been integrated and that this integration allows for engineering multimodal interactive systems. We show on a Range Slider case study how these tools can be used for prototyping interactive systems in general and multimodal interaction techniques in particular. We also present in details how the changes in the interaction techniques impact the models at various levels of the software architecture.

Formal description techniques to support the design, construction and evaluation of fusion engines for sure (safe, usable, reliable and evolvable) multimodal interfaces

Representing the behaviour of multimodal interactive systems in a complete, concise and non-ambiguous way is still a challenge for formal description techniques (FDT). Depending on the FDT, multimodal interactive systems feature specific characteristics that are either cumbersome or impossible to capture with classical FDT. This is due to the multiple (potentially synergistic) use of modalities and the strong temporal constraints usually encountered in this kind of systems that have to be dealt with exhaustively if FDT are used. This paper focuses on the requirements for the modelling and construction of fusion engines for multimodal interfaces. It proposes a formal description technique dedicated to the engineering of interactive multimodal systems able to address the challenges of fusion engines. Such benefits are presented on a set of examples illustrating both the constructs and the process.

Improving Modularity of Interactive Software with the MDPC Architecture

The Model - Display view - Picking view - Controller model is a refinement of the MVC architecture. It introduces the Picking View component, which offloads the need from the controller to analytically compute the picked element. We describe how using the MPDC architecture leads to benefits in terms of modularity and descriptive ability when implementing interactive components. We report on the use of the MDPC architecture in a real application: we effectively measured gains in controller code, which is simpler and more focused.

Improving interactive systems usability using formal description techniques: application to healthcare

In this paper we argue that the formal analysis of an interactive medical system can improve their usability evaluation such that potential erroneous interactions are identified and improvements can be recommended. Typically usability evaluations are carried out on the interface part of a system by human-computer interaction/ergonomic experts with or without end users. Here we suggest that formal specification of the behavior of the system supported by mathematical analysis and reasoning techniques can improve usability evaluations by proving usability properties. We present our approach highlighting that formal description techniques can support in a consistent way usability evaluation, contextual help and incident and accident analysis. This approach is presented on a wireless patient monitoring system for which adverse event (including fatalities) reports are publicly available from the US Food and Drug Administration (FDA) Manufacturer and User Facility Device Experience (MAUDE) database.

Task Models and System Models as A Bridge Between Hci and Software Engineering

This chapter claims that task models per se do not contain sufficient and necessary information to permit automatic generation of interactive systems. Beyond this, we claim that they must not contain sufficient and necessary information otherwise they could no longer be considered as task models. On the contrary we propose a way of exploiting in a synergistic way task models with other models to be built during the development process. This chapter presents a set of tools supporting the development of interactive systems using two different notations. One of these notations called ConcurTaskTree (CTT) is used for task modeling. The other notation called Interactive Cooperative Objects (ICO) is used for system modeling. Even though these two kinds of models represent two different views of the same world (a user interacting with an interactive system), they are built by different people (human factors specialist for the task models and software engineer for the system models) and are used independently. The aim of this chapter is to propose the use of scenarios as a bridge between these two views. On the task modeling side, scenarios are seen as a possible trace of user’s activity. On the system side, scenarios are seen as a trace of user’s actions. This generic approach is presented on a case study in the domain of Air Traffic Control. As both CTT and ICO notations are tool supported (environments are respectively CTTE and PetShop) an integration tool based on this notion of scenarios is presented. Its use on the selected case study is also presented in detail.

Very-high-fidelity prototyping for both presentation and dialogue parts of multimodal interactive systems

This paper presents a tool suite (made up of two previously unrelated approaches) for the engineering of multimodal Post-WIMP Interactive Systems. The first element of this integration is ICOM (a data-flow model dedicated to low-level input modelling) and its environment ICON which allows for editing and simulating ICOM models. The other element is ICOs (a formal description technique mainly dedicated to dialogue modelling) and its environment PetShop which allows for editing, simulating and verifying ICOs models. This paper shows how these two approaches have been integrated and how they support multimodal interactive systems engineering. We show on a classical rubber banding case study how these tools can be used for prototyping interactive systems. We also present in details how the changes in the interaction techniques impact the models at various levels of the software architecture.

Une approche à base de modèles pour l'ingénierie logicielle de techniques d'interaction

While model-based approaches have been used for over 30 years in the field of behavioral description of interactive systems [27], the link between these approaches and user-centered design process remain insufficiently explained. This paper offers a contribution to this problem by presenting how a model-based approach can be exploited to facilitate the tasks of evaluation of usability that are often laborious and repetitive. The basic principle of this approach promotes the use of recording and analysis of log data in a model-based environment. The results described in this paper show that the log data at model level can be used not only to identify usability problems but also to identify changes to these models in order to correct the encountered problems. This approach is integrated in a process and is supported by a modelbased CASE tool for modeling, simulating and evaluating interactive systems. The case study illustrates the principles of the approach and operation of the tool on an interaction technique. It shows how the analysis of log data allows the designer to easily tune the interaction technique (as the results of the analysis of log data are presented at the same abstraction level than models). It proposes an alternative to user tests that are very difficult to configure and to interpret especially when advanced interaction techniques are concerned.