Jean-François Ladry

ICOs: A model-based user interface description technique dedicated to interactive systems addressing usability, reliability and scalability

The design of real-life complex systems calls for advanced software engineering models, methods, and tools in order to meet critical requirements such as reliability, dependability, safety, or resilience that will avoid putting the company, the mission, or even human life at stake. When such systems encompass a substantial interactive component, the same level of confidence is required towards the human-computer interface. Conventional empirical or semiformal techniques, although very fruitful, do not provide sufficient insight on the reliability of the human-system cooperation, and offer no easy way to, for example, quantitatively and qualitatively compare two design options with respect to that reliability. The aim of this article is to present a user interface description language (called ICOs) for the engineering and development of usable and reliable user interfaces. The CASE tool supporting the ICOs notation (called Petshop) is a Petri nets-based-tool for the design, specification, prototyping, and validation of interactive software. In that environment models (built with the formal description technique ICOs) of the interactive application can be interactively modified and executed. This is used to support prototyping phases (when the models and the interactive application evolve significantly to meet late user requirements, for instance) as well as the operation phase (after the system is deployed). The use of ICOs and PetShop is presented on several large-scale systems such as a multimodal ground segment application for satellite control, an air traffic control interactive application, and an application for new generation of interactive cockpits in large civil aircraft such as Airbus A380 or Boeing 787. The article emphasizes the demonstration of the expressive power of the notation and how it can support the description of various aspects of user interfaces, namely interaction techniques (both WIMP and post-WIMP), interactive components (such as widgets), and the behavioral part of interactive applications such as the dialog and the functional core. It also demonstrates that PetShop provides dedicated support for prototyping activities of behavioral aspects at the various levels of the architecture of interactive systems. While the focus is on past work done on various large-scale applications, the article also highlights why and how ICOs and Petshop are able to address challenges raised by next-generation user interfaces.

Fusion engines for multimodal input: a survey

Fusion engines are fundamental components of multimodal inter-active systems, to interpret input streams whose meaning can vary according to the context, task, user and time. Other surveys have considered multimodal interactive systems; we focus more closely on the design, specification, construction and evaluation of fusion engines. We first introduce some terminology and set out the major challenges that fusion engines propose to solve. A history of past work in the field of fusion engines is then presented using the BRETAM model. These approaches to fusion are then classified. The classification considers the types of application, the fusion principles and the temporal aspects. Finally, the challenges for future work in the field of fusion engines are set out. These include software frameworks, quantitative evaluation, machine learning and adaptation.

High-Fidelity Prototyping of Interactive Systems Can Be Formal Too

The design of safety critical systems calls for advanced software engineering models, methods and tools in order to meet the safety requirements that will avoid putting human life at stake. When the safety critical system encompasses a substantial interactive component, the same level of confidence is required towards the human-computer interface. Conventional empirical or semi-formal techniques, although very fruitful, do not provide sufficient insight on the reliability of the human-system cooperation, and offer no easy way to, for example, quantitatively compare two design options. The aim of this paper is to present a method, with supporting tools and techniques, for engineering the design and development of usable user interfaces for safety-critical applications. More precisely we present the Petshop environment which is a Petri net based tool for the design specification, prototyping and validation of interactive software. In this environment models of the interactive application can be interactively modified and executed. This is used to support prototyping phases (when the models and the interactive application evolve significantly to meet late user requirements for instance) as well as in the operation phase (after the system is deployed). The use of the description technique (the ICO formalism) supported by PetShop is presented on a multimodal ground segment application for satellite control and more precisely how prototyping can be performed at the various levels of the architecture of interactive systems.

An Architecture and a Formal Description Technique for the Design and Implementation of Reconfigurable User Interfaces

This paper proposes an architecture that provides a means to handle failures of input and output devices. This handling is done by means of previously defined and designed configurations. According to the failure identified at runtime of the interactive system, the most appropriate configuration will be loaded and executed. Such reconfiguration aims at allowing operators to continue interacting with the interactive system even though part of the user interface hardware has failed. These types of problems arise in domains such as command and control systems where the operator is confronted with several display units and can use various combinations of input devices either in a mono-modal or in a multimodal manner.

Formal description techniques to support the design, construction and evaluation of fusion engines for sure (safe, usable, reliable and evolvable) multimodal interfaces

Representing the behaviour of multimodal interactive systems in a complete, concise and non-ambiguous way is still a challenge for formal description techniques (FDT). Depending on the FDT, multimodal interactive systems feature specific characteristics that are either cumbersome or impossible to capture with classical FDT. This is due to the multiple (potentially synergistic) use of modalities and the strong temporal constraints usually encountered in this kind of systems that have to be dealt with exhaustively if FDT are used. This paper focuses on the requirements for the modelling and construction of fusion engines for multimodal interfaces. It proposes a formal description technique dedicated to the engineering of interactive multimodal systems able to address the challenges of fusion engines. Such benefits are presented on a set of examples illustrating both the constructs and the process.

A formal approach supporting the comparative predictive assessment of the interruption-tolerance of interactive systems

This paper presents an approach for investigating in a predictive way potential disruptive effects of interruptions on task performance in a multitasking environment. The approach combines previous work in the field of interruption analysis, formal description techniques for interactive systems and stochastic processes to support performance analysis of user activities constrained by the occurrence of interruptions. The approach uses formal description techniques to provide a precise description of user tasks, and both system and interruptions behavior. The detailed mechanism by which systems and interruptions behave is first described using a Petri nets-based formal description technique called Interactive Cooperative Objects (ICO). The use of a formal modeling technique for the description of these three components makes it possible to compare and analyze different interaction techniques. In particular, it allows us to determine which of the system states are most affected by the occurrence of interruptions. Once composed together, models describing the system, user tasks and interruptions behavior are transformed into PEPA models (i.e. Performance Evaluation Process Algebra) that are amenable to performance analysis using the PRISM model checker. The approach is exemplified by a simple example that models two interaction techniques for manipulating icons in a desktop environment.

Designing for resilience to hardware failures in interactive systems: A model and simulation-based approach

The paper proposes a formal description technique and a supporting tool that provide a means to handle both static and dynamic aspects of input and output device configurations and reconfigurations. More precisely, in addition to the notation, the paper proposes an architecture for the management of failure on input and output devices by means of reconfiguration of in/output device configuration and interaction techniques. Such reconfiguration aims at allowing operators to continue interacting with the interactive system even though part of the hardware side of the user interface is failing. These types of problems arise in domains such as command and control systems where the operator is confronted with several display units. The contribution presented in the paper thus addresses usability issues (improving the ways in which operators can reach their goals while interacting with the system) by increasing the reliability of the system using diverse configuration both for input and output devices.

Une approche à base de modèles pour l'ingénierie logicielle de techniques d'interaction

While model-based approaches have been used for over 30 years in the field of behavioral description of interactive systems [27], the link between these approaches and user-centered design process remain insufficiently explained. This paper offers a contribution to this problem by presenting how a model-based approach can be exploited to facilitate the tasks of evaluation of usability that are often laborious and repetitive. The basic principle of this approach promotes the use of recording and analysis of log data in a model-based environment. The results described in this paper show that the log data at model level can be used not only to identify usability problems but also to identify changes to these models in order to correct the encountered problems. This approach is integrated in a process and is supported by a modelbased CASE tool for modeling, simulating and evaluating interactive systems. The case study illustrates the principles of the approach and operation of the tool on an interaction technique. It shows how the analysis of log data allows the designer to easily tune the interaction technique (as the results of the analysis of log data are presented at the same abstraction level than models). It proposes an alternative to user tests that are very difficult to configure and to interpret especially when advanced interaction techniques are concerned.

Une approche formelle pour i'evaluation de la tolérance aux interruptions des système interactifs

This paper presents an approach for investigating potential disruptive effects of interruptions on task performance in a multitasking environment. The approach combines previous work in the field of interruption analysis, formal description techniques for interactive systems and stochastic processes to support performance analysis of user tasks constrained by the occurrence of interruptions in the working environment. The approach uses formal description techniques to provide a comprehensive description of user tasks, system and interruption behaviour. The detailed mechanism by which systems and interruptions behave is presented using a Petri nets-based formal description technique called Interactive Cooperative Objects (ICO). The use of a formal modeling technique for the description of these three components makes it possible to compare, analyze and integrate them. In particular, it allows us to determine which of the system states are actually affected by the occurrence of interruptions. The approach is exemplified by a case study that implements two interaction techniques for manipulating icons in a desktop environment.