Philippe A. Palanque

ICOs: A model-based user interface description technique dedicated to interactive systems addressing usability, reliability and scalability

The design of real-life complex systems calls for advanced software engineering models, methods, and tools in order to meet critical requirements such as reliability, dependability, safety, or resilience that will avoid putting the company, the mission, or even human life at stake. When such systems encompass a substantial interactive component, the same level of confidence is required towards the human-computer interface. Conventional empirical or semiformal techniques, although very fruitful, do not provide sufficient insight on the reliability of the human-system cooperation, and offer no easy way to, for example, quantitatively and qualitatively compare two design options with respect to that reliability. The aim of this article is to present a user interface description language (called ICOs) for the engineering and development of usable and reliable user interfaces. The CASE tool supporting the ICOs notation (called Petshop) is a Petri nets-based-tool for the design, specification, prototyping, and validation of interactive software. In that environment models (built with the formal description technique ICOs) of the interactive application can be interactively modified and executed. This is used to support prototyping phases (when the models and the interactive application evolve significantly to meet late user requirements, for instance) as well as the operation phase (after the system is deployed). The use of ICOs and PetShop is presented on several large-scale systems such as a multimodal ground segment application for satellite control, an air traffic control interactive application, and an application for new generation of interactive cockpits in large civil aircraft such as Airbus A380 or Boeing 787. The article emphasizes the demonstration of the expressive power of the notation and how it can support the description of various aspects of user interfaces, namely interaction techniques (both WIMP and post-WIMP), interactive components (such as widgets), and the behavioral part of interactive applications such as the dialog and the functional core. It also demonstrates that PetShop provides dedicated support for prototyping activities of behavioral aspects at the various levels of the architecture of interactive systems. While the focus is on past work done on various large-scale applications, the article also highlights why and how ICOs and Petshop are able to address challenges raised by next-generation user interfaces.

Fusion engines for multimodal input: a survey

Fusion engines are fundamental components of multimodal inter-active systems, to interpret input streams whose meaning can vary according to the context, task, user and time. Other surveys have considered multimodal interactive systems; we focus more closely on the design, specification, construction and evaluation of fusion engines. We first introduce some terminology and set out the major challenges that fusion engines propose to solve. A history of past work in the field of fusion engines is then presented using the BRETAM model. These approaches to fusion are then classified. The classification considers the types of application, the fusion principles and the temporal aspects. Finally, the challenges for future work in the field of fusion engines are set out. These include software frameworks, quantitative evaluation, machine learning and adaptation.

StateWebCharts: A Formal Description Technique Dedicated to Navigation Modelling of Web Applications

This paper presents StateWebCharts (SWC), a formal description technique based on statecharts for describing navigation on web applications. This notation extends the classical statecharts notation by adding more necessary concepts such as an appropriate semantics for states and transitions in a Web context, including notions like dialog initiative control and client and server activities. As well as statecharts do, this formal description technique features a graphical representation thus making it easier to use for web designers and formal enough to allow to rigorously reason about properties of navigation models. In order to show the applicability of the notation, we show, in the paper, its use on two real-size web applications.

A formal description of multimodal interaction techniques for immersive virtual reality applications

Nowadays, designers of Virtual Reality (VR) applications are faced with the choice of a large number of different input and output devices leading to a growing number of interaction techniques. Usually VR interaction techniques are described informally, based on the actions users can perform within the VR environment. At implementation time, such informal descriptions (made at design time) yield to ambiguous interpretations by the developers. In addition, informal descriptions make it difficult to foresee the impact throughout the application of a modification of the interaction techniques. This paper discusses the advantages of using a formal description technique (called ICO) to model interaction techniques and dialogues for VR applications. This notation is presented via a case study featuring an immersive VR application. The case study is then used to show, through analysis of models, how the formal notation can help to ensure the usability, reliability and efficiency of virtual reality systems.

A Petri Net based Environment for the Design of Event-driven Interfaces

Modern window-based user interfaces are actually a special kind of reactive system, and Petri nets may be fruitfully used to design such user — computer dialogues. This paper describes a software engineering tool aimed at supporting the use of high-level Petri nets for the specification, design and implementation of user interfaces in an event-driven interface system. We assess the rationale for the use of Petri nets in such a perspective. We then detail the object-oriented software architecture of the environment, and present an original algorithm for interpreting high-level Petri nets in an event-driven environment.

Petri Net Based Design of User-Driven Interfaces Using the Interactive Cooperative Objects Formalism

The research work presented here belongs in the domain of formal specification of human-software interaction. More precisely, we are concerned by the applying a formal specification technique in the various stages of the construction of an user-driven application, the kind supported by most of the current UIMS. We use the Interactive Cooperative Objects (ICO) formalism, in which structural (or static) aspects are described in an object-oriented framework and dynamic (or behavioral) aspects are described with high-level Petri-nets. The formalism, a case study and some of its expected benefits are presented here.

Beyond modelling: an integrated environment supporting co-execution of tasks and systems models

This paper focuses on the articulations of task models and system models. Tasks models are meant to be used by human factor specialists whilst system models are supposed to be produced by software engineers. However, tasks models and systems models represent two different views on how users interacting with a computing system to reach a goal. This paper presents an integration framework aiming to take full advantage of task models and system models that have been developed initially in a separated manner and how these two views can be integrated at the model level and additionally at the tool level. The main contribution of the paper lies in the definition of such integration at the tool level to be used at runtime (while the user is operating the system). Indeed, thanks to this integration contextual help can be offered to the users supporting the construction of the mental bridge between what they have to do (defined in the tasks model) and what the interactive system allows (defined in the system model). The approach, the tools and the integration are presented on a case study of a Weather Radar System (WXR) embedded in aircraft cockpits.

Validating interactive system design through the verification of formal task and system models

This paper addresses the problem of the articulation between task modelling and system modelling in the design of interactive software. We aim at providing solutions allowing the software designers to use efficiently task models during the design process, and to check that the software being built actually corresponds to the requirements elicited during the task analysis phase. The proposed approach is twofold: Firstly, we use the User Action Notation, a semi-formal task modelling formalism, and we present a translation scheme allowing to transform the User Action Notation constructs into Petri nets. Secondly, we use the Interactive Cooperative Objects formalism (based on Petri nets and on the object-oriented approach) to build the model of the system. We finally use the mathematical analysis techniques stemming from the Petri net theory to analyse and validate the cooperation between task models and system model. The approach is presented through a case study, showing the User Action Notation task models, the equivalent Petri net models and the Interactive Cooperative Object system model.

Formal Specification and Prototyping of CORBA Systems

We propose to extend the CORBA interface definition of distributed objects by a behavioral specification based on high level Petri nets. This technique allows specifying in an abstract, concise and precise way the behavior of CORBA servers, including internal concurrency and synchronization. As the behavioral specification is fully executable, this approach also enables to early prototyping and testing of a distributed object system as soon as the behaviors of individual objects have been defined. The paper discusses several implementation issues of the multithreaded, distributed interpreter built for that purpose. The high level of formality of the chosen formalism allows for mathematical analysis of behavioral specifications.

Formal Methods in Human-Computer Interaction

From the Publisher: There are a number of ongoing problems with the design of Interactive Systems, due mainly to the inherent complexity of these systems (which are generally reactive, parallel and user-driven) and to the number of people - including designers, ergonomists, and computer scientists - involved. This volume examines the proposition that formal methods are one of the conceptual tools that can support the design of Interactive Systems, understanding of their behaviour, and reasoning about their properties. All the approaches considered take into account some aspect of the Web environment which is one of the most successful software products of recent years: millions of people use it every day in order to search for, exchange, and modify information. As the case study in this volume, it provides a familiar background against which problems can be discussed. In Formal Methods in Human-Computer Interaction some of the most well-known approaches developed in research laboratories and universities in different countries are presented, thus providing a comprehensive review of the state of the art in the field. It will be of particular interest to academic/industrial researchers and MSc students in the fields of human-computer interaction, software engineering, formal methods, and computer science in general. It will also be suitable for human-computer interaction engineers who are either carrying out research or tackling industrial problems.