Joëlle Coutaz

A Unifying Reference Framework for Multi-Target User Interfaces

This paper describes a framework that serves as a reference for classifying user interfaces supporting multiple targets, or multiple contexts of use in the field of context-aware computing. In this framework, a context of use is decomposed in three facets: the end users of the interactive system, the hardware and software computing platform with which the user have to carry out their interactive tasks and the physical environment where they are working. Therefore, a context-sensitive user interface is a user interface that exhibits some capability to be aware of the context (context awareness) and to react to changes of this context. This paper attempts to provide a unified understanding of context-sensitive user interfaces rather than a prescription of various ways or methods of tackling different steps of development. Rather, the framework structures the development life cycle into four levels of abstraction: task and concepts, abstract user interface, concrete user interface and final user interface. These levels are structured with a relationship of reification going from an abstract level to a concrete one and a relationship of abstraction going from a concrete level to an abstract one. Most methods and tools can be more clearly understood and compared relative to each other against the levels of this framework. In addition, the framework expresses when, where and how a change of context is considered and supported in the context-sensitive user interface thanks to a relationship of translation. In the field of multi-target user interfaces is also introduced, defined, and exemplified the notion of plastic user interfaces. These user interfaces support some adaptation to changes of the context of use while preserving a predefined set of usability properties

Context is key

Context is not simply the state of a predefined environment with a fixed set of interaction resources. Its part of a process of interacting with an ever-changing environment composed of reconfigurable, migratory, distributed, and multiscale resources.

A generic platform for addressing the multimodal challenge

Multimodal interactive systems support multiple interaction techniques such as the synergistic use of speech and direct manipulation. The flexibility they offer results in an increased complexity that current software tools do not address appropriately. One of the emerging technical problems in multimodal interaction is concerned with the fusion of information produced through distinct interaction techniques. In this article, we present a generic fusion engine that can be embedded in a multi-agent architecture modelling technique. We demonstrate the fruitful symbiosis of our fusion mechanism with PAC-Amodeus, our agentbased conceptual model, and illustrate the applicability of the approach with the implementation of an effective interactive system: MATIS, a Multimodal Airline Travel Information System.

PAC, an Object Oriented Model for Dialog Design

PAC is an implementation model that attempts to bridge the gap between the abstract sphere of theoretical models and the practical affairs of building user interfaces. It takes as a basis the vertical decomposition of human-computer interaction into semantic, syntactic and pragmatic layers as promoted by some theoretical models. However, PAC stresses the fact that these notions do not form strict monolithic layers but are distributed across related “chunks”, called interactive objects. For doing so, PAC recursively structures an interactive application in three parts: the Presentation, the Abstraction and the Control. The Presentation defines the the concrete syntax of the application whereas the Abstraction corresponds to the semantics. The Control maintains the mapping and the consistency between the abstract entities and their presentation to the user. The Presentation of an application is in turn decomposed into a set of interactive objects, entities specialized in man-machine communication. As for applications, an interactive object is organized according to the PAC model. PAC has been used for the construction of two interactive applications and is currently applied to the development of a User Interface Management System.

Perceptual Components for Context Aware Computing

In this paper we propose a software architecture for observing and modeling human activity. This architecture is derived from an ontology for context awareness. We propose a model in which a users context is described by a set of roles and relations. Different configurations of roles and relations correspond to situations within the context. The components of a context model are used to specify processes for observing activity. The ontology for context modeling is derived from both a bottom up systems perspective and a topdown users perspective. As we define each element, we describe the corresponding components of a process-based software architecture. Using these components, a context is translated into a federation of observational processes. This model leads to an architecture in which reflexive elements are dynamically composed to form federations of processes for observing and predicting the situations that make up a context.

CAMELEON-RT: A software architecture reference model for distributed, migratable, and plastic user interfaces

This paper defines the problem space of distributed, migratable and plastic user interfaces, and presents CAMELEON-RT, a technical answer to the problem. CAMELEON-RT1 is an architecture reference model that can be used for comparing and reasoning about existing tools as well as for developing future run time infrastructures for distributed, migratable, and plastic user interfaces. We have developed an early implementation of a run time infrastructure based on the precepts of CAMELEON-RT.

User interface plasticity: model driven engineering to the limit!

Ten years ago, I introduced the notion of user interface plasticity to denote the capacity of user interfaces to adapt, or to be adapted, to the context of use while preserving usability. The Model Driven Engineering (MDE) approach, which was used for user interface generation since the early eighties in HCI, has recently been revived to address this complex problem. Although MDE has resulted in interesting and convincing results for conventional WIMP user interfaces, it has not fully demonstrated its theoretical promises yet. In this paper, we discuss how to push MDE to the limit in order to reconcile high-level modeling techniques with low-level programming in order to go beyond WIMP user interfaces.

A Model-Driven Engineering Approach for the Usability of Plastic User Interfaces

Plastic User Interfaces (UI) are able to adapt to their context of use while preserving usability. Research efforts have focused so far, on the functional aspect of UI adaptation, while neglecting the usability dimension. This paper investigates how the notion of mapping as promoted by Model Driven Engineering (MDE), can be exploited to control UI adaptation according to explicit usability criteria. In our approach, a run-time UI is a graph of models related by mappings. Each model (e.g., the task model, the Abstract UI, the Concrete UI, and the final UI) describes the UI from a specific perspective from high-level design decisions (conveyed by the task model) to low-level executable code (i.e. the final UI). A mapping between source and target models specifies the usability properties that are preserved when transforming source models into target models. This article presents a meta-model for the notion of mapping and shows how it is applied to plastic UIs.

Meta-user interfaces for ambient spaces

In this article, we propose the concept of meta-User Interface (meta-UI) as the set of functions (along with their user interfaces) that are necessary and sufficient to control and evaluate the state of interactive ambient spaces. This set is meta-, since it serves as an umbrella beyond the domain-dependent services that support human activities in an ambient interactive space. They are User Interface-oriented since their role is to help users to control and evaluate the state of this space. We present a dimension space to classify, compare, and contrast disparate research efforts in the area of meta-UIs. We then exploit the generative power of our design space to suggest directions for future research.

Model-driven adaptation for plastic user interfaces

User Interface (UI) plasticity denotes UI adaptation to the context of use (user, platform, physical and social environments) while preserving usability. In this article, we focus on the use of Model-Driven Engineering and demonstrate how the intrinsic flexibility of this approach can be exploited by designers for UI prototyping as well as by end-users in real settings. For doing so, the models developed at design-time, which convey high-level design decisions, are still available at run-time. As a result, an interactive system is not limited to a set of linked pieces of code, but is a graph of models that evolves, expresses and maintains multiple perspectives on the system from top-level tasks to the final UI. A simplified version of a Home Heating Control System is used to illustrate our approach and technical implementation.