Peter Forbrig

Multi-devices Multiple user interfaces: development models and research opportunities

Today, Internet-based appliances can allow a user to interact with the server-side services and information using different kinds of computing platforms including traditional office desktops, palmtops, as well as a large variety of wireless devices including mobile telephones, Personal Digital Assistants, and Pocket Computers. This technological context imposes new challenges in user interface software engineering, as it must run on different computing platforms accommodating the capabilities of various devices and the different contexts of use. Challenges are triggered also because of the universal access requirements for a diversity of users. The existing approaches of designing a single user interface using one computing platform do not adequately address the challenges of diversity, cross-platform consistency, universal accessibility and integration. Therefore, there is an urgent need for a new integrative framework for modeling, designing and evaluating multi-device user interfaces for the emerging generation of interactive systems. This paper begins by describing a set of constraints and characteristics intrinsic to multi-device user interfaces, and then by examining the impacts of these constraints on the specification, design and validation processes. Then, it discusses the research opportunities in important topics relevant to multi-device user interface development, including task and model-based, pattern-driven and device-independent development. We will highlight how research in these topics can contribute to the emergence of an integrative framework for Multiple-User Interface design and validation.

Patterns in Model-Based Engineering

In this paper we demonstrate how patterns can act as a driving force for the development of interactive applications. As knowledge re-use is becoming more and more crucial, patterns can be an effective tool to represent knowledge of the HCI domain. Using a model-based development methodology, it is shown how patterns can act as building blocks for the establishment of these models. Starting from outlining the general process of pattern application, we discuss how and which patterns are suitable for several models. In particular we discuss the application and use of patterns for the task, dialog and presentation models. Furthermore, we suggest an interface for patterns using “generic classes” and give concrete examples to corroborate our approach. This allows for modular patterns reuse and plausible parameter exchange with the underlying system. Tool support is based on XML-representations of patterns using a template engine.

Modeling patterns for task models

Models allow us to describe complex systems at different abstract and conceptual levels, hence amplify our analytical and problem solving capabilities, However, a lot of human effort and experience is needed to build correct models, and to translate them to concrete artifacts: in our case a usable user interface. This paper introduces the concept of task and pattern models to leverage the process of task modeling, and show how it can help build generic task models, link them, and instantiate them more readily. Once seen as patterns, we will demonstrate that task models can be disseminated and reused more easily by representing them as predefined types.

Design, specification and verification of interactive systems

Invited Talks.- Retrospective and Challenges for Model-Based Interface Development.- Human Factors in Aeronautics.- Abstract Interpretation and Application to Interactive System Verification.- Moving Towards Implementation.- Device Models.- A Formal Description of Low Level Interaction and its Application to Multimodal Interactive Systems.- Deriving a Formal Model of an Interactive System from its UIL Description in order to Verify and Test its Behaviour.- Prototyping Device Interfaces with DSN/2.- Evaluating Formal Languages.- Toward More Understandable User Interface Specifications.- Towards an Integrated Proposal for Interactive Systems Design Based on TLIM and ICO.- The Evaluation of User Interface Notations.- Analysing Errors.- Supporting Error-Driven Design.- Risk Analysis, Impact and Interaction Modelling.- Design Assistance for User-Adapted Interaction.- Design, Specification and Verification.- GRALPLA: an Algebraic Specification Language for Interactive Graphic Systems.- Fusion Engines and Melting Pots.- Monolingual, Articulated Modeling of Users, Devices, and Interfaces.- Evaluating the Interfaces of Three Theorem Proving Assistants.- Validating Properties of Component-based Graphical User Interfaces.- Specifying and Reasoning About CSCW.- Reports from Working Groups.- The Namur Principles: Criteria for the Evaluation of User Interface Notations.- The Role of Formalisms.

Task models as basis for requirements engineering and software execution

In this paper we discuss an approach linking GUI specifications to abstract dialog models. Both specifications are based on task models describing behavioral features. It will be shown how first prototypes of interactive systems, which are generated from user interface models, can help to capture requirements. Users can interactively play with prototypes. Tool support is also provided for co-operative work of different users, which starts with abstract canonical prototypes that can evolve to concrete GUI specifications.

Model-Based User Interface Design Using Markup Concepts

In the field of model-based development of interactive systems, several approaches have been proposed to integrate task and object knowledge into the development process and its underlying representations. This paper follows such an approach with a special focus on mobile devices. It presents a concept of device independent user interface design based on the XML-technology. The concept is applied to an e-commerce example.

Towards an integrated approach for task modeling and human behavior recognition

Mobile and ubiquitous systems require task models for addressing the challenges of adaptivity and situation-aware assistance. Today, both challenges are seen as separate issues in system development, addressed by different modeling concepts. We propose an approach for a unified modeling concept that uses annotated hierarchical task trees for synthesizing models for both areas from a common basic description.

Practical extensions for task models

The current set of temporal operators is insufficient to make effective use of task models as specifications for user interfaces. Moreover, the predominant monolithic task tree structure does not scale well for sizable applications. In order to overcome these shortcomings, a small collection of practical extensions for task models is proposed. In particular, we define new temporal operators (stop, non-deterministic choice, deterministic choice and instance iteration), concepts in support of modularization and a high-level task diagram notation. Finally, we introduce a new concept for expressing cooperative task models that distinguishes between different roles as well as between actors fulfilling these roles.

Linking GUI elements to tasks: supporting an evolutionary design process

In this paper we discuss an approach for linking GUI specifications to more abstract dialogue models and supporting an evolutionary design process. These specifications are linked to task models describing behavioural characteristics. First prototypes of interactive systems are interactively generated. The automatically generated XUL specifications are refined using a GUI editor, which allows replacing of user interface elements by other elements or components. Components are predesigned parts of user interfaces, and could optionally be parameterised. This enables us to support GUI design using patterns.For illustration we are going to show a design cycle from task model to abstract user interfaces and finally to a concrete user interface. The design process is supported by patterns.Additionally, a proposal is presented of how to keep connections between concrete user interface (CUI), abstract user interface (AUI) and a task model. These connections can be used to propagate changes in a task model or an AUI only to affected parts of a CUI, instead of recreating the whole CUI after a modification has been done.In this paper we primarily focus on deletion, moving and adding tasks to an AUI and the implication of these procedures to a CUI. More complicated problems like splitting or merging of tasks are not addressed in this paper.The avoiding of unnecessary recreation processes for a CUI can reduce the amount of time and work needed in model-driven software development.

Higher-Order Task Models

Based on a revision of the common task understanding in model-based approaches a formal meta-model is proposed which allows more expressive task descriptions. A task is considered as a meta-action which produces models about situations, goals and actions by applying domain knowledge. The approach unifies procedural and state knowledge. As a consequence, higher-order actions are possible. The suggested approach is an answer to the need for more dynamic task models. It supports specifications of context sensitive and cooperative tasks.