Roman Popp

A discourse model for interaction design based on theories of human communication

Most current models of interaction design build on scenarios and task analysis. We think that interaction design should be more along the lines of communication between humans. With this motivation, our paper presents a new approach to modeling interaction design based on insights from theories of human communication. From such discourse models, we aim for automated generation of user interfaces.

Automatic generation of the behavior of a user interface from a high-level discourse model

In addition to the structure and “look” of a user interface (UI), its behavior needs to be defined. For a fullyautomated UI generation, of course, it will have to be generated fully automatically as well. We avoid that finite-state machines or similar would have to be created manually by a UI designer. Instead, we start from a largely declarative high-level discourse model including a few procedural constructs. Based on our definitions of the procedural semantics of all parts of such a discourse model, we are able to automatically generate a finite-state machine that fully defines the behavior of the generated UI. In this way, we show how automatic generation of the behavior of a user interface is possible from a high-level discourse model.

Optimized GUI Generation for Small Screens

More and more devices with small screens are used to run the same application. In order to reduce usability problems, user interfaces (UIs) specific to screen size (and related resolution) are needed, but it is time consuming and costly to implement all the different UIs manually.

Tool support for automated multi-device GUI generation from discourse-based communication models

Automated generation of graphical user interfaces (GUIs) from models is possible, but their usability is often not good enough for real-world use, in particular not for small devices. Also automated tailoring of GUIs for different devices is still an issue. Our tools provide such tailoring for different devices through automatic optimization of corresponding optimization objectives under given constraints. Currently, two different optimization strategies are implemented, with their focus on tapping and vertical scrolling on touchscreen, respectively. The constraints (relevant properties such as screen size and resolution) are to be provided by the users of our tools in device specifications. Through our tool support, WIMP (window, icon, menu, pointer) GUIs can be generated at a decent level of usability nearly automatically, in particular for small devices. This is important due to the more and more widespread use of smartphones.

Fully automatic user interface generation from discourse models

Automatic generation of user interfaces (UIs) has made some progress, but it still faces many challenges, especially when starting from high-level models. We developed an approach and a supporting tool for modeling discourses, from which the tool can generate WIMP (window, icon, menu, pointer) UIs automatically. This involves several complex steps, most of which we have been able to implement using model-driven transformations. When given specific target platform specifications, UIs for a variety of devices such as PCs, mobile phones and PDAs can be generated automatically.

Generating an Abstract User Interface from a Discourse Model Inspired by Human Communication

Programming user interfaces is hard, error-prone and expensive, but recent advances allow generating them from interaction design models. We present an approach for modeling interaction design that is inspired by human communication. Our interaction design models are discourse models, more precisely models of dialogues. They are based on theories of human communication and should, therefore, be more understandable to humans than programs implementing user interfaces. The main ingredients of our models are communicative acts (Speech Act Theory), which are connected as adjacency pairs (Conversation Analysis) and via RST relations (Rhetorical Structure Theory). While RST provides useful means for modeling discourse in the sense of monologue, a dialogue results from connecting monologues via adjacency pairs. This paper presents a new metamodel that integrates these approaches. Based on it, we show how abstract user interfaces can be generated from such discourse models. In a nutshell, we generate finite-state machinery and employ rules devised by us to map parts of a discourse model to abstract widgets.

Fully-automatic generation of user interfaces for multiple devices from a high-level model based on communicative acts

The problems involved in the development of user interfaces become even more severe through the ubiquitous use of a variety of devices such as PCs, mobile phones and PDAs. Each of these devices has its own specifics that require a special user interface. Therefore, we developed and implemented an approach to generate user interfaces for multiple devices fully automatically from a high-level model. In contrast to previous approaches focusing on abstracting the user interface per se, we make use of speech act theory from the philosophy of language for the specification of desired intentions in interactions. Our new approach of using communicative acts in high-level models of user interfaces allows their creation with less technical knowledge, since such models are easier to provide than user-interface code in a usual programming language. From one such high-level model, multiple user interfaces for diverse devices are rendered fully automatically using a number of heuristics. A generated user interface for a PDA is already in real-world use and its usability was informally evaluated as good

Modeling of interaction design by end users through discourse modeling

End users of software typically have to let someone else develop it and its user interface, or to learn to design and to program it themselves. Especially user interfaces developed by someone else may not fit well the given task. Designing and programming is hard and takes a lot of effort in general, and even more so for people not especially trained or experienced. Therefore, we propose end-user development of user interfaces through a new approach and interface for discourse modeling. End users may themselves model an interaction design as a discourse (in the sense of a dialogue between human and computer). From such an interaction design, eventually a user interface is to be generated automatically by a tool. As a consequence, end-user development becomes end-user modeling instead of programming.

Using communicative acts in high-level specifications of user interfaces for their automated synthesis

User interfaces are very important for the success of many computer-based applications these days. However, their development takes time, requires experts for user-interface design as well as experienced programmers and is very expensive. This problem becomes even more severe through the ubiquitous use of a variety of devices such as PCs, mobile phones, PDAs etc., since each of these devices has its own specifics that require a special user interface.Therefore, we developed a tool-supported approach to automatically synthesize multi-device user interfaces from high-level specifications in the form of models. In contrast to previous approaches focusing on abstracting the user interface per se, we make use of communicative acts derived from speech act theory for the specification of desired user intentions in interactions. In this way, we approach a solution to the given problem, since user interfaces can be efficiently provided without experience in implementing them.

Automated generation of device-specific WIMP UIs: weaving of structural and behavioral models

Any graphical user interface needs to have defined structure and behavior. So, in particular, models of Window / Icon / Menu / Pointing Device (WIMP) UIs need to represent structure and behavior at some level of abstraction, possibly in separate models. High-level conceptual models such as Task or Discourse Models do not model the UI per se. Therefore, in the course of automated generation of (WIMP) UIs from such models, structure and behavior of the UI need to be generated, and they need to fit together. In order to achieve that, we devised a new approach to weaving structural and behavioral models on different levels of abstraction.