Robin R. Penner

Model-based automation of the design of user interfaces to digital control systems

Digital control systems, like those controlling the functions of buildings or industrial processes, pose a number of special problems for good user interface design. The general problems of providing usability, common to all systems, include difficulty in accessing and applying principles of good design. In addition, digital control systems can have multiple users, with multiple roles, and each installation has different configurations of systems, controls, and user interface devices. Providing interactions for the users of building control systems is often achieved by manually implementing each required display. This is an expensive solution, which often produces less than optimal results. We address these problems through the automation of user interface design. Our solution, called DIGBE (dynamic interface generation for building environments), separates the domain knowledge, interaction design, and presentation heuristics into multiple collaborating models. Each model contains knowledge about a particular aspect of interface design, and uses this knowledge to dynamically create each user interface that is needed to support the users of a control system. DIGBE demonstrates that it is possible to automatically and dynamically create consistent and individualized user interfaces from model-based design knowledge.

Dynamic user interface adaptation based on operator role and task models

Proper design of human computer interactions currently requires the application of specialized knowledge possessed by talented individuals, often responding to rapidly changing technology and functionality. As computing power and data become more distributed, static design of interactions for dynamic environments may not always produce usable interfaces between people and machines. We have designed a model based, dynamic interaction design system called DIG (Dynamic Interaction Generation) and implemented it in a research prototype using the domain of digital building control systems. DIGBE is a three-tiered system modeling the domain data, the current interaction, and the device-specific presentation of the interaction. It responds to changes in the environment and creates, in real time, an interaction tailored to the current user role, access level, and task. Each interaction is also specialized to match the nature of the information that is currently the focus of the interaction. Full automatic synthesis of any interface at any time may not yet be feasible, but DIGBE demonstrates that it is both possible and useful to automatically generate user interfaces. This may be especially true for complex system domains with variable components but known task structures.

Implementation of automated interaction design with collaborative models

Abstract This paper summarizes the current status of an ongoing research program to explore automated alternatives to the current manual method of designing, implementing, and delivering user interfaces to complex digital control systems. Using examples from two implementations of the model-based interface automation approach that resulted from this research, we explore the models and collaboration required to perform on-demand user interface design. We first discuss the need for automation of the user interface design process and place the work into a research context. Using examples from two implemented systems, we then review the object-oriented models and processes that we used to support interaction design automation. Our findings support the application of model-based automated design approaches in digital control system domains, and particularly emphasize the need for rich semantic support for automated design.

Automating user interface design

In assistive systems that include humans, the human-computer interface provides the substrate for all collaboration. Good user interface design is difficult for even non-assistive applications; the introduction of automated reasoning and agent capabilities only aggravates the problems of human interaction with automation. We have identified a need for a process that mediates between software agents and humans, dynamically adjusts to the situation, and presents a well designed user interface to allow the human to fully collaborate. We report on one such system, ADIEU. We are currently applying ADIEU to joint forces military search and rescue (SAR) operations. The system we are developing, called Search and Rescue Assistant (SARA), is intended to support SAR operations by providing task assistance in an environment where human and software agents act independently as well as collaborate with others to perform search and rescue operations. This effort has provided us with an opportunity to apply and evaluate promising agent architecture and communications concepts, and to develop a testbed for our approach to facilitating human collaboration via dynamic design using ADIEU. The paper presents a current status of ADIEU.

Automated Phase Design and Timing Adjustment for Signal Phase Design

In this paper we describe the design processes of human traffic engineers and the development of an automated system that solves the problem of timing adjustment in signal phase sequence design. Signal phase sequencing refers to the sequence of signal changes and the timing of each change for a system of traffic lights at an isolated intersection. In designing signal sequences, one must decide how much time to allocate for each phase of the sequence so that traffic can flow with a minimum of delay. Other automated phase design systems can generate an initial phase design, but if the initial design is not precisely correct, the timing sequence must be adjusted by hand. This paper presents TIMELY, an automated signal phase design generator that generates an initial phase design and automatically adjusts the timing if necessary. TIMELY uses an interactive redesign strategy which largely mimics that used by human experts. The system invokes a set of redesign rules (which we have dubbed the “Robin Hood” procedure), gathered from human experts, which tweak the phase design. The new design is fed back into a simulator and tested again. The process stops when we find that the average delay is not improved. We take the design from the previous step as our final design. An expert has verified the soundness of all of TIMELYs results.

Facilitating human interactions in mixed initiative systems through dynamic interaction generation

In our work to develop mixed-initiative, collaborative multi-agent societies, we have found that the introduction of automated reasoning capabilities into already complex systems that aid human process management exacerbates the already serious problem of human interaction with automation. Our solution is to incorporate a specially designed team of agents, the interaction management team, into the collaborative agent society. The teams main function is to facilitate the interaction between the societys human participants and the software participants. The interaction management team utilizes an automated object process, called dynamic interaction generation (DIG), to react to the dynamic interaction needs of the human and software collaborators. This paper presents an overview of the teams role in a multi-agent system, followed by a deeper discussion of the design and implementation of DIG.

DIGBE: ONLINE MODEL-BASED DESIGN AUTOMATION

We describe the results of a research and development program which led to the implementation of a model-based system which automatically generates user interfaces to digital control systems. This system, called DIGBE, is based on constraint-based collaboration between three models. DIGBE produces interfaces on demand; each detail of the interface is tailored to the user, the task, and the available data. The models in DIGBE include: a domain model, which builds a representation of the objects in the control system; an interaction model, which builds an abstract representation of the human-computer interaction; and a presentation model, which presents the interaction to the user on a particular hardware device such as a personal computer or a PDA.

Automated interaction design for command and control of military situations

We will demonstrate the SHARED software, which contains an implementation of the Automated Interaction Design (AID) approach to dynamic creation of user interfaces. AID uses multiple agents, multiple models, and productive compositional processes to generate need-based user interfaces within a complex control domain. In addition to demonstrating operational software that responds to military interaction needs, we will present details of the underlying models and operations that support user interface generation in this domain.

CoRaven: model-based design of a cognitive tool for real-time intelligence monitoring and analysis

Describes a model-based design method to develop CoRaven, a decision support tool that is intended to assist military intelligence analysts in managing and interpreting large quantities of battlefield information. In this method, we use observations of practitioners solving specific tasks in order to understand and model how they use information. We use this model of the task to help identify user needs that the tool must support, and, during initial prototyping, to guide usability analyses. We have found task models to be an important consideration in the decision support tool design process that can help to constrain the design space and reduce the time required to develop an effective decision support tool prototype.

SARA: a multi-agent system for collaborative command and control

The Search and Rescue Assistant (SARA) is a multi-agent system that provides task assistance in a real-world situation-dependent military command and control environment. Organized around the metaphor of a society, agents with expertise in various elements of the domain collaborate to provide concise, context-sensitive situation awareness in support of the human user. The report introduces the underlying architecture of SARA, describes the current implementation, and summarizes the conclusions drawn from that experience.