Michael E. Atwood

Project ernestine: validating a GOMS analysis for predicting and explaining real-world task performance

Project Ernestine served a pragmatic as well as a scientific goal: to compare the worktimes of telephone company toll and assistance operators on two different workstations and to validate a GOMS analysis for predicting and explaining real-world performance. Contrary to expectations, GOMS predicted and the data confirmed that performance with the proposed workstation was slower than with the current one. Pragmatically, this increase in performance time translates into a cost of almost

A Survey of Design Rationale Systems: Approaches, Representation, Capture and Retrieval

2 million a year to NYNEX. Scientifically, the GOMS models predicted performance with exceptional accuracy. The empirical data provided us with three interesting results: proof that the new workstation was slower than the old one, evidence that this difference was not constant but varied with call category, and (in a trial that spanned 4 months and collected data on 72,450 phone calls) proof that performance on the new workstation stabilized after the first month. The GOMS models predicted the first two results and explained all three. In this article, we discuss the process and results of model building as well as the design and outcome of the field trial. We assess the accuracy of GOMS predictions and use the mechanisms of the models to explain the empirical results. Last, we demonstrate how the GOMS models can be used to guide the design of a new workstation and evaluate design decisions before they are implemented.

GOMS meets the phone company: analytic modeling applied to real-world problems

This paper provides a survey on recent research in the area of design rationale. The study of design rationale spans a number of diverse disciplines, touching on concepts from research communities in mechanical design, software engineering, artificial intelligence, civil engineering, computer-supported cooperative work, and human-factors and human-computer interaction research. We focus this survey on prototype design rationale systems for these application domains, and put forward several major criteria with which to describe and classify design rationale systems, including argumentation-based, descriptive, process-based approaches. Further, we attempt to abstract the place of systems and tools for design rationale capture and retrieval in the context of contemporary knowledge-based engineering and Computer-Aided Design (CAD) tools. This survey is structured around classes of fundamentally different approaches, their representation schema, their capture methods and retrieval techniques. A number of recent design rationale systems, including JANUS, COMET, ADD. REMAP, HOS, PHIDIAS, DRIVE and IBIS are analysed. We conclude with an assessment of current state-of-the-art and a discussion of critical open research issues.

Effective Design Rationale: Understanding the Barriers

GOMS analyses were used to interpret some perplexing data from a field evaluation of two telephone operator workstations. The new workstation is ergonomically superior to the old and is preferred by all who have used it. Despite these advantages telephone operators who use the new workstation are not faster than those who use the old but are, in fact, significantly slower. This bewildering result makes sense when seen with the aid of GOMS. With GOMS we can see that very few of the eliminated keystrokes or ergonomic advantages affect tasks that determine the operators work time. Indeed, GOMS shows that some presumed procedural improvements have the contrary effect of increasing the time an operator spends handling a phone call. We conclude that if GOMS had been done early on, then the task, not the workstation, would have been redesigned.

Dynamic Forms: An Enhanced Interaction Abstraction Based on Forms

One goal of design rationale systems is to support designers by providing a means to record and communicate the argumentation and reasoning behind the design process. However, there are several inherent limitations to developing systems that effectively capture and utilize design rationale. The dynamic and contextual nature of design and our inability to exhaustively analyze all possible design issues results in cognitive, capture, retrieval, and usage limitations. In this chapter we analyze these issues in terms of current perspectives in design theory, and describe the implications to design research. We discuss the barriers to effective design rationale in terms of three major goals: reflection, communication, and analysis of design processes. We then suggest alternate means to achieve these goals that can be used with or instead of design rationale systems.

Design rationale: the rationale and the barriers

Dynamic Forms supports developers in creating a single, dynamic, scrollable form using a form description language. The virtual form is structured into sections and subsections so that effective organization and navigation of the information are possible. The object-oriented, textual, and interpretive nature of the language allow developers to incorporate user suggestions concerning changes to a dynamic form quickly and with minimal tools. Dynamic Forms is based on an understanding of the environment in which data-entry tasks are carried out, the manner in which users perform their tasks, and how the users utilize human and computer resources in solving these tasks.

Facilitating communication in software development

One goal of design rationale systems is to support designers by providing a means to record and communicate the argumentation and reasoning behind the design process. However, there are several inherent limitations to developing systems that effectively capture and utilize design rationale. The dynamic and contextual nature of design and our inability to exhaustively analyze all possible design issues results in cognitive, capture, retrieval, and usage limitations. In addition, there are the organizational limitations that ensue when systems are deployed. In this paper we analyze these issues in terms of current perspectives in design theory and describe the implications to design research. We discuss the barriers to effective design rationale in terms of three major goals: reflection, communication, and analysis of design processes. We then suggest alternate means to achieve these goals that can be used with or instead of design rationale systems.

Changing perspectives on evaluation in HCI: past, present, and future

ABSTRACT Effective communication is critical to the success of a softwaredevelopment project. It factors into the productivity of individualsand organizations, and has particular impact when change occurs.Yet communication is generally left unsupported by the softwaredevelopment process and by the communication infrastructure. Weaddress this issue in the context of two software developmentprojects at N YNEX through a conceptual framework called DesignIntent. There are three innovations in our approach. Design Intentencourages stakeholders to engage in active listening, enablesstakeholders to collaboratively construct a consistent understand-ing of the development effort, and provides a communicationinfrastructure for stakeholders to share ideas and participate in dis-cussions. INTRODUCTION Effective communication among the stakeholders of a softwaredevelopment project is crucial to its success. The importance ofthis communication has been well documented by Curtis, Krasner,and Iscoe [8], who noted frequent, recurring problems related tothe lack of adequate communication among those involved in thedevelopment effort. In order to improve the communication amongmembers of a software development team, an effective process andthe infrastructure to support it must be provided.As designers, we have a new role of “designing experiences” orways for people within our corporation to appreciate new ideas.This role is “designing the boundary objects that facilitate commu-nication and the interpretative moves [leaping] of overlappingcommunities of practice”[18]. We need to build not only “proto-types of need or use” and “prototype systems” [2] — but also theinfrastructures that support relationships, work practices, andsocial intercourse in communities of learners and knowledgeworkers.We feel that three main activities are essential for producing goodsoftware systems:1. Active listening andinterpretive leaping: understanding theproblem and how to solve it in a significant way — offering amodel of transcendence.2. Designing boundary objects that help peopleexperiencethepower and possibilities of new ideas.3. Facilitating the communication of ideas and innovations bybuilding the infrastructures.We refer to them as the three dimensions of Design Intent. We willdiscuss our experiences along these dimensions using two projectsat

Prototyping Considered Dangerous

Evaluation has been a dominant theme in HCI for decades, but it is far from being a solved problem. As interactive systems and their uses change, the nature of evaluation must change as well. In this paper, we outline the challenges our community needs to address to develop adequate methods for evaluating systems in modern (and future) use contexts. We begin by tracing how evaluation efforts have been shaped by a continuous adaptation to technological and cultural changes and conclude by discussing important research directions that will shape evaluations future.

Pattern languages in the wild: exploring pattern languages in the laboratory and in the real world

In this paper, we argue that prototypes can hinder, rather than aid effective communication. The dangers are: (1) prototypes may contain hidden assumptions which might surface too late; (2) obtaining feedback in the context of use is prohibitively expensive and rarely done; and (3) partly as a consequence of these problems, prototypes cause a focus on displays and other surface features of computers, not on the more difficult problem of how people function in their environments to solve problems. We propose design intent which specifies how a system will fit in and interact with the environment in which it is placed and expectation agents which monitor the system in use and detect uses counter to the intent as ways to alleviate these dangers.