Antonio C. Siochi

The UAN: a user-oriented representation for direct manipulation interface designs

Many existing interface representation techniques, especially those associated with UIMS, are constructional and focused on interface implementation, and therefore do not adequately support a user-centered focus. But it is in the behavioral domain of the user that interface designers and evaluators do their work. We are seeking to complement constructional methods by providing a tool-supported technique capable of specifying the behavioral aspects of an interactive system–the tasks and the actions a user performs to accomplish those tasks. In particular, this paper is a practical introduction to use of the User Action Notation (UAN), a task- and user-oriented notation for behavioral representation of asynchronous, direct manipulation interface designs. Interfaces are specified in UAN as a quasihierarchy of asynchronous tasks. At the lower levels, user actions are associated with feedback and system state changes. The notation makes use of visually onomatopoeic symbols and is simple enough to read with little instruction. UAN is being used by growing numbers of interface developers and researchers. In addition to its design role, current research is investigating how UAN can support production and maintenance of code and documentation.

Computer analysis of user interfaces based on repetition in transcripts of user sessions

It is generally acknowledged that the production of quality user interfaces requires a thorough understanding of the user and that this involves evaluating the interface by observing the user working with the system, or by performing human factors experiments. Such methods traditionally involve the use of video tape, protocol analysis, critical incident analysis, etc. These methods require time consuming analyses and may be invasive. In addition, the data obtained through such methods represent a relatively small portion of the use of a system. An alternative approach is to record all user input and systems output, i.e., log the user session. Such transcripts can be collected automatically and non-invasively over a long period of time. Unfortunately, this produces voluminous amounts of data. There is, therefore, a need for tools and techniques that allow an evaluator to identify potential performance and usability problems from such data. It is hypothesized that repetition of user actions is an important indicator of potential user interface problems.

A study of computer-supported user interface evaluation using maximal repeating pattern analysis

Maximal repeating pattern (MRP) analysis is a recently developed user interface evaluation technique that uses an algorithm to analyze transcripts of user sessions by detecting repeated user actions. Encouraged by results of an initial study of the MRP technique, we conducted a study in which we evaluated a simple prototype interface using both the MRP technique and observation. Interface problems found by observation were also found by MRP analysis. Although the MRP algorithm produced large amounts of data that an interface evaluator had to analyze, we found that by mapping raw user inputs in the transcripts into more abstract classes via prefiltering, we could perform more useful MRP analyses.

Task-oriented representation of asynchronous user interfaces

A simple, task-oriented notation for describing user actions in asynchronous user interfaces is introduced. This User Action Notation (UAN) allows the easy association of actions with feedback and system state changes as part of a set of asynchronous interface design techniques, by avoiding the verbosity and potential vagueness of prose. Use within an actual design and implementation project showed the UAN to be expressive, concise, and highly readable because of its simplicity. The task- and user-oriented techniques are naturally asynchronous and a good match for object-oriented implementation. Levels of abstraction are readily applied to allow definition of primitive tasks for sharing and reusability and to allow hiding of details for chunking. The UAN provides a critical articulation point, bridging the gap between the task viewpoint of the behavioral domain and the event-driven nature of the object-oriented implementational domain. The potential for UAN task description analysis may address some of the difficulties in developing asynchronous interfaces.

A state transition analysis of image search patterns on the web

Image seeking behavior is a complex interaction among many factors. One approach to the study of this behavior is to examine and categorize the search moves made by individuals as they transition from one search state to another. Seventy-one image searches conducted by graduate students on the web were analyzed to identify patterns of search state transitions used and the overall frequency of specific state transitions. Over a thousand state transitions were identified within eighteen state categories. The categories included search tool and collection selection, queries, context moves, and relevance judgments. Maximal Repeating Pattern Analysis (MRP) was used to examine patterns of transition from one search state to another. The patterns of state transition sequences were conceptualized within a framework of search tactics and search strategies.

Customer responsibility for ensuring usability: requirements on the user interface development process

Abstract An organization developing an interactive system will often find usability a compelling, but elusive, goal. The goal of ensuring usability becomes even more difficult to attain when the system is contracted to an outside developer, and control of both process and product becomes remote. Without a clear statement of requirements for the user interface of the system, this control may be lost from the beginning. Further, the cost of interactive system usage is especially significant in the case of contracted-out development, because the customer—the organization that lets a contract for interactive system development—bears the costs of training and poor user productivity. In spite of the good publicity that usability has received lately, most customers still do not state requirements for the user interface of an interactive system. Although it has been shown that the process by which a user interface is developed has an overwhelmingly large effect on usability of the product , almost never are requirements for the interface development process included in the overall system requirements. How, then, can a customer hope to ensure usability in an interactive system that the customer is contracting out for development? We propose a solution: The customer establishes requirements for the user interface development process , even as early as a Request for Proposal (RFP). To successfully write and enforce such requirements, the customer (or a knowledgeable representative of the customer) must be well informed about human-computer interaction, usability, and existing user interface development techniques. As a case study, we describe how one customer organization, the Bureau of Land Management, produced user interface development process requirements and included them in an RFP. We also discuss how a customer can use product requirements in an RFP to describe the desired general interface style without constraining the developer to a specific design.

Software Usability: Requirements by Evaluation

Recent research has established the importance of defining usability requirements as part of the total requirements for a system. Instead of deciding in an ad hoc manner whether or not a human-computer interface is usable, measurable usability requirements are established at the outset. It is common to state such requirements in an operational manner: U% of a sample of the intended user population should accomplish T% of the benchmark tasks within M minutes and with no more than E errors. The formal experiments needed to test compliance with the requirements makes this method costly. This paper presents an alternative method of specifying usability requirements currently being developed and tested on a large software project at Virginia Tech. Briefly, usability requirements are specified by having every member of the software design team and the user interface design team specify the ease of use desired for each proposed functional requirement of the system under development. The individual ratings are then compared in order to arrive at a consensus. It is this consensus that leads to the formal usability requirements which the interface must meet or exceed. As the interface is built, it is rated in the same manner as that used originally to specify the requirements. This method thus provides a structured means of specifying measurable usability requirements and a means of determining whether or not the interface satisfies those requirements. Several other benefits of this method are presented as well.

Computer analysis of user session transcripts for evaluation of the human-computer interface

A joint research venture between Virginia Techs Dialogue Management Project and the Contel Technology Center (CTC) in Chantilly, Virginia, is discussed. Its goal for the Virginia Tech researchers to provide an interactive tool to be used in CTCs usability laboratory to aid in analysis of usability test results and improve the human-computer interface. The authors present the interactive tool, called the MRP (maximal repeating pattern) tool, and the technique on which it is based from two viewpoints: that of Virginia Tech and that of Contel. First, from the academic viewpoint, they describe the MRP technique and why such techniques for usability testing are needed, work on the tool done prior to this joint venture, and the tool itself and how it automates the MRP technique. Then, from the industrial viewpoint, they describe why the MRP tool and technique are of interest, what the MRP tool is being used for in the CTC usability lab, and what results are expected from its use.<<ETX>>

WebWolf: Towards a Simple Framework for Automated Assessment of Webpage Assignments in an Introductory Web Programming Class

Grading student submissions of webpage assignments is difficult and time-consuming because it involves reading and evaluating source code for compliance with assignment specifications. In addition, the pages must be examined in a browser to check for required functionality such as working links. As a result fewer assignments tend to be given, or instructors spend significant time grading at the expense of working with students or other necessary activities. There is thus a need for automated assessment of webpage assignments. This paper describes WebWolf, a simple framework for automated assessment of assignments in an introductory web programming class. A WebWolf program can load web pages, find and inspect elements, click links and make assertions about expected results. WebWolf was evaluated with assignments from three different classes and on a set of assignment submissions with injected errors. WebWolf correctly identified errors in the submissions. Manual grading of a four page assignment took an average of 17 minutes, while the WebWolf program took 35 seconds.