Peter G. Polson

An approach to the formal analysis of user complexity

A formal approach to analysing the user complexity of interactive systems or devices is described, based on theoretical results from cognitive psychology. The users knowledge of how to use a system to accomplish the various tasks is represented in a procedural notation that permits quantification of the amount and complexity of the knowledge required and the cognitive processing load involved in using a system. Making a system more usable can be accomplished by altering its design until the knowledge is adequately simplified. By representing the device behaviour formally as well, it is possible to simulate the user-device interaction to obtain rigorous measures of user complexity.

Cognitive walkthroughs: a method for theory-based evaluation of user interfaces

This paper presents a new methodology for performing theory-based evaluations of user interface designs early in the design cycle. The methodology is an adaptation of the design walkthrough techniques that have been used for many years in the software engineering community. Traditional walkthroughs involve hand simulation of sections of code to ensure that they implement specified functionality. The method we present involves hand simulation of the cognitive activities of a user, to ensure that the user can easily learn to perform tasks that the system is intended to support. The cognitive walkthrough methodology, described in detail, is based on a theory of learning by exploration presented in this paper. There is a summary of preliminary results of effectiveness and comparisons with other design methods.

Testing a walkthrough methodology for theory-based design of walk-up-and-use interfaces

The value of theoretical analyses in user interface design has been hotly debated. All sides agree that it is difficult to apply current theoretical models within the constraints of real-world development projects. We attack this problem in the context of bringing the theoretical ideas within a model of exploratory learning [19] to bear on the evaluation of alternative interfaces for walk-up-and-use systems. We derived a “cognitive walkthrough” procedure for systematically evaluating features of an interface in the context of the theory. Four people independently applied this procedure to four alternative interfaces for which we have empirical usability data. Consideration of the walkthrough sheds light on the consistency with which such a procedure can be applied as well as the accuracy of the results.

The acquisition and performance of text-editing skill: a cognitive complexity analysis

Kieras and Polson (1985) proposed an approach for making quantitative predictions on ease of learning and ease of use of a system, based on a production system version of the goals, operators, methods, and selection rules (GOMS) model of Card, Moran, and Newell (1983). This article describes the principles for constructing such models and obtaining predictions of learning and execution time. A production rule model for a simulated text editor is described in detail and is compared to experimental data on learning and performance. The model accounted well for both learning and execution time and for the details of the increase in speed with practice. The relationship between the performance model and the Keystroke-Level Model of Card et al. (1983) is discussed. The results provide strong support for the original proposal that production rule models can make quantitative predictions for both ease of learning and ease of use.

A Comprehension-based Model of Web Navigation and Its Application to Web Usability Analysis

CoLiDeS, a comprehension-based cognitive model of Web navigation, offers a theoretical explanation of the impasses users often encounter during information search and retrieval from the W W W, and also identifies the determinants of success cases. In this model, acting on a single Web page screen object is regarded as the outcome of a multi-step process: parsing the current display containing up to about 200 screen objects into five to ten top-level schematic objects; focusing on one of these top-level schematic objects; comprehending and elaborating the screen objects within the focused-on area; and then selecting one of the actual screen objects as the target for the next action, the object whose representation bears the highest degree of semantic similarity to the user’s goal.

A process model for water jug problems

Abstract We develop and evaluate a model for the water jug task in which a subject is required to find a sequence of moves (pouring operations) which produce a specified amount of water in each jug. Experiment 1 was designed to evaluate the meansends, move selection heuristics that are assumed by the model. Experiment 2 tested the models predictions concerning those aspects of the water jug task that determine problem difficulty. A three stage process model incorporating GPS-like, means-ends heuristics and assumptions concerning the utilization of short- and long-term memory was able to account for differences across problems as well as details of the performance of subjects solving a given problem. We conclude that a GPS-like model that only selects one move at a time (no forward planning of move sequences or setting up of subgoals) can provide a good account of solution behavior in the water jug task.

Tool for accurately predicting website navigation problems, non-problems, problem severity, and effectiveness of repairs

The Cognitive Walkthrough for the Web (CWW) is a partially automated usability evaluation method for identifying and repairing website navigation problems. Building on five earlier experiments [3,4], we first conducted two new experiments to create a sufficiently large dataset for multiple regression analysis. Then we devised automatable problem-identification rules and used multiple regression analysis on that large dataset to develop a new CWW formula for accurately predicting problem severity. We then conducted a third experiment to test the prediction formula and refined CWW against an independent dataset, resulting in full cross-validation of the formula. We conclude that CWW has high psychological validity, because CWW gives us (a) accurate measures of problem severity, (b) high success rates for repairs of identified problems (c) high hit rates and low false alarms for identifying problems, and (d) high rates of correct rejections and low rates of misses for identifying non-problems.

A Process Model for Missionaries--Cannibals and Other River-Crossing Problems.

Abstract We extend a model originally developed by Atwood and Polson (1976) for the water jug task to four isomorphs of the Missionaries-Cannibals problem. Our results show that variation in cover story produced no differences in number of legal moves to solution, but caused large differences in illegal moves. A three-stage process model incorporating means-ends heuristics, assumptions about the utilization of memory, and an illegal move-detection process is able to account for both legal and illegal move data from all four versions of the problem.

A comprehension-based model of correct performance and errors in skilled, display-based, human-computer interaction

This paper describes a computational model of skilled use of an application with a graphical user interface. The model provides a principled explanation of action slips, errors made by experienced users. The model is based on Hutchins, Holland and Normans analysis of direct manipulation and is implemented using Kintsch and Manness construction-integration theory of action planning. The model attends to a limited number of objects on the screen and then selects action on one of them, such as moving mouse cursor, clicking mouse button, typing letters, and so on, by integrating information from various sources. These sources include the display, task goals, expected display states, and knowledge about the interface and the application domain. The model simulates a graph drawing task. In addition, we describe how the model makes errors even when it is provided with the knowledge sufficient to generate correct actions.

A comprehension-based model of exploration

The linked model of comprehension-based action planning and instruction taking (LICAI) simulates performing by exploration tasks using applications hosted on systems with graphical user interfaces. The tasks are given to the user as written exercises containing no information about the correct action sequences. LICAIs comprehension and action-planning processes are based on Kintschs construction-integration (C-I) theory for text comprehension. The model assumes that comprehending instructions is a strategic process; instruction texts must be elaborated using specialized strategres that guide goal generation. LICAI comprehends the instructions and generates goals that are then stored in memory. The action-planning processes are controlled by goals retrieved from memory. Representations of goals that can guide exploration are restricted by the C-I architecture. The model predicts that successful exploration requires linking of the goal representation with the label on the correct object. The model is evaluated by comparing its predictions with results from an experimental study of learning by exploration by Franzke (1994, 1995). We discuss the implications of LICAI for designing instruction materials and interfaces that facilitate exploration.