Stephen J. Payne

Algebra Mal‐Rules and Cognitive Accounts of Error

On page 457, Table 7 entitled “Distribution of Mal-Rules Across Question Types” contained two misrepresented symbols within the table. Thus, in the line adjacent to number 4, under the table column heads Ml2 and M13, the plus symbols after the number 2 digits should have been superscript swords. The footnote to this table was also in error due to an asterisk which should have been a superscript sword. The complete and corrected table is reprinted on p. 642.

Display-based action at the user interface

This paper examines the hypothesis that information flow, from device to user, is a vital part of skilled activity in human-computer interaction. Two studies are reported. The first study questions users of keyboard-driven word processors about the effects of cursor-movement, finding and word-deletion commands in various contexts. The second study questions users of the Apple Macintosh-based systems, MacWrite and Microsoft Word, about the behaviour of the menu-driven find command. In both studies it is discovered that users often do not know the precise effects of frequently-used actions, such as the final position of the cursor, even though these effects are vital for future planning. It is concluded that even experienced users must acquire the information they need from the devices display during interactions, and that they do not necessarily remember regular details that are available in this way. This conclusion conflicts with those current models of user psychology that assume routine skill relies on complete mental specifications of methods for performing tasks.

Understanding calendar use

This article is an interview study of calendar use and a cognitive analysis of the interactions between the design of calendars and the task of prospective remembering. The study and analysis are coordinated to present a general critique of current electronic calendar designs and to note opportunities for future designs. The interview data reveal continued use of paper calendars in a highly computerized setting. A key conclusion is that paper calendars support prospective remembering by promoting browsing of existing appointments during subsequent calendar keeping but that this advantage is compromised in current electronic designs. Other aspects of the interviews and the analyses address the representational limitations of both paper and electronic calendars. This research explores a methodology in which design implications of qualitative empirical data are understood and systematized through theoretical analyses of existing artifacts.

Display-based competence: towards user models for menu-driven interfaces

This paper discusses the critical role played by aspects of the display in the use of many computer systems especially those driven by menus. We outline a formal model of “display-based competence” by extending the Task-Action Grammar notation ( Payne & Green, 1986 ). The model, D-TAG (for display-orinted task-action grammar) is illustrated with examples for the well-known Macintosh desk-top interface, and from a more deeply-nested menu interface to a device used for the remote testing of telephone line (RATES). D-TAG exploits two extensions of TAG to address important aspects of interface consistency. The most important extension uses a featural description of the display to capture the role of the display in structuring task-action mappings; the second describes the “side-effects” of a task, i.e. those effects not described by the semantic attributes of a task. By embedding these extensions within the organizing framework of TAGs feature-grammar, we are able to develop descriptions of interfaces which highlight aspects of (display) design that are outside the scope of other formal user models.

The nature of device models: the yoked state space hypothesis and some experiments with text editors

To construct a conceptual model of a device, the user must conceptualize the devices representation of the task domain. This knowledge can be represented by three components: a device-based problem space, which specifies the ontology of the device in terms of the objects that can be manipulated and their interrelations, plus the operators that perform the manipulations; a goal space, which represents the objects in terms of which users goals are expressed; and a semantic mapping, which determines how goal space objects are represented in the device space. The yoked state space (YSS) model allows an important distinction concerning the mental representation of procedures. If a step in a procedure specifies a transformation of the users device space, then it has an autonomous meaning for the user, independent of its role in the sequence or method. The device space provides a figurative account of the operator. However, some operators do not affect the minimal device space, and their only meaning for the user derives from their role in a method: The method affords an operational account of the operator. Figurative accounts can be constructed from operational accounts only by elaborating the device space with new concepts. The YSS is illustrated through a simple description of a device model for a cut-and-paste text editor. Three experiments addressed the claims of this model. The first experiment used a sorting paradigm to show that users do acquire the novel device space concept of a string of adjacent characters (including space and return). The second and third experiments asked novices to make inferences about text editor behavior on the basis of simple demonstrations. They showed that (a) the availability of the string concept is critically dependent on the details of interface design, (b) figurative accounts of the copy operation afford more efficient methods and may be promoted by appropriate names for procedure steps, and (c) a conceptual model may transfer from one device to another. Together, the three experiments supported the YSS hypothsis.

Reflection and goal management in exploratory learning

Abstract We report two experiments which examine the nature of exploratory learning of interactive devices. We argue that the success of exploratory learning is dependent on the degree to which learners reflect on their interactions, and on how well they manage their goals. In Experiment 1, a keystroke limit was imposed on subjects, i.e. a limit on the amount of physical interaction with the device. In addition some subjects were provided with a list of goals in order to help them manage their exploration. As predicted, the first intervention resulted in more successful learning as compared to the performance of subjects who explored without any constraints. Experimenter-provided goals also yielded some benefits, but had a much less strong effect than the keystroke limit. Experiment 2 confirmed and extended the main findings of the first experiment. Farther, having noted that subjects preferred to switch opportunistically from goal to goal and mode to mode we predicted that limiting them to exploring one part of the device at a time would result in better learning. We found that imposing a keystroke limit or forcing subjects to explore one mode at a time led to large and significant improvements in exploratory learning. The goal list manipulation also apparently improved exploration, but to a lesser degree than the other two manipulations.

Problem Difficulty and Response Format in Syllogistic Reasoning

It was hypothesized that the perceived irrelevance of the proposition “Some X are not Y” is a factor contributing to the difficulty of nearly all the determinate syllogisms classed as multiple model by Johnson-Laird and Byrne (1991), according to mental models theory. Experiment 1 supported this hypothesis by showing that subjects frequently correctly evaluate valid “Some … not” conclusions but rarely produce them, even when they have evaluated them elsewhere. Explanations of these findings based on the use of superficial strategies were ruled out. Experiment 2 further supported the hypothesis by showing that performance increased across the no-conclusion, multiple-choice, and evaluation task formats, and that this effect generalized to problems containing the quantifier “only”. However, the initial hypothesis was rejected in light of Experiment 3, which found no difference between multiple-choice and no-conclusion formats when the number of allowable conclusions was controlled for. Nevertheless, superior performance remained in the evaluation format, and it is suggested that offered conclusions may be used as a goal for the reasoning process. This interpretation is supported by the finding (Experiments 1 and 3) that subjects appear to search only for alternative conclusions that maintain the subject-predicate structure of the offered conclusion.

Animated demonstrations for exploratory learners

In an animated demonstration the devices display behaves as if the device were in use. We investigate the instructional potential of a ‘pure’ version of animated demonstrations, in which there is no commentary or supporting documentation for the animation. Such animations are heavily used in video games (such as PacMan), but are rarely seen in computer-based office systems, although modern designs allow them to be readily and meaningfully implemented. We report two experiments that test the efficacy of animated demonstrations as an aid to exploratory learning of the MacDraw graphics editor. The animated demonstration is simulated by playing a short, uncommented, silent video recording of the screen-in-use. Experiment 1 shows that this technique offers large learning advantages over a no-instruction condition, and is, in our limited experiment, as effective as text-based instruction. Experiment 2 replicates the main effect of Experiment 1, again revealing a significant positive effect of a short animated demonstration on first-time exploratory learners. A notable feature of uncommented animated demonstrations is that they do not offer a complete method specification for the performance of any tasks. In the face of this limitation, their success as instructions can perhaps best be understood in terms of psychological models of performance that do not demand complete cognitive encoding of plans or methods.

COMPLEX PROBLEM SPACES: MODELLING THE KNOWLEDGE NEEDED TO USE INTERACTIVE DEVICES

A model is presented of the way users construct device-oriented problem spaces. The core of the model is a complex, layered problem space, consisting of two state spaces, the goal space and the device space, and a semantic mapping betwen them. Structural redundancy is identified as a formal property of such problem spaces that predicts the learnability of devices. Within the framework, two modes of learning are described which lead to qualitatively different versions of the problem space: Operational accounts treat primitive interactions with the device as syntactic components of useful methods; Figurative accounts elaborate the conceptual model of the device to provide semantic interpretations of the primitive interactions.

Semantic analysis during exploratory learning

This paper addresses the problem of how a novice computer user, engaged in exploratory learning, accounts for the behaviour of the device. Exploratory learning is the norm for many users who encounter computers in the work place. Exploratory learners must acquire methods from a suboptimal stream of task directed behaviour and its observable effects. A candidate model of analysis, EXPL [9] is taken as the baseline for the development of a new model, called Explor, which employs semantic knowledge of the lexical items used in the interface to relate user actions to system responses. The strengths and weaknesses of Explor are illustrated and discussed.