David J. Duke

Systems, interactions, and macrotheory

A significant proportion of early HCI research was guided by one very clear vision: that the existing theory base in psychology and cognitive science could be developed to yield engineering tools for use in the interdisciplinary context of HCI design. While interface technologies and heuristic methods for behavioral evaluation have rapidly advanced in both capability and breadth of application, progress toward deeper theory has been modest, and some now believe it to be unnecessary. A case is presented for developing new forms of theory, based around generic “systems of interactors.” An overlapping, layered structure of macro- and microtheories could then serve an explanatory role, and could also bind together contributions from the different disciplines. Novel routes to formalizing and applying such theories provide a host of interesting and tractable problems for future basic research in HCI.

A Review of Formalisms for Describing Interactive Behaviour

This paper reviews the state of research linking formal specification and interactive systems. An appreciation of Human Computer Interaction has become increasingly important within Software Engineering. As systems have become more complex there is an increasing awareness of the consequences of human error. As a result the formal specification of interactive behaviour has become a pressing topic of research. The notations considered here describe both the capabilities and resources of users in relation to a specific system and those aspects of an interactive system that must be analysed from a user perspective before implementation. The review concludes by surveying ongoing work which attempts to bridge the gap between disciplinary standpoints.

Unifying views of interactors

Interactors are components in the description of an interactive system that encapsulate a state, the events that manipulate the state, and the means by which the state is made perceivable to users of the system (the presentation). This paper concerns the relationship between the models of interactors that are being developed, at York and Pisa, in the context of Esprit Basic Research Action 7040 (Amodeus-2). The models differ in their expression of the three components of an interactor, and after relating the models to the informal notion of interactor we describe the context in which the view of interaction afforded by each model is appropriate.

Systematic development of the human interface

The problem of developing software to meet precise specifications has lead to the development of mathematical notations for expressing and reasoning about the behaviour of a required or extant system. We describe a different use of formal models: as tools for gathering and consolidating requirements on interaction between engineered systems and their users. This change in focus reflects the growing use of sophisticated interactive technology in domains such as medicine, where human comfort or safety is an issue. Not only must software systems function correctly, but the demands that the interface places on users of those systems need to be understood. This problem cannot be addressed by formal models in isolation. Instead, we describe an approach that uses formal models of human information processing to augment models of system functions. As a result it becomes possible, at an early stage in system design, to consider the role of human cognition in the correct behaviour of the system.

The Hybrid World of Virtual Environments

Much of the work concerned with virtual environments has addressed the development of new rendering technologies or interaction techniques. As the technology matures and becomes adopted in a wider range of applications, there is, however, a need to better understand how this technology can be accommodated in software engineering practice. A particular challenge presented by virtual environments is the complexity of the interaction that is supported, and sometimes necessary, for a particular task. Methods such as finite‐state automata which are used to represent and design dialogue components for more conventional interfaces, e.g. using direct manipulation within a desktop model, do not seem to capture adequately the style of interaction that is afforded by richer input devices and graphical models. In this paper, we suggest that virtual environments are, fundamentally, what are known as hybrid systems. Building on this insight, we demonstrate how techniques developed for modelling hybrid systems can be used to represent and understand virtual interaction in a way that can be used in the specification and design phases of software development, and which have the potential to support prototyping and analysis of virtual interfaces.

A standard model for multimedia synchronization: PREMO synchronization objects

Abstract. This paper describes an event-based synchronization mechanism, which is at the core of the inter-media synchronization in the upcoming standard for multimedia presentation, PREMO. The synchronization mechanism of PREMO is a powerful tool, based on a small number of concepts, and on cooperation among active objects, and represents a synthesis of various synchronization models described in the literature. This model can serve as a basis for the implementation of complex synchronization patterns in multimedia presentations, both purely event-based, as well as time-based.

Reasoning About Gestural Interaction

Many of the reported developments in the design of virtual spaces or visualisation systems are based on improvements in technology, either physical devices or algorithms for achieving realistic renderings within real‐time constraints. While this experimental approach produces a wealth of empirical results, it operates largely without a sound underlying theory that can be used to design systems that will effectively support users in real‐world domains. One of the main problems is that these sophisticated technologies rely on, but rarely assess, the cognitive abilities of the user. This paper introduces a new approach to modelling human‐system interaction. A syndetic model combines a formal expression of system behaviour with an approximate representation of cognitive resources to allow reasoning about the flow and utilisation of information within the combined system. The power of the approach to provide insight into novel interaction techniques is illustrated by developing a syndetic model of a gesture‐driven user interface.

Rendering and Affect

Previous studies at the intersection between rendering and psychology have concentrated on issues such as realismand acuity. Although such results have been useful in informing development of realistic rendering techniques,studies have shown that the interpretation of images is influenced by factors that have little to do with realism. Inthis paper, we summarize a series of experiments, the most recent of which are reported in a separate paper, thatinvestigate affective (emotive) qualities of images. These demonstrate significant effects that can be utilized withininteractive graphics, particularly via non‐photorealistic rendering (NPR). We explain how the interpretation ofthese results requires a high‐level model of cognitive information processing, and use such a model to account forrecent empirical results on rendering and judgement.

Demonstrating the cognitive plausibility of interactive system specifications

Abstract. Much of the behaviour of an interactive system is determined by its user population. This paper describes how assumptions about the user can be brought into system models in order to reason about their behaviour. We describe a system model containing reasonable assumptions about the user as being ‘cognitively plausible’. Before asserting the plausibility of a model however we must first be able to make the assumptions made in that model inspectable.There is a tension between the inspectability of user assumptions and the tractability of models; inspectable models tend to not be very tractable and vice versa. We describe how we can get round this tension, by deriving tractable models from explicit user assumptions. The resulting models may not of themselves be very inspectable to human-factors workers, but the process by which they are derived is inspectable. Hence we claim that we can have both tractability and inspectability. We exemplify our claims using a simple cognitive model and ‘Meeting Maker’, an interactive electronic diary system.

Mapping user requirements to implementations

The trend towards systems that support a rich variety of interaction techniques has consequences for the traditional understanding of software design practice. Formal approaches to software development have mainly focused on design expressions that avoid explicit discussion of interactive behaviour. The emphasis in rigorous development is on transforming a mathematical specification into an implementation through a series of valid refinement steps. However, any design process for interactive systems must also add user oriented requirements. It is suggested that a suitable concern for human factors can be folded into a rigorous development process by generalising from functional models and refinement to models that provide multiple viewpoints onto the design artefacts. The paper provides an overview of the relevant perspectives and describes work aimed at integrating them within the specification phase of software development.