Juergen Sauer

The influence of design aesthetics in usability testing: Effects on user performance and perceived usability

This article examined the effects of product aesthetics on several outcome variables in usability tests. Employing a computer simulation of a mobile phone, 60 adolescents (14-17 yrs) were asked to complete a number of typical tasks of mobile phone users. Two functionally identical mobile phones were manipulated with regard to their visual appearance (highly appealing vs not appealing) to determine the influence of appearance on perceived usability, performance measures and perceived attractiveness. The results showed that participants using the highly appealing phone rated their appliance as being more usable than participants operating the unappealing model. Furthermore, the visual appearance of the phone had a positive effect on performance, leading to reduced task completion times for the attractive model. The study discusses the implications for the use of adolescents in ergonomic research.

The influence of prototype fidelity and aesthetics of design in usability tests: Effects on user behaviour, subjective evaluation and emotion

An empirical study examined the impact of prototype fidelity on user behaviour, subjective user evaluation and emotion. The independent factors of prototype fidelity (paper prototype, computer prototype, fully operational appliance) and aesthetics of design (high vs. moderate) were varied in a between-subjects design. The 60 participants of the experiment were asked to complete two typical tasks of mobile phone usage: sending a text message and suppressing a phone number. Both performance data and a number of subjective measures were recorded. The results suggested that task completion time may be overestimated when a computer prototype is being used. Furthermore, users appeared to compensate for deficiencies in aesthetic design by overrating the aesthetic qualities of reduced fidelity prototypes. Finally, user emotions were more positively affected by the operation of the more attractive mobile phone than by the less appealing one.

Effects of training on short- and long-term skill retention in a complex multiple-task environment.

The paper reports the results of an experiment on the performance and retention of a complex task. This was a computer-based simulation of the essential elements of a spacecrafts life support system. It allowed the authors to take a range of measures, including primary and secondary task performance, system intervention and information sampling strategies, mental model structure, and subjective operator state. The study compared the effectiveness of two methods of training, based on low level (procedure-based) and high level (system-based) understanding. Twenty-five participants were trained extensively on the task, then given a 1-h testing session. A second testing session was carried out 8 months after the first (with no intervening practice) with 17 of the original participants. While training had little effect on control performance, there were considerable effects on system management strategies, as well as in structure of operators mental model. In the second testing session, the anticipated general performance decrement did not occur, though for complex faults there was an increase in selectivity towards the primary control task. The relevance of the findings for training and skill retention in real work environments is discussed in the context of a model of compensatory control.

A conceptual framework for designing micro-worlds for complex work domains : a case study of the Cabin Air Management System

A conceptual framework for the design of micro-worlds is presented in this paper. This is illustrated by a micro-world, called Cabin Air Management System (CAMS), that has been used in a research program to investigate human behavior during work with complex systems. Although CAMS is based on the operational context of spaceflight, its underlying principles correspond to a process control task. The authors propose a theoretical framework for micro-world design, which was used to guide the development of the CAMS micro-world. This may provide a helpful guideline for researchers wishing to develop a micro-world by using a more structured approach. A detailed technical description of CAMS is given to demonstrate the kind of real-world scenarios that may be simulated with this task environment. Finally, general aspects of the utility of micro-world research are discussed.

Effects of display design on performance in a simulated ship navigation environment

Although automation is playing an increasing role on the ships bridge, empirical research on the effectiveness of alternative bridge designs is limited. In this paper, we describe an experimental study of the benefits of integrated information display, using a computerized simulation of a highly automated ships bridge. The study compared three types of interface design, which presented radar and electronic chart information to the operator in different ways: (a) integrated display, (b) functionally-separate display, and (c) spatially-separate display. Effects were examined in relation to time on watch and scenario complexity. Following extensive training on the task, 39 participants were tested over a 4-h experimental session, during which they encountered a sequence of collision scenarios of varying complexity. Using a dual-task methodology, a range of measures of primary and secondary task performance were taken, together with assessment of information sampling behaviour and subjective operator state (workload, fatigue, anxiety and situation awareness). The results indicated slight navigational advantages of the integrated display over the two alternative display types, although it also incurred higher levels of operator cost, particularly fatigue. There were no marked effects of time on watch, but more complex scenarios were associated with impaired performance, increased workload and reduced situation awareness. Overall, the findings have suggested some benefits of integrating primary information sources in a ships bridge environment. The study further confirms the value of experimental simulations as tools for investigating design issues for ships bridge automation.

The influence of product aesthetics and usability over the course of time: a longitudinal field experiment

A longitudinal field experiment was carried out over a period of 2 weeks to examine the influence of product aesthetics and inherent product usability. A 2 × 2 × 3 mixed design was used in the study, with product aesthetics (high/low) and usability (high/low) being manipulated as between-subjects variables and exposure time as a repeated-measures variable (three levels). A sample of 60 mobile phone users was tested during a multiple-session usability test. A range of outcome variables was measured, including performance, perceived usability, perceived aesthetics and emotion. A major finding was that the positive effect of an aesthetically appealing product on perceived usability, reported in many previous studies, began to wane with increasing exposure time. The data provided similar evidence for emotion, which also showed changes as a function of exposure time. The study has methodological implications for the future design of usability tests, notably suggesting the need for longitudinal approaches in usability research. Practitioner Summary: This study indicates that product aesthetics influences perceived usability considerably in one-off usability tests but this influence wanes over time. When completing a usability test it is therefore advisable to adopt a longitudinal multiple-session approach to reduce the possibly undesirable influence of aesthetics on usability ratings.

Designing automation for complex work environments under different levels of stress

This article examines the effectiveness of different forms of static and adaptable automation under low- and high-stress conditions. Forty participants were randomly assigned to one of four experimental conditions, comparing three levels of static automation (low, medium and high) and one level of adaptable automation, with the environmental stressor (noise) being varied as a within-subjects variable. Participants were trained for 4 h on a simulation of a process control environment, called AutoCAMS, followed by a 2.5-h testing session. Measures of performance, psychophysiology and subjective reactions were taken. The results showed that operators preferred higher levels of automation under noise than under quiet conditions. A number of parameters indicated negative effects of noise exposure, such as performance impairments, physiological stress reactions and higher mental workload. It also emerged that adaptable automation provided advantages over low and intermediate static automation, with regard to mental workload, effort expenditure and diagnostic performance. The article concludes that for the design of automation a wider range of operational scenarios reflecting adverse as well as ideal working conditions needs to be considered.

The influence of laboratory set-up in usability tests: effects on user performance, subjective ratings and physiological measures

This article examines the influences of situational factors on user behaviour in usability tests. Sixty participants carried out two tasks on a computer-simulated prototype of a mobile phone. Employing a 3 × 2 mixed experimental design, laboratory set-up was varied as a between-subjects variable (presence of facilitator and two non-interactive observers, presence of facilitator or no person present) while task difficulty was manipulated as a within-subjects variable (low vs. high). Performance data, subjective measures and physiological parameters (e.g. heart rate variability) were taken. The results showed that the presence of non-interactive observers during a usability test led to a physiological stress response, decreased performance on some measures and affected the emotional state of test participants. The presence of a facilitator (i.e. a participating observer) also influenced the emotional state of the test participant. Practitioners involved in usability testing need to be aware of undue influences of observers, in particular, if the observers are non-interactive. The findings presented in this paper have implications for the practice of usability testing. They indicated a considerable influence of observers on test participants (physiology and emotions) and on the outcomes of usability tests (performance measures). This should be considered when selecting the set-up of a usability testing procedure.

Designing for the home: a comparative study of support aids for central heating systems.

The study examined the influence of different types of enhanced system support on user performance during the management of a central heating system. A computer-based simulation of a central heating system, called CHESS V2.0, was used to model different interface options, providing different support facilities to the user (e.g., historical, predictive, and instructional displays). Seventy-five participants took part in the study and completed a series of operational scenarios under different support conditions. The simulation environment allowed the collection of performance measures (e.g., energy consumption), information sampling, and system control behaviour. Subjective user evaluations of various aspects of the system were also measured. The results showed performance gains for predictive displays whereas no such benefits were observed for the other display types. The data also revealed that status and predictive displays were valued most highly by users. The implications of the findings for designers of central heating systems are discussed.

Feedback quality and environmentally friendly use of domestic central heating systems

The study examined the influence of system-embedded feedback on user behaviour during the environmentally friendly operation of a central heating system. A PC-based simulation, called CHESS, was developed to model the critical features of a central heating system. After having received 30 min of training on the simulation task, 60 participants worked on a series of operational scenarios under different levels of system feedback. In addition to the collection of various performance measures (e.g. energy consumption, energy wastage), a range of user variables was collected (e.g. environmental concern). As hypothesized, the results showed that increased feedback resulted in improved environmentally friendly performance and, more importantly, the specific feedback indicator influenced the type of strategy used to improve human – machine system performance. A major implication is that system designers need to develop feedback indicators that are chiefly influenced by user behaviour and are largely immune to factors that are beyond the users control (e.g. weather conditions).