Nigel Bevan

Usability measurement in context

Abstract Different approaches to the measurement of usability are reviewed and related to definitions of usability in international standards. It is concluded that reliable measures of overall usability can only be obtained by assessing the effectiveness, efficiency and satisfaction with which representative users carry out representative tasks in representative environments.This requires a detailed understanding of the context of use of a product. The ESPRIT MUSiC project has developed tools which can be used to measure usability in the laboratory and the field. An overview is given of the methods and tools for measuring user performance, cognitive workload, and user perceived quality.

Measuring usability as quality of use

The conventional assumption that quality is an attribute of a product is misleading, as the attributes required for quality will depend on how the product is used. Quality of use is therefore defined as the extent to which a product satisfies stated and implied needs when used under stated conditions. Quality of use can be used to measure usability as the extent to which specific goals can be achieved with effectiveness, efficiency and satisfaction by specified users carrying out specified tasks in specified environments. Practical and reliable methods of measuring quality of use have been developed by the MUSiC project. These provide criteria for usability which can be incorporated into a quality system. A description is given of the MUSiC methods for specifying the context of use and measuring effectiveness, efficiency, and satisfaction.

International standards for HCI and usability

Over the last 15 years, a comprehensive range of international standards has been developed to define the general principles of user-centred design and good practice in user interface design. Most of the standards specify general principles rather than the precise details of the interface. The paper briefly describes how standards are created and reviews the definitions of usability. HCI and usability standards are described in the categories: usability definitions, use in context, software interface and interaction, hardware interface, documentation, the development process and capability of the organization. The applicability of the standards is discussed.

Quality in use: Meeting user needs for quality

There is an increasing demand for software that matches real user needs in a working environment. The paper describes the new framework for software product quality developed for ISO/IEC 9126-1: internal quality (static properties of the code), external quality (behaviour of the software when it is executed) and quality in use (the extent to which the software meets the needs of the user). Quality in use is a broader view of the ergonomic concept of usability in ISO 9241-11. Achieving quality in use requires a user-centred design process which has cultural, strategic and technical implications.

Usability is Quality of Use

Publisher Summary This chapter discusses the distinction between broad and narrow approaches to usability and identifies the broad approach to usability with the higher level quality objective of quality of use. It is suggested that quality of use should be the major design objective for an interactive product. This relates usability to business objectives and elevates usability from an optional extra to the prime design goal. The narrow approach is complementary and is concerned with the design of features of the product that are a pre-requisite for quality of use. The two different interpretations of usability lead to two approaches to the specification and evaluation of usability. The chapter describes that user-based evaluation can be used to validate achievement of these requirements. Usability attributes provide a contribution to achieving quality of use. The presence or absence of these attributes can be verified early in design. In addition, frequent user-based evaluation of early mock-ups and prototypes is required to give feedback on the quality of use of potential solutions.

The MUSiC performance measurement method

Abstract This paper reports a method for measuring usability in terms of task performance-achievement of frequent and critical task goals by particular users in a context simulating the work environment. The terms usability and quality in use are defined in international standards as the effectiveness, efficiency and satisfaction with which goals are achieved in a specific context of use. The performance measurement method gives measures which, in combination with measures of satisfaction, operationalize these definitions. User performance is specified and assessed by measures including task effectiveness (the quantity and quality of task performance) and User efficiency (effectiveness divided by tasktime). Measures are obtained with users performing tasks in a context of evaluation which matches the intended context of use. This can also reveal usability problems which may not become evident if the evaluator interacts with the user. The method is supported by tools which make it practical in commercial t...

Extending Quality in Use to Provide a Framework for Usability Measurement

ISO has recently developed a new more comprehensive definition of quality in use, which has usability, flexibility and safety as subcharacteristics that can be quantified from the perspectives of different stakeholders, including users, managers and maintainers. While this provides a more complete set of requirements for operational use of a product, it also presents new challenges for measurement.

The improvement of human-centred processes-facing the challenge and reaping the benefit of ISO 13407

Human-centred design processes for interactive systems are defined in ISO 13407 and the associated ISO TR 18529. The publication of these standards represents a maturing of the discipline of user-centred design. The systems development community see that (at last) Human Factors has processes which can be managed and integrated with existing project processes. This internationally agreed set of human-centred design processes provides a definition of the capability that an organization must possess in order to implement user-centred design effectively. It can also be used to assess the extent to which a particular development project employs user-centred design. As such, it presents a challenge to the Human Factors community, and indeed a definition of good practice may even be regarded by some as an unwelcome constraint. This paper presents the background to the process-level definition of user-centred design and describes how it relates to current practice. The challenges, benefits and use of a defined human-centred design process are presented. The implications for Human Factors and other disciplines are discussed. In Appendices A?D, the process terminology and the contents of ISO 13407 and ISO TR 18529 are described in more detail, and three examples are given (in Appendix D) of using this process improvement approach to improve the actual design methods in three organizations.

Human-Computer Interaction Standards

Publisher Summary This chapter presents human-computer interaction (HCI) standards. Standard user interfaces provide the benefit of consistency; they become out of date as technology changes, and are usually only appropriate for limited types of users and tasks. Thus most work on international standards for HCI has not been about precise specification, but instead has concentrated on the principles that need to be applied in order to produce an interface, which meets user and task needs. These standards broadly fall into two categories—one is a top-down approach, which is concerned with usability as a broad quality objective: the ability to use a product for its intended purpose. The other is a product-oriented bottom-up view, which is concerned with aspects of the interface that make a system easier to use. The broad quality view originates from human factors, and standards of this type are applicable in the broad context of design and quality objectives. The product-oriented view concentrates on the design of specific attributes, and relates more closely to the needs of the interface designer and the role of usability in software engineering.

The "magic number 5": is it enough for web testing?

Common practice holds that 80% of usability findings are discovered after five participants. Recent findings from web testing indicate that a much larger number of participants is required to get results and that independent teams testing the same web-based product do not replicate results. How many users are enough for web testing?