Robert W. Root

Questionnaires as a software evaluation tool

This paper reports on a study investigating the strengths and weaknesses of questionnaires as software evaluation tools. Two major influences on the usefulness of questionnaire-based evaluation responses are examined: the administration of the questionnaire, and the background and experience of the respondent. Two questionnaires were administered to a large number of students in an introductory programming class. The questionnaires were also given to a group of more experienced users (including course proctors). Respondents were asked to evaluate the text editor used in the class along a number of dimensions; evaluation responses were solicited using a number of different question types. Another group of students received the questionnaire individually, with part of it presented on the computer; a third group also evaluated an enhanced version of the editor in followup sessions.

A Participatory Design Technique for High-Level Task Analysis, Critique, and Redesign: The CARD Method

CARD (Collaborative Analysis of Requirements and Design) is a participatory technique for analyzing task flows and for redesigning task flows, in software systems. It provides a macroscopic complement to the more microscopic design activities that are supported by the PICTIVE technique. CARD uses the metaphor of a card game as the vehicle for communication and collaboration among users, developers, and designers. We describe the technique, and provide illustrative session protocols and assessment data. The paper closes with a comparison to other relevant participatory practices, and a discussion of CARDs shortcomings.

Teaching experienced developers to design graphical user interfaces

Five groups of developers with experience in the design of character-based user interfaces were taught graphical user interface design through a short workshop with a focus on practical design exercises using low-tech tools derived from the PICTIVE method. Several usability problems were found in the designs by applying the heuristic evaluation method, and feedback on these problems constituted a way to make the otherwise abstract usability principles concrete for the designers at the workshop. Based on these usability problems and on observations of the design process, we conclude that object-oriented interactions are especially hard to design and that the developers were influenced by the graphical interfaces of personal computers with which they had interacted as regular users.

Comparing studies that compare usability assessment methods: an unsuccessful search for stable criteria

Four studies that compared inspection methods with usability testing were re-analyd using six distinct criteria for the superiority of one mefhcd to another. Each study’s own results were found —toa greater or lesser extent — to be in infernuf conflict when examined across the six criteria. These analyses, added to the well-known contradictions across the studies, argue that any conclusions regarding overall superiority of one method with respect to another are premalure. I%ey also lead to questions regarding tie selection of criteria.

Assessing a groupware implementation of a manual participatory design process

Our attempt to implement a groupware version of a manual participatory design process (Muller, Miller, Smith, White, and Wildman, 1992) has revealed several constmints that may apply to other groupware systems for collaboration — especially those that involve skills from outside the computer domain. The Manual PICTIVE Process. PICTIVE1 is a deliberately low-tech approach to the participatory design of systems (Muller, 1991, 1992). It was developed to bring users, developers, human factors workers, and other stakeholders in a product, directly into the design process as peer co-designers (e.g., Mtdler, 1991). PICTIVE uses a simple objec[ model of office tools, such as colored pens, Post-Itm notes, scissors, and so on, slightly customized for the intended computer domain. This facilitates the PICTIVE parficipafion model: Diverse stakeholders need a common language (Dykstra, 1991) or “equal opportunity” design environment (Mttller, 1992) for which (a) their skills are roughly equal within the environment, and (b) their domain knowledge can be easily expressed through the environment.

Bellcore's user-centred-design support centre

Abstract Bellcore recently replaced its small laboratory that was designed primarily for formal testing of software usability. The new facility is a suite of rooms that handles multiple, independent activities. More importantly, the new space is a manifestation of our philosophy that the best approach to interface design is the cultivation of eclectic design practices early in and throughout the software development process. To that end, the new lab supports other kinds of user-centred design (UCD) activities in addition to formal testing of computerized prototypes of software interfaces. To encourage participatory design, nearly all the rooms are large enough for design meetings, contain entire walls of movable whiteboards, and have small tables so design teams can huddle over paper prototypes and task layouts. In this article we describe the new lab, the rationales behind its features, and the process by which it was designed.