Rune Thaarup Høegh

Is It Worth the Hassle? Exploring the Added Value of Evaluating the Usability of Context-Aware Mobile Systems in the Field

Evaluating the usability of mobile systems raises new concerns and questions, challenging methods for both lab and field evaluations. A recent literature study showed that most mobile HCI research projects apply lab-based evaluations. Nevertheless, several researchers argue in favour of field evaluations as mobile systems are highly context-dependent. However, field-based usability studies are difficult to conduct, time consuming and the added value is unknown. Contributing to this discussion, this paper compares the results produced by a laboratory- and a field-based evaluation of the same context-aware mobile system on their ability to identify usability problems. Six test subjects used the mobile system in a laboratory while another six used the system in the field. The results show that the added value of conducting usability evaluations in the field is very little and that recreating central aspects of the use context in a laboratory setting enables the identification of the same usability problem list.

Evaluating usability: using models of argumentation to improve persuasiveness of usability feedback

Usability evaluation is widely accepted as a valuable activity in software development. However, how results effectively are fed back to developers is still a relatively unexplored area. We argue that usability feedback can be understood as an argument for a series of usability problems, and that basic concepts from argumentation theory can help us understand how to create persuasive feedback. We revisit two field studies on usability feedback to study if concepts from Toulmins model for argumentation and Aristotles modes of persuasion can explain why some feedback formats outperform others. We recommend that evaluators specifically back up the warrants behind their usability claims, that their arguments use several modes of persuasion, and that they present feedback in browsable amounts not to overwhelm developers with information. For complex and controversial problems, we advise evaluators to involve developers in a learning process and provide the opportunity to experience and discuss the findings.

Usability problems: do software developers already know?

The result of usability evaluations is often accentuated as a distinctive input for developers to improve the usability of a software system. On the other hand developers say that many of the results from the usability evaluations are issues already known to them. This paper presents a study of usability problems as developers perceive them in their own emerging software in relation to usability problems experienced by users in a usability evaluation. The results indicate that having developers explicating their expectation on emerging software can provide a low-cost identification of problem areas, whereas a full scale usability evaluation provides specific knowledge of usability problems and their severity.

Software Development and Feedback from Usability Evaluations

This paper presents a study of the strengths and weaknesses of written, multimedia and oral feedback from usability evaluations to developers. The strengths and weaknesses are related to how well the feedback supports the developers in addressing usability problems in a software system. The study concludes that using the traditional written usability report, as the only form of feedback from usability evaluations is associated with problems related to the report not supporting the process of addressing the usability problems. The report is criticized for representing an overwhelming amount of information, while still not offering the required information to address usability problems. Other forms of feedback, such as oral or multimedia feedback helps the developer in understanding the usability problems better, but are on the other hand less cost-effective than a written description.