Judith Ramey

Thinking aloud: reconciling theory and practice

Thinking-aloud protocols may be the most widely used method in usability testing, but the descriptions of this practice in the usability literature and the work habits of practitioners do not conform to the theoretical basis most often cited for it: K.A. Ericsson and H.A. Simons (1984) seminal work. After reviewing Ericsson and Simons theoretical basis for thinking aloud, we review the ways in which actual usability practice diverges from this model. We then explore the concept of speech genre as an alternative theoretical framework. We first consider uses of this new framework that are consistent with Ericsson and Simons goal of eliciting a verbal report that is as undirected, undisturbed and constant as possible. We then go on to consider how the proposed new approach might handle problems that arise in usability testing that appear to require interventions not supported in the older model.

A selected bibliography: a beginner's guide to usability testing

Many people who are interested in learning about usability testing have trouble finding an entry point into the literature in the field. This briefly annotated bibliography addresses their needs by identifying recent, reasonably accessible articles and books organized in the following categories: general introduction to usability testing; usability testing of documentation (the process and general methods and tools); print documentation (discussions and examples of tests); and online documentation (discussions and examples of tests); the human-machine interface (the process, general methods and tools, and discussions and examples of tests); and textbooks on research design. >

A framework for understanding mobile internet motivations and behaviors

Why do people access information via the mobile Internet? This qualitative study examines a group of active users and proposes a new preliminary framework for understanding their motivations and behaviors.

Does think aloud work?: how do we know?

The think aloud method is widely used in usability research to collect users reports of the experience of interacting with a design so that usability evaluators can find the underlying usability problems. However, concerns remain about the validity and usefulness of think aloud in usability studies. In this panel we will present current studies of the think aloud method, examine and question its usage in the field, discuss the possible pitfalls that may threaten the validity of the method, and provide comments/suggestions on the application of the method. Panel participants will discuss results drawn from both applied research and basic research.We believe that this panel discussion will be useful for HCI designers and usability practitioners in that it will acquaint them with concerns that people have about the think aloud method and provide them with suggestions for improved use of the method. For HCI or usability researchers, this panel discussion will address the importance of formally investigating currently used or newly designed usability methods.

Scenarios, personas and user stories: User-centered evidence-based design representations of communicable disease investigations

PURPOSE Despite years of effort and millions of dollars spent to create unified electronic communicable disease reporting systems, the goal remains elusive. A major barrier has been a lack of understanding by system designers of communicable disease (CD) work and the public health workers who perform this work. This study reports on the application of user-centered design representations, traditionally used for improving interface design, to translate the complex CD work identified through ethnographic studies to guide designers and developers of CD systems. The purpose of this work is to: (1) better understand public health practitioners and their information workflow with respect to CD monitoring and control at a local health agency, and (2) to develop evidence-based design representations that model this CD work to inform the design of future disease surveillance systems. METHODS We performed extensive onsite semi-structured interviews, targeted work shadowing and a focus group to characterize local health agency CD workflow. Informed by principles of design ethnography and user-centered design we created persona, scenarios and user stories to accurately represent the user to system designers. RESULTS We sought to convey to designers the key findings from ethnographic studies: (1) public health CD work is mobile and episodic, in contrast to current CD reporting systems, which are stationary and fixed, (2) health agency efforts are focused on CD investigation and response rather than reporting and (3) current CD information systems must conform to public health workflow to ensure their usefulness. In an effort to illustrate our findings to designers, we developed three contemporary design-support representations: persona, scenario, and user story. CONCLUSIONS Through application of user-centered design principles, we were able to create design representations that illustrate complex public health communicable disease workflow and key user characteristics to inform the design of CD information systems for public health.

Always On: A Framework for Understanding Personal Mobile Web Motivations, Behaviors, and Contexts of Use

Mobile data services offer a growing alternative means of accessing the Web and have drawn significant attention from the mobile industry. However, design efforts are hampered people’s motivations, behaviors, and contexts of use when they access the Web on their phones. To help address this need, we conducted a study to explore the following questions for U.S. mobile phone users: 1) What motivations lead people to access the Web on their mobile phones?; 2) What do they do?; and 3) Where do they do it? Based on the findings from Part One of the study, we constructed a taxonomy of behaviors, motivations, and contexts associated with mobile Web usage. In Parts Two and Three, we validated the taxonomy as well as compared iPhone versus non-iPhone user behaviors. We conclude this report by considering the design implications of our findings and future research directions.

The systematic literature review as a research genre

The “systematic literature review” as a research genre was first formulated in the field of medicine; the basic approach has since been adapted to serve the differing needs of a wide range of disciplines. The systematic literature review was intended to improve the synthesis of research by introducing a systematic, transparent, and reproducible literature-review process. This paper first characterizes the systematic literature review as practiced in medicine and as modified for use in other disciplines. Then, drawing on the example of a systematic literature review conducted by the authors, it explains and illustrates the entire systematic-literature-review process: the development of a review protocol, selection of databases and creation of a set of search terms, definition of inclusion/exclusion criteria, characterization of the corpus, and synthesis of the findings. The paper closes with an evaluation of the methods strengths and weaknesses in the context of our fields characteristics.

The design response to usability test findings: a case study based on artifacts and interviews

Usability testing during product development has gained wide acceptance as a strategy for improving product quality, but it is difficult to find published studies that actually document the impact of usability test findings and recommendations on product design. This case study examines a usability test that was done during the development of a software feature and is based on the usability test artifacts and interviews with people who were involved in designing, developing and implementing the software feature. We attempt to determine if and how the recommended changes from the usability test were implemented, to identify other changes made to the feature being designed, and to determine the source of those other changes. Based on our findings in this case study, we can say that the usability test findings and recommendations had a great impact on the design of the feature and the design was also influenced by seven other sources.

Use of mobile phones by non-literate and semi-literate people: A systematic literature review

This paper reports on a systematic literature review conducted to discover the state of research concerning the use of mobile phones by non-/semi-literate users. This is an important and growing area of research because mobile phone users in the developing world represent 73% of global subscriptions, and, of the 796 million people around the world who cannot read or write, 98% live in the developing countries. Mobile phones offer low-literate people critical access to information and services. By understanding how these people currently use mobile phones, we can develop better ways to meet their needs. To learn what research has been done in this area, we conducted a systematic literature review to identify good-quality, evidence-based studies that included design recommendations. This paper will describe the detailed selection process and characterize the corpus articles along a number of dimensions: type of use, locale of the study, domain, research methodology, and design recommendations. Then the paper will present a thematic analysis of the usages identified in those papers and discuss the related design recommendations.

A self-reporting methodology for rapid data analysis in usability testing

The author describes a methodology for collecting data in usability testing that asks subjects of a test to do structured self-reporting using a questionnaire in an iterative process. The subjects fill out one copy of the questionnaire after each of the tasks that comprise the usability test. The author discusses the problems in usability testing that motivated the design of this methodology, the methodology itself, its advantages and disadvantages, the kinds of data that it is best suited to collect, and how to design a test that uses it.<<ETX>>