Joseph S. Dumas

Interacting with diverse populations

This chapter provides guidelines for interacting with a several populations, such as people with physical disabilities, the elderly, people with cognitive disabilities or low literacy skills, children and teens and people from cultures different from the moderators. Usability professionals, have the responsibility to design and evaluate products for the widest possible market. User interfaces should be intuitive for all their users, not just the computer-literate and able-bodied ones. Moderating usability tests with some populations, however, requires additional knowledge, training, and preparation. This chapter provides general guidelines, such as: treat each participant as an individual, ask for feedback, ask before you help, learn about the population, make instructions and explanations simple and clear, be sure participants understand and sign the informed consent form, and many more. The chapter provides a brief discussion on people with physical disabilities. A number of usability practitioners have extensive experience moderating tests with participants who have physical disabilities. There are some guidelines that a moderator must keep in mind while interacting, such as providing accessible facilities and restrooms and providing accessible snacks. When interacting with a blind participant, one must include questions such as the following: Will you have an escort? Do you need transportation to and from the facility? Do you need assistance getting from public transportation or from a taxicab? Will you bring a service animal? If so, does the animal have special needs? What is its name? Do you read Braille? Do you use assistive technology? That and many more concepts are explained in this chapter.

Getting started as a test moderator

This chapter provides some guidance to moderators who are just starting out. It discusses the attributes of a great moderator, types of testing, moderator (sometimes conflicting) roles, the basics of running a test, and finally some ways to get started quickly as a moderator. To be an effective moderator, one is required to have a firm understanding of what usability testing is all about. One is required to know the following: the purpose of the test that one is moderating (the purpose determines how much and when to interact with participants), and how usability testing differs from other evaluation methods. The emphasis here is on obtaining valid data from a small sample of typical users, and letting the users speak. Fundamental knowledge of usability testing is vital to understand the study material of this chapter. The study discusses the role of a moderator, as one of the things that makes moderating test sessions different from, say, conducting interviews or moderating focus groups is that one must simultaneously fill multiple roles. As a moderator, an individual must be unbiased and neutral with regard to the product, be in control of the session, and be open to and approachable by participants. Getting this mixture of neutrality and attachment right is the challenge of moderating effectively. There are primarily three roles of a moderator as follows: the gracious host, the leader, and the neutral observer.

Chapter 6 – Interacting during the session

This chapter covers the issues that moderators spend most of their time dealing with during the 60 to 90 minutes when participants are working on tasks. It focuses on common situations that occur most of the time, not unusual events that rarely occur. This chapter contains a set of “good practice” guidelines for the everyday situations, such as how much to interact, keeping them talking, probing for more information, providing encouragement, dealing with failure, providing assistance, and many more. There is further discussion on moderators, as they have their own styles of interacting. During diagnostic tests, effective moderators fall at both ends of the activity scale, which means that some interact a lot and some very little. There is no research on whether the amount of interaction matters to the validity of testing. When a moderator talks, he or she influences the results of the test in some way. Sometimes the intervention makes a test session more productive. Other times, it can compromise the validity of the data. Probes and questions should further the objectives of the test, such as by clarifying whats happening or revealing additional information. Also covered is how to provide assistance, as usability problems often reveal themselves sequentially as participants move through the steps in a task. A problem that occurs early in the sequence and stops progress can prevent the participant from uncovering additional problems that would come later. Providing assistance is a way to move past one step so that later problems might be uncovered.

Chapter 4 – Golden rules 6 through 10

This chapter briefly discusses five golden rules, as follows: “let the participants speak, remember that your intuition can hurt and help you, be unbiased, do not give away information inadvertently, and watch yourself to keep sharp.” The rules emphasize that it is important to let the participant speak, rather than dropping into a conversation. One of the most obvious things that distinguishes experienced moderators from inexperienced ones is the amount of talking they do while participants are working on tasks. The experienced moderator speaks only when necessary and watches and listens more, carefully observing and analyzing what is going on. Less experienced moderators, on the other hand, interrupt participants while they’re talking or try to finish their thoughts, sometimes putting words in their mouths or giving elaborate explanations for actions they’ve taken. In addition, this chapter explains the designers intent, or being defensive, as it is very difficult to moderate test sessions when one is designing the product because he/she is too familiar with it. One knows all about the goals of the design, the underlying technology constraints, the legacy systems that had to be accommodated, the tradeoffs that were made along the way, the politics involved, plans for future enhancements, et cetera. Thus, it is similar to trying to proofread ones own writing. This is one reason that development teams often ask people from outside the team to moderate a usability test.

Integrating the videos

The chapter provides seven videos, and also explains their purpose and provides some advice about how to view them. Six videos show test sessions in progress. They are not highlighted videos; rather, each one focuses on one 5- to 7-minute segment of a typical test session. Two videos focus on the pretest instructions, two focus on interacting while tasks are being performed, one focuses on the post-test interview, and one illustrates a remote test. Each video shows several situations that illustrate the major points. The description of each video includes an introduction about its purpose and a table that describes the actions in sequence and offers comments about their significance. Each video has an accompanying panel discussion by a group of experts. The purpose of the first video is to illustrate the way an experienced moderator typically gives the instructions at the beginning of a session. The moderator and the participant in this video are sitting facing each other. Video 2 has the same scope as Video 1 but with two differences: the moderator is inexperienced and he reads most of the instructions from a script. The third video is one of two videos that deal with events that occur while participants are working on tasks. This video presents a situation in which the participant is increasingly annoyed while working on a task. The moderator handles some situations well but he is less than perfect in others. This chapter further explains all the videos very briefly.

Chapter 5 – Initial contacts

Publisher Summary This chapter focuses on the interaction with participants at three points of contact as follows: during recruiting, when they arrive, and during the pretest briefing. It discusses the critical time before the test starts, when one has the chance to make a great first impression, establish rapport with participants, and put them at ease. Steps of recruiting include choosing a contact method, explaining what the test is about, screening candidates for eligibility, and confirming their appointments. There are a number of ways to recruit participants for a usability test; the best method depends on the nature of the test and the types of people one is looking for. Many practitioners hire third-party recruiting agencies and pay them (per hour or per person) to locate, screen, and schedule test participants. Other practitioners do their own recruiting, building up a database of potential participants over time. Regardless of who does the recruiting, one needs to decide how to reach out to the target population. The most common methods are advertising via the Internet, phone, email, or a combination of these.