William Hudson

Playing your cards right: getting the most from card sorting for navigation design

Card sorting is a knowledge-elicitation technique often used by information architects, interaction designers, and usability professionals to establish or assess the navigation hierarchy of a Web site. The items are typically menu entries or hyperlinks, while the groups are categories or headings. The process involves asking participants to sort items into meaningful groups. In open card sorts, the number and names of groups are decided by each participant, while in the closed card sorts, these factors are fixed by the researcher in advance. Analysis of card-sorting results range from simple counting of the number of times items were grouped together to the rather intimidating “monothetic agglomerative cluster analysis” (known simply as cluster analysis in most cases). Unfortunately, no single technique provides everything a researcher needs to know, especially if convincing evidence is needed to persuade colleagues or customers of the effectiveness of a proposed design. The evidence we need falls into three categories: Participants. Are these the right participants for our site? Are they all thinking about the items and their groupings in a similar way? Do they have a clear understanding of the card-sorting task itself? Items. Are the item names well-understood by participants? Are there alternatives that should be considered—perhaps terms users are more familiar with? Groups. For closed card sorts, have we chosen the right number of groups and names for each? For open sorts, are participants largely in agreement about the number of groups needed? How well do participants feel the items fit into their groups? Happily, the answer to this last question—how well participants feel the items fit into their groups—can also help us with many of the other issues listed. Coupled with a few data-collection guidelines and alternative presentations of results, we can collect fairly comprehensive evidence about what will and will not work in our navigation hierarchies. So let’s examine this last question in some more detail: How well do participants feel the items fit into their groups? It is possible to argue that this question is redundant, that the items must fit into their groups relatively well in any given set of results, because that is how the participant decided to group them. However, practical experience says otherwise. Consider the following example that I use as a practice sorting exercise when teaching: Participants are given the names of 14 wines and asked to sort them into three groups (full-bodied reds, dry whites, and sparkling). Participants are instructed to omit any items they feel do not really belong to any of the groups. The cluster analysis dendogram shown in figure 1 is a fairly typical set of results for 12 participants. The dendogram shows the three groups, connected in the characteristic tree-like structure that gives this form of presentation its name. The vertical connections between branches indicate the strength of the relationship between items, with stronger relationships to the right and weaker to the left. So for example, the relationship between Riesling and White Zinfandel is the strongest in this dendogram, meaning that those two items appeared in the same group more frequently than any other pair of items. The relationship between Beaujolais and Claret is only slightly less strong, while the weakest relationship between any single item and its groups is Pinot Grigio. But for wine lovers, there is something fishy about this result. If you remember, participants were asked to group the wines into three categories, one of which was full-bodied reds. While Beaujolais is a red wine, it certainly cannot be described as full-bodied (there is also a problem with White Zinfandel that I am not going to deal with here—it was a nasty trick played on participants that will p eo p le whudson@acm.org

Inclusive design: accessibility guidelines only part of the picture

largest of its kind ever undertaken, but sadly the results are no sur-prise—certainly not to anyone who has ever tried to navigate a Web site without a mouse. However, some of the findings are quite interesting and have even managed to spark a minor conflict with the W3Cs Web Accessibility Initiative, which may increase interest in the topic, if nothing else. First, the predictable information: All 1,000 sites were tested against the WAIs Web Content Accessibility guidelines with an automatic testing tool. The results are as follows: • Fewer than 19 percent of home pages were Level A compliant (assuming that some would also fail the manual tests required by the guidelines). • Only six of the 1,000 home pages passed the automatic testing for Level AA, with four of those failing the manual tests. That left only two sites as AA compliant (0.2 percent). • No home page was Level AAA compliant. Of the 1,000 sites originally chosen , 100 were tested by a panel of disabled users and a further 20 underwent expert inspection. The results of these further studies were compared with the automated tests, with some slightly more interesting outcomes: • There was no correlation between the number of WAI checkpoint violations and the results of the user tests. • 69 percent of the warnings raised by automated testing needed to be manually checked, but only five percent of those resulted in violations (so 95 percent of the warnings that required manual checking were false positives). These are pretty discouraging results, though automated testing does have some compensating strong points: : / 55 i n t e r a c t i o n s / j u l y + a u g u s t 2 0 0 4 hci and the web hci and the web

User stories don't help users: introducing persona stories

User stories are one of the most popular alternatives to traditional user requirement specifications (see Figure 1). But despite their promising name, user stories are not about – and don’t necessarily help – users at all. In most cases, user stories are written about roles that users adopt and take no account of the needs and behaviours of real users. Were that not indictment enough, user stories suffer from demonstrable flaws in structure and are often written by the wrong people at the wrong time.

A tale of two tutorials: a cognitive approach to interactive system design and interaction design meets agility

I usually only make it to one conference a year, but in 2004 I had the unusual pleasure (or duty, depending on your point of view) of attending both CHI in Vienna and OOPSLA in Vancouver. (Next year I will do the W’s—Warsaw and Walla Walla). I presented my own material on user-centered design and had the opportunity to attend tutorials on two very different approaches to improving the usability of interactive systems.

Fitts at 50: for link design, size does matter

© ACM 1072-5220/05/0500

The cost of more: psychology of choice in interaction design

5.00 I missed the 50th anniversary of Paul Fitts’ paper The information capacity of the human motor system in controlling the amplitude of movements. I wasn’t exactly expecting street parties, but June 2004 slipped by without even a celebratory cup of tea. The title of the paper may not mean much on its own, but I end up discussing Fitts’ Law with interaction designers at least once a week, so I would count it as one of the more robust works in our field. The conversations usually go something like this:

Foraging à la carte: an appetite for popup menus?

of choice offered to consumers. Too little choice means a store may be omitting services or products important to some users (and worse still, that a competitor might include). Too much choice adds complexity with increased potential for confusing or frustrating potential purchasers. This may sound like a big enough challenge to cope with, but according to Barry Schwartz, the author of The Paradox of Choice, the difficulty I have just described is only the tip of the iceberg. While Schwartz deals primarily with choice outside the digital world, I believe that too much choice and how it is presented to users can have serious implications for the Web. A fundamental theme of Paradox is the difference in strategy between people who are “satisficers” and those who are “maximizers.” While satisficers are content to select products or services that meet a minimum set of requirements, maximizers want to make sure they have made the best possible decision. Schwartz and his colleagues discovered that:

Books and mortar: the science of Web shopping

By William Hudson Principal Consultant Syntagm Ltd whudson@syntagm.co.uk Information foraging theory1 gives us a useful analogy for explaining user behavior when searching for information: Much as they might in a forest, users try to follow the scent of their prey. On the Web, this scent takes the form of visual clues, for the most part links displayed either in navigation panels or within the content area of each page. A central tenet of this theory is that users attempt to maximize the gain of useful information while minimizing the cost of their effort in obtaining it. This can be seen by observing users performing information-gathering tasks during usability testing; each page is examined for clues of the trail where the information scent is strongest. If there are too many false clues, users perceive that the cost of continuing is too high and they adopt another strategy, such as using a search facility or site map or abandoning the current site altogether.

Don't make me read: use and abuse of text in Web page design

One of the Web’s best-known booksellers appears to have become bored with the idea of selling books. The trickle of products other than books has burgeoned into a torrent flooding from its pages. In the early days, paddling among CDs and DVDs when trying to find books was not a particular problem. Few categories overlapped, so by searching for a few words in the title, you could usually find what you were looking for. But no more. Entering the words “easy RUP”1 results in the following product offerings (in order of appearance):

The whiteboard: metaphor: a double-edged sword

I’m a great admirer of Steve Krug’s Don’t Make Me Think and recommend it to anyone who wants only a single book on the subject of Web usability. Steve’s chapter on text is based on a reminder from The Elements of Style by William Strunk and E.B. White: to “omit needless words.” It’s a point well worth making and the chapter does an admirable job of reducing 103 words of instruction on a Web page to a just-about-bearable 41.