Torkil Clemmensen

Getting access to what goes on in people's heads?: reflections on the think-aloud technique

One of the basic usability testing techniques the HCI community draws on, and which stands out as unique, is thinking aloud. We introduce the many names, uses and modifications of the classical think aloud technique, and ask the rhetorical question: What do researchers think they get when they ask people to think aloud? We answer it by discussing the classical work of Ericsson and Simon(1984), in particular their distinction between vocalisation, verbalisation and retrospective reports and the relation to short term memory. Reintroducing the psychological perspective and the focus on higher order cognitive processes, we argue that access to subjective experience is possible in terms of introspection and describe a technique that invites the user to become a participant in the analysis of his or her own cognitive processes. We suggest that use of think aloud has as a prerequisite explicit descriptions of design, test procedure and framework for analysis. We point out, however, that if the aim is to get access to human thinking, HCI research may benefit from experimental research.

Non-universal usability?: a survey of how usability is understood by Chinese and Danish users

Most research assumes that usability is understood similarly by users in different cultures, implying that the notion of usability, its aspects, and their interrelations are constant across cultures. The present study shows that this is not the case for a sample of 412 users from China and Denmark, who differ in how they understand and prioritize different aspects of usability. Chinese users appear to be more concerned with visual appearance, satisfaction, and fun than Danish users; Danish users prioritize effectiveness, efficiency, and lack of frustration higher than Chinese users. The results suggest that culture influences perceptions of usability. We discuss implications for usability research and for usability practice.

An Overview of a Decade of Journal Publications about Culture and Human-Computer Interaction (HCI)

In this paper, we analyze the concept of human-computer interaction in cultural and national contexts. Building and extending upon the framework for understanding research in usability and culture by Honold [3], we give an overview of publications in culture and HCI between 1998 and 2008, with a narrow focus on high-level journal publications only. The purpose is to review current practice in how cultural HCI issues are studied, and to analyse problems with the measures and interpretation of this studies. We find that Hofstede’s cultural dimensions has been the dominating model of culture, participants have been picked because they could speak English, and most studies have been large scale quantitative studies. In order to balance this situation, we recommend that more researchers and practitioners do qualitative, empirical work studies.

Usability problem identification in culturally diverse settings

There are indications that established methods for evaluating information system usability that have been developed for use in, e.g. Europe or the USA, fail to give reliable results in countries such as India, China or Malaysia. This paper presents the theoretical background, related work and a definition of culture that should be useful for studies of multiple‐country usability testing. This includes a discussion of cultural fit and the consequences of cultural (in)consistencies between stakeholders in system development and use. As an illustrative example of the kind of academic research that needs to be done, a pilot study is described. The pilot study exemplifies themes to explore, who should be participants and where should the study be done, how to find examples of multiple‐country usability testing, how to collect data and how to analyse that data and what kind of results and discussion of results that may be expected. The conclusion summarises the paper, discusses the limitations of the pilot study and gives detailed suggestions to future research.

Usability constructs: a cross-cultural study of how users and developers experience their use of information systems

Whereas research on usability predominantly employs universal definitions of the aspects that comprise usability, people experience their use of information systems through personal constructs. Based on 48 repertory-grid interviews, this study investigates how such personal constructs are affected by two factors crucial to the international development and uptake of information systems: cultural background (Chinese, Danish, or Indian) and stakeholder group (developer or user). We find that for the user group frustrating and useful systems are experienced similarly, whereas for the developers frustrating systems are experienced similarly to easy-to-use systems. Looking at the most characteristic construct for each participant we find that Chinese participants use constructs related to security, task types, training, and system issues, whereas Danish and to some extent Indian participants make more use of constructs traditionally associated with usability (e.g., easy-to-use, intuitive, and liked). Further analysis of the data is ongoing.

Cultural usability tests - how usability tests are not the same all over the world

The cultural diversity of users of technology challenges our methods for usability evaluation. In this paper we report on a multi-site, cross-cultural grounded theory field study of think aloud testing in seven companies in three countries (Denmark, China and India). The theoretical model that emerges from the data suggests that the production of a usability problem list is multicausal and subject to cultural variations. Even the way usability problems are experienced by test participants may be different. In the discussion we outline practical guidelines for a test that is more sensitive towards cultural usability.

Templates for Cross-Cultural and Culturally Specific Usability Testing: Results From Field Studies and Ethnographic Interviewing in Three Countries

The cultural diversity of users of technology challenges our methods for usability testing. This article suggests templates for cross-culturally and culturally specific usability testing, based on studies of usability testing in companies in Mumbai, Beijing, and Copenhagen. Study 1 was a cross-cultural field study of think-aloud testing done by usability vendor companies in the three countries. The result was a grounded theory of cultural variations in the production of a usability problem list. Study 2 was a follow-up, ethnographic interview study of how the companies typically perform usability tests. The result was the construction of templates for usability testing. The culturally specific templates were in Mumbai “user-centered evaluation,” Copenhagen “client-centered evaluation,” and Beijing “evaluator-centered evaluation.” The findings are compared with related research, and the implications are pointed out. The templates can be seen as a simple and practical way to plan, compare, and improve the way usability testing is carried out in multiple, different cultures and countries.

Personal Usability Constructs: How People Construe Usability across Nationalities and Stakeholder Groups

Whereas the concept of usability is predominantly defined analytically, people relate to systems through personal usability constructs. Based on 48 repertory-grid interviews, this study investigates how such personal constructs are affected by two factors crucial to the international development and uptake of systems: nationality (Chinese, Danish, or Indian) and stakeholder group (developer or user). We find no significant overall difference across nationalities, but further analyses suggest that conventional usability aspects such as ease of use and simplicity are prominent for Chinese and Danish but not Indian participants and that a distinction between work and leisure-related communication is central to Chinese and Indian but not Danish participants. For stakeholder groups, we find a significant overall difference between developers and users. Unlike developers, users associate ease of use with leisure and, conversely, difficulty in use with work-relatedness. Further, users perceive usefulness as related to frustration and separate from ease of use, whereas developers construe usefulness, fun, and ease of use as related. In construing usability, participants make use of several constructs that are not part of prevailing usability definitions, including usefulness, fun, and security.

The cultural usability (CULTUSAB) project: studies of cultural models in psychological usability evaluation methods

Cultural models in terms of the characteristics and content of folk theories and folk psychology have been important to social scientists for centuries. We suggest that they should be at the heart of the scientific study of human-computer interaction (HCI). The CULTUSAB project is conducting an in-depth investigation of the key dimensions of culture that affect usability testing situations, including language, power distance, and cognitive style. All phases of the usability test are being evaluated for cultural impact, including planning, conducting, and reporting results. Special attention is being focused on subject-evaluator communication and cultural bias in the test design and structure of the user interface being tested. Experiments are being replicated in three countries: Denmark, India and China. The research will result in new testing methods and guidelines that increase the validity of usability tests by avoiding cultural bias, and allow us to produce comparable results across different countries.

Whatever happened to the psychology of human‐computer interaction?: A biography of the life of a psychological framework within a HCI journal

Purpose – To outline how psychology as one of the original approaches to human‐computer interaction (HCI) has formed a key part of the HCI literature, and to discuss the need for psychological approaches to HCI and system development.Design/methodology/approach – The contributions to the journal Human‐Computer Interaction is examined from the journals start in 1985 up to the millennium. The analysis focuses the three main elements, task, user and computer, in the classic study “Psychology of human‐computer interaction” from 1983.Findings – Provides information about authorship, and form and focus of research published. The paper concludes that already from the beginning, HCI researchers too narrowly used Card et al.s analytical framework. Today it has developed into a sub‐theory within a multidisciplinary HCI science and in this role it continues to be an important cumulative factor in HCI.Research limitations/implications – The main conclusion about the role of psychology in HCI only applies to the mai...