Elisabeth Cuddihy

Does think aloud work?: how do we know?

The think aloud method is widely used in usability research to collect users reports of the experience of interacting with a design so that usability evaluators can find the underlying usability problems. However, concerns remain about the validity and usefulness of think aloud in usability studies. In this panel we will present current studies of the think aloud method, examine and question its usage in the field, discuss the possible pitfalls that may threaten the validity of the method, and provide comments/suggestions on the application of the method. Panel participants will discuss results drawn from both applied research and basic research.We believe that this panel discussion will be useful for HCI designers and usability practitioners in that it will acquaint them with concerns that people have about the think aloud method and provide them with suggestions for improved use of the method. For HCI or usability researchers, this panel discussion will address the importance of formally investigating currently used or newly designed usability methods.

Methods for assessing web design through the internet

Web design guidelines are often derived from best practices, conventional wisdom, or small-scale usability studies conducted in labs. We contend that if Web design guidelines are to inform the design of Web sites serving varied audiences with varied needs, the guidelines must be derived from empirical research that assesses users in their native environments as they interact with real Web sites. While we believe that the delivery of a remote Web-based experiment has many potential benefits, we acknowledge that it can be difficult to exercise experimental control so as to acquire reliable data, capture user behavior unobtrusively, extract meaningful information from server logs, and collect valid survey data. Therefore, we report on how we addressed some of the challenges of conducting remote empirical studies of the effect of navigational cues on Web browsing behavior.

I thought this was going to be a waste of time:How portfolio construction can support student learning from project-based experiences

In this work, we sought to understand ways that students experienced a small-scale portfolio assignment provided to them as an opportunity to reflect on their experiences in a project-based class. This work was motivated by research in various instructional contexts showing that portfolio construction results in important learning outcomes. We wanted to see if such findings would extend to project-based learning situations in engineering. In addition, our research was motivated by the need to better understand specific portfolio assignments in terms of the effort required of students, the extent to which students value such assignments, and the ways in which students’ overall experiences differ in terms of learning, effort, and value. To address these issues, we interviewed thirteen students who completed a portfolio assignment as a culminating activity for a project-based manufacturing class. We selected these students because their individual responses to a screening survey suggested that they had very different experiences with the portfolio assignment. We then conducted open-ended interviews with the students in order to better understand their experiences. In this paper, we show how the themes of epistemically different, manageably effortful, and ultimately valuable can be used to understand three ways of experiencing the assignment (as a significant experience, a muted experience, and a limited experience). This work provides practical information to educators interested in supporting reflection while also contributing to the scholarship on how portfolios support learning from experience.

Internet-based research for the desktop and beyond: Building a foundation of excellence for information design on the web

The internet has become a primary vehicle for delivering information, but the online platforms used to access that information are constantly shifting. Web users are more sophisticated in the ways they access information online, but it is unclear whether design best practices have kept pace. The need to develop a set of effective best practices for web design, based on empirical research, becomes even more critical as people migrate from desktops to mobile browsing. We strongly support the use of internet-based research to ensure the ecological soundness of information design best practices.

Web page previews: effect on comprehension, user perceptions, and site exploration

Informational websites that contain pages of related hyperlinked information are often organized in a hierarchical tree, such that child-node pages contain primary content and higher-level parent-n...Informational websites that contain pages of related hyperlinked information are often organized in a hierarchical tree, such that child-node pages contain primary content and higher-level parent-node pages link to sets of related child-node pages. In such designs, the parent-node page content can act as topic previews for content contained in their child pages. This article describes a 3 × 2 factorial experiment that examined the effect of preview styles and hierarchical navigation menus on website users’ reading comprehension, perceptions of user experience and site exploration behaviour. Significant differences were found for inferential reading comprehension, with higher comprehension occurring when readers were exposed to previews with embedded links, even though previews with embedded links negatively affected user perceptions. Presence or absence of hierarchical menus did not affect comprehension but lack of menus significantly degraded usability perceptions and site exploration. Recommendations for informational website designs are discussed based on these findings.

The Effect of Structural Cues on User Comprehension, Navigational Behavior, and Perceptions

Web authors need writing strategies based on empirical studies of real Web users, strategies that will produce comprehensible Web documents that facilitate readers on the Web. The study reported here investigated the effect of structural cues (text previews and navigational tab menus) on user comprehension, navigational behavior, and perceptions. We found that text previews with embedded links to related pages (children nodes) enhance comprehension and discourage site exploration. We also found that lists of links (in lieu of previews) encourage site exploration and are well liked. And, we found that navigational tab menus encourage site exploration and enhance user perceptions. Our findings underscore that good Web design must be context specific - structural cues that promote understanding are not necessarily those that promote exploration or enjoyment. Keywords: hypertext comprehension, navigation, perceptions of use, remote user testing, signaling, structural cues.

Beyond current user research: designing methods for new users, technologies, and design processes

With the rapid diversification of computing technologies, user researchers often encounter new applications, new users and scenarios of use, and even new design processes that require new research approaches. (For instance, how do you assess the usability of a roving, multi-user, activity-based system? Or how do you assess a device intended to support a social network?) In this workshop we will examine case studies from user researchers who have modified a classic user research technique or created a new technique to meet the exigencies of such challenges. The workshop is organized around three aspects of methods design: the impetus for creating the new or modified method, the challenges of implementing the method, and the impact of the innovation on the design or design process.