Linden J. Ball

An Eye Movement Analysis of Web Page Usability

An experiment is reported that investigated the application of eye movement analysis in the evaluation of Web page usability. Participants completed two tasks on each of four Web site homepages. Eye movements and performance data (Response Scores and Task Completion Times) were recorded. Analyses of performance data provided reliable evidence for a variety of Page and Task effects, including a Page by Task interaction. Four eye movement measures (Average Fixation Duration, Number of Fixations, Spatial Density of Fixations, and Total Fixation Duration) were also analysed statistically, and were found to be sensitive to similar patterns of difference between Pages and Tasks that were evident in the performance data, including the Page by Task interaction. However, this interaction failed to emerge as a significant effect (although the main effects of Page and Task did). We discuss possible reasons for the nonsignificance of the interaction, and propose that for eye movement analysis to be maximally useful in interface-evaluation studies, the method needs to be refined to accommodate the temporal and dynamic aspects of interface use, such as the stage of task processing that is being engaged in.

Putting ethnography to work: the case for a cognitive ethnography of design

The methods of ethnography and cognitive psychology are frequently set in opposition to each other. Whilst such a view may be appropriate in defining pure, or prototypical, classes of each activity, the value and necessity of such a distinction is broken down when researchers are goal-directed to study complex work domains in order to foster technological change. In this paper, we outline a rapprochement of these methods, which we term cognitive ethnography. The value of qualifying ethnography in this way is to emphasize systematically the differences between ethnography as a radial category and the kinds of legitimate method used to study work practices which are often referred to as ethnographic, but which in practice differ in important ways from prototypical ethnographic studies. Features of cognitive ethnography such as observational specificity, verifiability and purposivenes challenge many of the tenets of a pure ethnographic method, yet they are essential for studies that are undertaken to inform technological change. We illustrate our arguments with reference to a project to develop a tool for supporting design re-use in innovative design environments.

Structured and opportunistic processing in design: a critical discussion

We present a critical discussion of research into the nature of design expertise, in particular evaluating claims that opportunism is a major influence on the behaviour of expert designers. We argue that the notion of opportunism has been under-constrained, and as a consequence the existence of opportunism in expert design has been exaggerated. Much of what has been described as opportunistic design behaviour appears to reflect a mix of breadth-first and depth-first modes of solution development. Whilst acknowledging that opportunities can arise in the design process (e.g. serendipitous solution discovery), such events might equally confirm structured behaviour as cause unstructured behaviour. We argue that the default mode for truly expert designers is typically a top-clown and breadth-first approach, since longer-term considerations of cost-effectiveness are more important for expert designers than short-term considerations of cognitive cost. However, there are situations (e.g. when faced with a highly unfamiliar design task) where it is cost-effective for experts to pursue a depth-first mode of solution development. The implications of our analysis for the development of methods and tools to support the design process are also discussed.

Problem-solving Strategies and Expertise in Engineering Design.

A study is reported which focused on the problem-solving strategies employed by expert electronics engineers pursuing a real-world task: integrated-circuit design. Verbal protocol data were analysed so as to reveal aspects of the organisation and sequencing of ongoing design activity. These analyses indicated that the designers were implementing a highly systematic solution-development strategy which deviated only a small degree from a normatively optimal top-down and breadth-first method. Although some of the observed deviation could be described as opportunistic in nature, much of it reflected the rapid depth-first exploration of tentative solution ideas. We argue that switches from a predominantly breadth-first mode of problem solving to depth-first or opportunistic modes may be an important aspect of the experts strategic knowledge about how to conduct the design process effectively when faced with difficulties, uncertainties, and design impasses.

The role of answer fluency and perceptual fluency as metacognitive cues for initiating analytic thinking.

Although widely studied in other domains, relatively little is known about the metacognitive processes that monitor and control behaviour during reasoning and decision-making. In this paper, we examined the conditions under which two fluency cues are used to monitor initial reasoning: answer fluency, or the speed with which the initial, intuitive answer is produced (Thompson, Prowse Turner, & Pennycook, 2011), and perceptual fluency, or the ease with which problems can be read (Alter, Oppenheimer, Epley, & Eyre, 2007). The first two experiments demonstrated that answer fluency reliably predicted Feeling of Rightness (FOR) judgments to conditional inferences and base rate problems, which subsequently predicted the amount of deliberate processing as measured by thinking time and answer changes; answer fluency also predicted retrospective confidence judgments (Experiment 3b). Moreover, the effect of answer fluency on reasoning was independent from the effect of perceptual fluency, establishing that these are empirically independent constructs. In five experiments with a variety of reasoning problems similar to those of Alter et al. (2007), we found no effect of perceptual fluency on FOR, retrospective confidence or accuracy; however, we did observe that participants spent more time thinking about hard to read stimuli, although this additional time did not result in answer changes. In our final two experiments, we found that perceptual disfluency increased accuracy on the CRT (Frederick, 2005), but only amongst participants of high cognitive ability. As Alter et al.s samples were gathered from prestigious universities, collectively, the data to this point suggest that perceptual fluency prompts additional processing in general, but this processing may results in higher accuracy only for the most cognitively able.

Structure in idea sketching behaviour

Abstract This paper describes a study designed to investigate the structure of idea sketches, here defined as the initial free-hand drawn externalisations produced by a designer of envisioned or partially envisioned entities. In the study, participants were asked to sketch freely from memory a number of chairs and to design a chair. The results obtained from analyzing the drawing process provide clear evidence for structure in idea sketching behaviour which is largely explained by reference to either volumetrical or functional cognitive models of the recalled or designed objects. This suggests that there is an intimate relationship between the cognitive and perceptual processes that are brought to bear on the recall and design tasks and idea sketching. It is concluded that the detailed study of sketching behaviour may provide a fruitful approach to understanding the relationship between cognition, the sketch, and sketching.

Working memory, metacognitive uncertainty, and belief bias in syllogistic reasoning

Studies of syllogistic reasoning have shown that the size of the belief bias effect varies with manipulations of logical validity and problem form. This paper presents a mental models-based account, which explains these findings in terms of variations in the working-memory demands of different problem types. We propose that belief bias may reflect the use of a heuristic that is applied when a threshold of uncertainty in ones processing—attributable to working-memory overload-is exceeded during reasoning. Three experiments are reported, which tested predictions deriving from this account. In Experiment 1, conclusions of neutral believability were presented for evaluation, and a predicted dissociation was observed in confidence ratings for responses to valid and invalid arguments, with participants being more confident in the former. In Experiment 2, an attempt to manipulate working-memory loads indirectly by varying syllogistic figure failed to produce predicted effects upon the size of the belief bias effect. It is argued that the employment of a conclusion evaluation methodology minimized the effect of the figural manipulation in this experiment. In Experiment 3, participants’ articulatory and spatial recall capacities were calibrated as a direct test of working-memory involvement in belief bias. Predicted differences in the pattern of belief bias observed between high and low spatial recall groups supported the view that limited working memory plays a key role in belief bias.

In Search of Salience: A Response-time and Eye-movement Analysis of Bookmark Recognition

Bookmarks are a valuable webpage re-visitation technique, but it is often difficult to find desired items in extensive bookmark collections. This experiment used response-time measures and eye-movement tracking to investigate how different information structures within bookmarks influence their salience and recognizability. Participants were presented with a series of news websites. The task following presentation of each site was to find the bookmark indexing the previously-seen page as quickly as possible. The Informational Structure of bookmarks was manipulated (top-down vs. bottom-up verbal organizations), together with the Number of Informational Cues present (one, two or three). Only this latter factor affected gross search times: Two cues were optimal, one cue was highly sub-optimal. However, more detailed eye-movement analyses of fixation behaviour on target items revealed interactive effects of both experimental factors, suggesting that the efficacy of bookmark recognition is crucially dependent on having an optimal combination of information quantity and information organization.

Applying ethnography in the analysis and support of expertise in engineering design

Whilst many contemporary studies of design have claimed to be using ethnographic methods, the techniques which have been employed often diverge from the characteristics of pure ethnography as used in traditional anthropological and sociological research. We argue that this is entirely appropriate for applied ethnography that is conducted in pursuit of explicit design goals. In this paper, we explore the relationship between pure and applied ethnography, and their use in cognitive and social research. We also discuss how the outcomes of applied ethnography can be applied to i) the design of a computer-based design support tool and ii) the development of controlled experimental studies that retain sufficient ecological validity to capture realistic design expertise. We additionally argue that objectivity in empirical studies of design can be obtained only by triangulating observations across methodologies that embrace both ethnographic and laboratory-based traditions.

Belief-Logic Conflict Resolution in Syllogistic Reasoning: Inspection-Time Evidence for a Parallel-Process Model

An experiment is reported examining dual-process models of belief bias in syllogistic reasoning using a problem complexity manipulation and an inspection-time method to monitor processing latencies for premises and conclusions. Endorsement rates indicated increased belief bias on complex problems, a finding that runs counter to the “belief-first” selective scrutiny model, but which is consistent with other theories, including “reasoning-first” and “parallel-process” models. Inspection-time data revealed a number of effects that, again, arbitrated against the selective scrutiny model. The most striking inspection-time result was an interaction between logic and belief on premise-processing times, whereby belief – logic conflict problems promoted increased latencies relative to non-conflict problems. This finding challenges belief-first and reasoning-first models, but is directly predicted by parallel-process models, which assume that the outputs of simultaneous heuristic and analytic processing streams lead to an awareness of belief – logic conflicts than then require time-consuming resolution.