J. M. Christian Bastien

Usability testing: a review of some methodological and technical aspects of the method

The aim of this paper is to review some work conducted in the field of user testing that aims at specifying or clarifying the test procedures and at defining and developing tools to help conduct user tests. The topics that have been selected were considered relevant for evaluating applications in the field of medical and health care informatics. These topics are: the number of participants that should take part in a user test, the test procedure, remote usability evaluation, usability testing tools, and evaluating mobile applications.

Effect of Task and Eccentricity of the Target on Detection Thresholds in Mesopic Vision: Implications for Road Lighting

Objective: The aim of this work is to assess how adding a driving-related task affects the detection of objects in peripheral vision, under mesopic conditions. Background: The main index used to assess the quality of road lighting installations refers to simple detection tasks in foveal vision, which raises methodological and practical questions. Method: The experimental design consisted of a three-phase experiment. In the first phase, two groups (control and experimental) performed a peripheral detection task (simple task). Based on these results an individual detection threshold was computed for each participant and eccentricity. A tracking task was performed in Phase 2 for both groups (steering a tracking target along a circuit, on a screen). In the third phase, the control group performed the same task as in Phase 2. The experimental group performed a double task, with a tracking (primary) task and a peripheral detection (secondary) task. Results: The data show an effect of the tracking task and eccentricity on peripheral event detection. The tracking task caused detection performance to decrease from 84.2% to 67.5%, p < .001. Conclusion: The small target visibility model used in road lighting may be improved, taking into account the effects of task and eccentricity on target detection. Application: This study supports improved roadway lighting design by guiding consideration of sign eccentricity and task load.

Effects of the viewing context on target detection. Implications for road lighting design

The Small Target Visibility (STV) model is the main model used to assess the quality of road lighting installations (IESNA, 2000). However, this model is based on a simple detection task in foveal vision using psychophysical data from laboratory conditions. The purpose of this study was to evaluate the impact of a complex background and apparent motion on target detection performance in mesopic vision, for three luminance contrasts, with reference to the STV scenario. To do so, participants were invited to detect standard square targets varying in terms of contrast presented in three Conditions: a uniform background, still images, and a video. Luminance levels were chosen in the mesopic domain relevant for road lighting at night. Images and video were chosen in relation to a driving task at night. The results showed that both the spatial context and the apparent motion had a negative impact on peripheral target detection performance: contrasts which are easy to detect in conditions close to the STV reference data may lead to poor performance if one adds context variables. These results give evidence that the STV model used for road lighting design based on laboratory data is limited, which strengthens previous results (Mayeur et al., 2008). The results are discussed in relation to the field factor used by practitioners to compensate for the differences between the STV reference scenario (detection of a small square target on a lit road while driving) and the STV psychophysical reference data.

From gaze plots to eye fixation patterns using a clustering method based on Hausdorff distances

Eye-tracking studies have provided us with interesting findings regarding the way users explore Web pages. Usually, visual fixations are represented using gaze plots and heatmaps. Probably one of the most cited studies on Web page exploration is the one by Nielsen [15] who demonstrated that users often read Web pages in an F-shaped pattern. However, this conclusion is based on aggregated data that do not represent any real user. In order to characterize and to compare scanpaths so as to uncover possible scan-patterns, a clustering method based on the Hausdorff distance has been applied to the data from 113 users. The results have shown that scan-patterns could be identified. Groups of users have been identified and their behaviors have been described with diverse eye-tracking metrics. The contributions of this study are outlined as well as its drawbacks. Research perspectives are proposed.

Les tests utilisateurs: avantages et inconvénients des passations individuelles et par paires

This article presents the advantages and drawbacks of individual and paired-user testing in terms of number and type of usability problems uncovered. Thirty-two participants had to perform 8 tasks designed to allow the evaluation of interactive tv services. For half of the participants (16) the test session was conducted individually: for the other half, the test session employed pairs of users (8×2 participants) performing the tasks together. The main results indicate that task completion times do not differ statistically between groups and that paired-user testing involve better success rates. Individual sessions allow the identification of more usability problems while paired-user sessions involve a better understanding of the difficulties users encountered. As for the type of usability problems uncovered (Guidance, Compatibility, etc.), differences between the two groups were observed. The results are discussed in relation to the commentairies published on the paired-user testing.

Analyzing Web pages visual scanpaths: between and within tasks variability

In this paper, we propose a new method for comparing scanpaths in a bottom-up approach, and a test of the scanpath theory. To do so, we conducted a laboratory experiment in which 113 participants were invited to accomplish a set of tasks on two different websites. For each site, they had to perform two tasks that had to be repeated ounce. The data were analyzed using a procedure similar to the one used by Duchowski et al. [8]. The first step was to automatically identify, then label, AOIs with the mean-shift clustering procedure [19]. Then, scanpaths were compared two by two with a modified version of the string-edit method, which take into account the order of AOIs visualizations [2]. Our results show that scanpaths variability between tasks but within participants seems to be lower than the variability within task for a given participant. In other words participants seem to be more coherent when they perform different tasks, than when they repeat the same tasks. In addition, participants view more of the same AOI when they perform a different task on the same Web page than when they repeated the same task. These results are quite different from what predicts the scanpath theory.

Y a-t-il de l'ergonomie dans les ordinateurs de vote français?: evaluation ergonomique des ordinateurs de vote français

There is, in many countries, the lack of usability of voting machines, can prevent vulnerable people to vote, especially the senior citizens. This awareness is not present in France where these voters have been totally neglected. To demonstrate that problem, we describe the problems of usability of three different voting systems used in France. We compared the interface of one of these voting computers with an interface that we have created with some usability guidelines. This comparison was made with users tests carried out on 16 elderly users. The results demonstrate the exclusion of the majority of these voters.