Nicole Ummelen

Measuring reading behavior in policy documents: a comparison of two instruments

Techniques for observing selection and reading behavior in professional documents, such as the thinking-aloud and the click-and-read methods, may affect the reading process to be observed. Such so-called reactivity problems complicate the use of these instruments in experimental research and usability testing. If their influence is unknown, any experimental results obtained with these instruments may be caused by the testing method. One way to detect reactivity effects is to compare different instruments in a series of experimental studies. In this initial study, we compared the thinking-aloud method, the click-and-read method, a combination of these two methods and a silent reading condition. Subjects read and judged a 53-page policy document in one of these conditions. We investigated whether or not different observation instruments caused specific differences in information selection, judgment and knowledge. Thinking aloud did not cause any differences in the selection of information. Both the thinking-aloud method and the click-and-read method affected the judgment task outcome. Thinking aloud led to many positive and few negative judgments, whereas silent reading led to many negative and few positive judgments. The results for the click-and-read method showed a tendency toward the same effect. Neither method affected the knowledge test results.

Procedural and declarative information in user instructions: what we do and don't know about these information types

The use and the effects of different information types in user instructions are not completely clear. Research showed that procedural information (information about the actions) is the most important information type during use. Research results about the effects of declarative information (explanatory information) are not conclusive. It is known that users are interested in declarative information, but it is not known in which situations they read this information and in which situations this information affects task performance. Furthermore, it is not clear which different types of declarative information have to be distinguished. In this paper, we describe what is known about different information types in user instructions and what is not yet known. On the basis of our analysis, we propose follow-up experiments and we recommend practitioners to consider the possible advantages and disadvantages of including declarative information in instructions for use.

STRUCTURING JOB RELATED INFORMATION ON THE INTRANET: AN EXPERIMENTAL COMPARISON OF TASK VS. AN ORGANIZATION-BASED APPROACH

In this article, we present a usability experiment in which participants were asked to make intensive use of information on an intranet in order to execute job-related tasks. Participants had to work with one of two versions of an intranet: one with an organization-based hyperlink structure, and one with a task-based hyperlink structure. Efficiency and effectiveness were measured in terms of execution time and task accuracy, respectively. After the task execution, participants were asked to evaluate the task as well as the intranet. The results show that participants perform more efficiently with the organization-based structure, which is probably due to their familiarity with this structure. A post hoc analysis revealed, however, a learning effect in the task condition, which suggests that once users are acquainted with it, a task structure is at least as efficient.