Leo Lentz

Improving the usability of patient information leaflets

OBJECTIVE This study assesses the usability of three patient information leaflets and attempts to improve them while complying with the current EU regulations. METHODS Three original leaflets were tested among 154 potential users. Every participant answered 15 scenario questions for one of the leaflets. The leaflets were subsequently redesigned based on the test results and evidence-based Document Design principles. The revised texts were tested among 164 participants. RESULTS All three original leaflets suffered from usability problems, especially problems related to finding relevant information. On average, only 75% of the topics could be located. Comprehension of the information, once found, was around 90%. The revisions led to better performance. Information was found faster and more successful. Comprehension scores were higher as well. A follow-up study shows that these findings can be generalized over paper formats. CONCLUSION Although the current EU regulations for patient information leaflets do not guarantee leaflet usability, the leaflets can be improved somewhat within the regulations. However, further research should evaluate the text structure currently imposed on leaflets. PRACTICAL IMPLICATIONS Information leaflets must be written, or rewritten, according to Document Design principles. Furthermore, they must be user tested in a rigorous way.

Coherence Marking, Prior Knowledge, and Comprehension of Informative and Persuasive Texts: Sorting Things Out

Coherence plays a central role when readers construct meaning from a text. Previous research has shown how coherence marking affects text processing and representation. However, this effect seems to depend on readers prior knowledge of the text content: Low knowledge readers benefit from coherence marking, whereas high knowledge readers benefit from a more implicit text (McNamara & Kintsch, 1996). Because this interaction was not consistently found in previous research, this article takes a closer look at the operationalization of the experimental variables: coherence marking, prior knowledge, and text comprehension. Also, this article compares the effect on both informative and persuasive texts. Results indicate that linguistic marking of coherence indeed interacts with prior knowledge in the informative genre, but not in the persuasive genre.

Website evaluation questionnaire: development of a research-based tool for evaluating informational websites

Online questionnaires are frequently used to monitor the quality of municipal and other governmental websites. In the present situation, many government organizations seem to reinvent the wheel and develop their own questionnaire. This leads to the undesirable situation that website quality is often assessed with instruments that are not comparable with each other and are not empirically validated. This article presents a generic Website Evaluation Questionnaire (WEQ) for the evaluation of informational websites. The WEQ was developed on the basis of the literature on usability and user satisfaction and was tested and revised in several rounds. This has resulted in a reliable questionnaire measuring clearly distinct quality dimensions of informational websites. The WEQ can be used by governmental organizations for evaluating their websites and for benchmarking their results against each other.

Usable guidelines for usable websites? An analysis of five e-government heuristics

Many government organizations use web heuristics for the quality assurance of their websites. Heuristics may be used by web designers to guide the decisions about a website in development, or by web evaluators to optimize or assess the quality of an existing website. Despite their popularity, very little is known about the usefulness of heuristics for web designers and evaluators. In this study, five government heuristics are examined with special attention to their presumed use, content, validity, and presentation format. Our findings raise questions about the usability of heuristics for web design and web evaluation purposes. More research into the actual use and effects of web heuristics is needed.

The evaluation of text quality: expert-focused and reader-focused methods compared

The authors compare a reader-focused text evaluation with an expert-focused evaluation by technical writers and subject/audience experts. The experts were asked to predict the problems readers had signaled in a government brochure about alcohol. On average, they predicted less than 15% of the reader problems and produced a lot of new problem detections. In addition, the experts showed little mutual agreement in their problem detections. Their results suggest that a reader-focused evaluation should not be substituted for an expert-focused evaluation. The paper ends with a discussion of methodological issues for this type of research.

Measuring the quality of governmental websites in a controlled versus an online setting with the 'Website Evaluation Questionnaire'

The quality of governmental websites is often measured with questionnaires that ask users for their opinions on various aspects of the website. This article presents the Website Evaluation Questionnaire (WEQ), which was specifically designed for the evaluation of governmental websites. The multidimensional structure of the WEQ was tested in a controlled laboratory setting and in an online real-life setting. In two studies we analyzed the underlying factor structure, the stability and reliability of this structure, and the sensitivity of the WEQ to quality differences between websites. The WEQ proved to be a valid and reliable instrument with seven clearly distinct dimensions. In the online setting higher correlations were found between the seven dimensions than in the laboratory setting, and the WEQ was less sensitive to differences between websites. Two possible explanations for this result are the divergent activities of online users on the website and the less attentive way in which these users filled out the questionnaire. We advise to relate online survey evaluations more strongly to the actual behavior of website users, for example, by including server log data in the analysis.

The Forewarning Effect of Coherence Markers in Persuasive Discourse: Evidence From Persuasion and Processing

Several studies showed how coherence markers, like connectives and lexical cue phrases, influence the processing and representation of informative text. Although discourse analysts have repeatedly argued that coherence markers influence the processing of persuasive text as well, there is hardly any empirical evidence for this idea. This article reports on 2 experiments investigating the possible forewarning effect of coherence marking: When coherence markers cause readers to recognize an attempt to influence them, they build up resistance and it becomes difficult to persuade them. The experiments show that objective marking is more persuasive than subjective marking. Also, subjective marking causes readers to recognize the persuasive authors intent more easily. Furthermore, subjective markers seem to cause resistance to persuasion, whereas objective markers improve integration of information.

Expert Judgments Versus Reader Feedback: A Comparison of Text Evaluation Techniques

Are technical writers able to predict the results of a reader-focused text evaluation? In this article we report a study with fifteen technical writers, who were asked to point out the reader problems in a public information brochure. The brochure was also evaluated with thirty readers from the target audience (using a combination of the plus-minus method, a questionnaire, and user protocols). The results of both kinds of text evaluation show little overlap. The technical writers only predicted a small proportion of the reader feedback, and produced a lot of new problem detections. In addition, there was little agreement among the technical writers with regard to their problem detections.

Users' abilities to review website pages

Web sites increasingly encourage users to provide comments on the quality of the content by clicking on a feedback button and filling out a feedback form. Little is known about users’ abilities to provide such feedback. To guide the development of evaluation tools, this study examines to what extent users with various background characteristics are able to provide useful comments on informational Web sites. Results show that it is important to keep the feedback tools both simple and attractive so that users will be able and willing to provide useful feedback on Web site pages.

Municipalities on the web: user-friendliness of government information on the internet

Municipal websites are highly visible manifestations of e-government developments. Though the content and functionality of these websites are rapidly expanding, the usability of municipal websites is as yet underexposed. This paper reports on the results of a scenario-based evaluation of 15 Dutch municipal websites. Despite the often positive scores of the websites in national checklist-based rankings, an overwhelming number of usability problems was found with each scenario. A qualitative description is given of two important categories of user problems: navigation and perspective-taking. Both problem categories suggest that a strict HCI perspective on website evaluation may not suffice: it is the interplay between context, content and interface that will be crucial for the optimization of municipal websites.