Hans van der Meij

Supporting reflective web searching in elementary schools

In this contribution, two design experiments are presented in which reflective web searching is implemented in six elementary classrooms. Reflective web searching is viewed to comprise three steps: (1) develop ownership over search questions, (2) interpret and personalize web content, and (3) adapt web content into personally meaningful answers. A portal and a worksheet supported reflective web searching. A wide range of qualitative data, including observations, interviews and group products, was collected to gain insight in the emerging practices. The findings show that the portal and worksheet successfully contributed to the development of ownership, and the interpretation and personalization of retrieved information. Enabling children to search the web collaboratively further enhanced interpretation and personalization. The prototypes of both portal and worksheet were improved across the design experiments, and their success rates increased. Despite the improvements, the process of adaptation was not sufficiently supported by the portal and worksheet.

Error-information in tutorial documentation

Novice users make many errors when they first try to learn how to work with a computer program like a spreadsheet or wordprocessor. No matter how user-friendly the software or the training manual, errors can and will occur. The current view on errors is that they can be helpful or disruptive, depending on the extent to which they are controlled in the learning process. This study examines one of the ways in which such error control can be brought about, namely by investigating the design and role of error-information in a (tutorial) manual. The error-information was designed to support the detection, diagnosis and correction of errors of novice users, and it was based on a general model of error-handling. In an experiment a manual that contained ample error-information was compared to a manual in which there was hardly any error-information. The outcomes showed that the presence of the error-information in the manual helped subjects perform better during practice as well as after practice. Among others, these subjects completed training faster and showed superior corrective knowledge and skill after practice, in addition to having acquired the same level of constructive skill. The discussion addresses the compensating roles of support for error-handling on screen and on paper.

A comparison of paper-based and video tutorials for software learning

Instruction on software usage has long been dominated by the paper-based tutorial. This dominance is now being challenged with the rise of facilities for producing and publishing recorded demonstrations (video). Typically, each instructional medium has its own qualities. The present study aimed to optimize the design of a video tutorial for software training by attending to both its strengths and its weaknesses vis-a-vis a paper-based tutorial. Based on a distinction between two functionally different components in software tutorials, four tutorial configurations were compared: Paper-based, Mixed A (paper-based preview and video procedure), Mixed B (video preview and paper-based procedure), and Video. The 111 fifth and sixth grade participants (mean age 11.8) received instructions about Words formatting options. The findings indicated significant and substantial improvements from pre-test to training in all conditions. In addition, participants in the Mixed A, Mixed B, and Video conditions outperformed those in the Paper-based condition. Significant and substantial learning gains were found from pre-test to post-test. Both the Mixed B and Video conditions outperformed the Paper-based condition. The success of the Mixed A, Mixed B, and Video tutorials is ascribed to the use of design guidelines for software training that direct the designer to optimize videos strong qualities and moderate or reduce its relative weaknesses

Assessment of the minimalist approach to computer user documentation

The minimalist approach (Carroll, 1990a) advocates the development of a radically different type of manual when compared to a conventional one. For example, the manual should proceed almost directly to procedural skills development rather than building a conceptual model first. It ought to focus on authentic tasks practised in context, as opposed to mock exercises and isolated practice. In addition, it should stimulate users to exploit their knowledge and thinking, as opposed to imposing the writers view and discussing everything that users should see or know. In the first part of the paper the construction of a tutorial based on the minimalist principles is described. A parallel is drawn with constructivism with which minimalism shares important notions of instruction. In the second part, an experiment is described in which the minimal manual was tested against a conventional one. The outcome favoured the new manual. For example, minimal manual users completed about 50% more tasks successfully on a performance test and displayed significantly more self-reliance (e.g. more self-initiated error-recoveries, and fewer manual consultations).

Does individual or collaborative self-debriefing better enhance learning from games?

The primary aim of this study is to find out whether use of different self-debriefing modes affects learning from a game. In self-debriefing participants are led to reflect upon their game experiences by a set of debriefing questions. Two conditions were compared: Individual and Collaborative self-debriefing. The 45 participants first played the game of Lemonade Tycoon Deluxe, were tested for knowledge and self-debriefed in pairs or alone. Then they played the game once more and were tested again. Game scores increased significantly from the first to the second round of gameplay to an equal degree in both conditions. Knowledge scores of participants in individual self-debriefing increased significantly more than those of participants in the Collaborative condition. The study shows that game-based learning can be effectively scaffolded with self-debriefing. Future studies might investigate whether the type of self-debriefing differentially affects game motivation. In addition, attention to the role of feedback is called for.

A test of the design of a video tutorial for software training

The effectiveness of a video tutorial versus a paper-based tutorial for software training has yet to be established. Mixed outcomes from the empirical studies to date suggest that for a video tutorial to outperform its paper-based counterpart, the former should be crafted so that it addresses the strengths of both designs. This was attempted in the present study. Two consecutive experiments were conducted to examine the effect of tutorial type video vs. paper based on task relevance, self-efficacy, mood, flow and task performance. Participants were students from junior high school. Both studies reported significant, positive contributions of the tutorials to task relevance, self-efficacy, mood and flow. Both studies also found significant and substantial effects on task performance. A learning gain of about 30% was achieved in both studies. A retention task, completed only in Study 2, further revealed that the learning effect was stable. More importantly, performance on this task also indicated a significant interaction with tutorial type, favouring the video. The success of the video tutorial is ascribed to its design, which attended to and even incorporated key qualities of paper-based tutorials, while also capitalizing on the strengths of video.

The effects of reviews in video tutorials

This study investigates how well a video tutorial for software training that is based on Demonstration-Based Teaching supports user motivation and performance. In addition, it is studied whether reviews significantly contribute to these measures. The Control condition employs a tutorial with instructional features added to a dynamic task demonstration. The Review condition additionally includes video reviews. Participants were 55 seventh graders who viewed task demonstrations and reviews followed by practice. Both tutorials increased motivation i.e., task relevance and self-efficacy and performance. In addition, the Review condition had significantly better results for training time, self-efficacy and scores on an immediate post-test. Reviews have rarely been studied in dynamic visualizations. The present study suggests that there may be important advantages to be gained from concluding a demonstration video with a summary of the main points.

Self-assessed and actual Internet skills of people with visual impairments

The Internet can make available to people with a visual impairment information and services that are otherwise inaccessible. But do visually impaired users actually use common Internet applications and do they have the necessary skills? This article reports a two-part study addressing these questions. The first part was an interview study in which 73 young and 69 older Dutch people with a visual impairment were questioned about usage of applications such as e-mail, chat, and web forms, and their self-perceived competence. The young participants reported more frequent use of Internet applications and mentioned multiple goals (i.e., social and educational), compared to the older. Both groups considered themselves reasonably competent, with the young rating themselves higher. The second part was a case study with 20 young and 20 older participants from the first study, who performed common Internet tasks, using websites or applications that complied with accessibility guidelines. Task performance was analyzed in detail for demonstrated skills. Actual performance proved to be unrelated to self-rated competence. Moreover, the competence of both young and older participants fell far short of what active participation in society requires, especially for the more complex information and strategic skills. The success rate on the performance tasks was low. People with a visual impairment should receive extensive support for the acquisition of higher-level skills that are called upon when using Internet information and services in order to participate in society.

A Closer Look at Visual Manuals

This article examines the genre of visual manuals by discussing the main forms and functions of two types in detail: step-by-step and guided tour manuals. Step-by-step manuals have a one-on-one correspondence between picture and text (explanations and instructions), reflecting the action-reaction mode in which users tend to interact with computers. Guided tour manuals give users a visual impression of the program. The pictures, mostly full-screen captures, are annotated with several paragraphs of text. An experiment is reported in which we examined whether a visual manual helps users realize tasks faster and more accurately than a non-visual manual. No effects on accuracy were found, but the visual manual did increase the speed of task execution with a significant and substantial gain of 35 percent. The conclusion draws attention to the fact that there is no single best type of visual manual, but that each has its own strengths and weaknesses.This article examines the genre of visual manuals by discussing the main forms and functions of two types in detail: step-by-step and guided tour manuals. Step-by-step manuals have a one-on-one correspondence between picture and text (explanations and instructions), reflecting the action-reaction mode in which users tend to interact with computers. Guided tour manuals give users a visual impression of the program. The pictures, mostly full-screen captures, are annotated with several paragraphs of text. An experiment is reported in which we examined whether a visual manual helps users realize tasks faster and more accurately than a non-visual manual. No effects on accuracy were found, but the visual manual did increase the speed of task execution with a significant and substantial gain of 35 percent. The conclusion draws attention to the fact that there is no single best type of visual manual, but that each has its own strengths and weaknesses.

Do Pedagogical Agents Enhance Software Training

This study investigates whether a tutorial for software training can be enhanced by adding a pedagogical agent, and whether the type of agent matters (i.e., cognitive, motivational, or mixed). The cognitive agent was designed to stimulate students to process their experiences actively. The motivational agent was designed to increase perceived task relevance and self-efficacy beliefs. A mixed agent combined these features. Process and product data were recorded during and after software training of students from the upper grades of vocational education (M age = 16.2 years). Comparison of scores on performance measures during training revealed a significant advantage of working with the motivational and mixed agents for two important motivational mediators for learning (i.e., strategy systematicity and mood). All students were highly successful during training, improving from an average 30% task completion score on the pretest to a 77% posttest score. On a retention measure 3 weeks later, task completion was still at 66%. Working with the motivational and control agents yielded significantly higher retention scores, whereas working with the motivational and mixed agents led to significantly higher scores on task relevance and self-efficacy beliefs after training. The discussion reflects on the possibilities for improving the internal and external properties of the agents.This study investigates whether a tutorial for software training can be enhanced by adding a pedagogical agent, and whether the type of agent matters i.e., cognitive, motivational, or mixed. The cognitive agent was designed to stimulate students to process their experiences actively. The motivational agent was designed to increase perceived task relevance and self-efficacy beliefs. A mixed agent combined these features. Process and product data were recorded during and after software training of students from the upper grades of vocational education M age = 16.2 years. Comparison of scores on performance measures during training revealed a significant advantage of working with the motivational and mixed agents for two important motivational mediators for learning i.e., strategy systematicity and mood. All students were highly successful during training, improving from an average 30% task completion score on the pretest to a 77% posttest score. On a retention measure 3 weeks later, task completion was still at 66%. Working with the motivational and control agents yielded significantly higher retention scores, whereas working with the motivational and mixed agents led to significantly higher scores on task relevance and self-efficacy beliefs after training. The discussion reflects on the possibilities for improving the internal and external properties of the agents.