Jon M. Pearce

Peer review in higher education: Student perceptions before and after participation:

Peer review is integral to academic endeavour, but opportunities for students to benefit from peer review in higher education remain limited, and relatively little is known about how student perceptions influence their appreciation of peer review. University student perceptions were examined before and after experiencing student peer review in four university subjects differing in discipline, year level and class size. Prior to engaging in peer review, students had remarkably high expectations of both the process and the competence of their peers as reviewers. Students reported high satisfaction levels with the peer-review process and its positive impact on their learning, and particularly showed an enhanced appreciation of the influence of review writing on learning. However, a modest overall downward shift in positive perceptions was observed following peer review, suggesting that student opinions of peer review became somewhat more polarized as a consequence of their experiences. Drawing on survey response data and student comments, several recommendations are made that may help to improve student perceptions of peer review and ensure that the diverse learning benefits of the peer-review process are achieved.

How does student peer review influence perceptions, engagement and academic outcomes? A case study

Involving students in peer review has many pedagogical benefits, but few studies have explicitly investigated relationships between the content of peer reviews, student perceptions and assessment outcomes. We conducted a case study of peer review within a third-year undergraduate subject at a research-intensive Australian university, in which we examined: (1) students’ perceptions of the peer review process before and after peer review, (2) content of the peer reviews and what kinds of feedback were adopted and (3) the effect of participation in peer review on performance (grades) in the assessment task. Students overwhelmingly perceived peer review to be beneficial, and the opportunity to participate in peer review resulted in a significant improvement in the quality of work submitted for assessment. Students who benefited most from peer review were those of below-median performance, and the magnitude of benefit was related to the degree to which students engaged with the peer review process. Our study confirms that participation in peer review can lead to important improvements in performance and learning outcomes.

Sharing designer and user perspectives of web site evaluation: a cross-campus collaborative learning experience

In this paper we present an online, collaborative process that facilitates usability evaluation of web sites. The online workspace consists of simple and effective proformas and computer-mediated discussion space to support usability evaluation. The system was designed and used by staff and students at two universities. Students, working in small teams, at each university, developed web sites and then evaluated the usability of web sites developed at the other university, using the results to improve their own sites. Our project evaluations show that the process provides valuable feedback on web site usability and provides students with the experience of usability evaluation from two important perspectives: those of a user and of a developer. Further, students develop important generic skills: the ability to participate in and critique computer supported cooperative work environments.

Designing for Flow in a Complex Activity

One component of a user’s interaction with computer systems is commonly referred to as ’flow’. Flow is an important consideration in interactive system design as it encapsulates some of the affective aspects of human behavior. The majority of current thinking conceptualises flow as a desirable and somewhat enduring emotional state that a user may enter during an activity. Analysis of data from 59 users engaged in an interactive online learning task contradicts this prevailing view. We show firstly that flow, rather than being enduring, is highly changeable during the term of an interaction. This challenges both current theoretical models of flow, and the current research methodology used to study the phenomenon. Secondly, we show that flow arises from an engagement either with the interactive artefact or the task being performed. This is an aspect of flow not well distinguished in other studies. Finally, we present initial analysis that suggests flow can be undesirable in some circumstances – that there may be competition between task and artefact for the attention of the user. In response, we present a ’process’ view of flow as a counterpoint to the existing ’state’ based models.

Search or explore: do you know what you're looking for?

This paper explores the distinctions between searching and exploring when looking for information. We propose that, while traditional search engines work well in supporting search behaviour, they are more limited in assisting those who are looking to explore new information, especially when the exploration task is ill-defined. We ran a pilot study using two systems: one based on a traditional database search engine, and the other -- a highly innovative, engaging and playful system called iFISH -- that we designed specifically to support exploration through the use of user preferences. We looked for evidence to support the concept that exploration requires a different kind of interaction. The initial results report a positive response to our exploration system and indicate the differences in preferences amongst users for systems that match their searching or exploring behaviours.

Exploration without keywords: the bookfish case

As a generation of information seekers we tend to think in terms of targeted search results, yet there are some contexts in which a more exploratory behaviour might be desirable. This paper presents the design and development of a playful Web app called Bookfish that seeks to encourage exploratory behaviour amongst children wishing to discover new books. Bookfish sets out to return more open, or even unexpected, serendipitous results through using an approach designed for exploration rather than search. The paper reports on the design considerations and initial user expectations of such a system and contributes both through this discussion of design as well as the application of a novel exploration environment. Strategies for managing these design challenges are also discussed.

Things you don't want to know about yourself: ambivalence about tracking and sharing personal information for behaviour change

Technologies that facilitate the collection and sharing of personal information can feed peoples desire for enhanced self-knowledge and help them to change their behaviour, yet for various reasons people can also be reluctant to use such technologies. This paper explores this tension through an interview study in the context of smoking cessation. Our findings show that smokers and recent ex-smokers were ambivalent about their behaviour change as well as about collecting personal information through technology and sharing it with other users. We close with a summary of three challenges emerging from such ambivalence and directions to address them.

SmartGardenWatering: experiences of using a garden watering simulation

SmartGardenWatering is an innovative software tool that advises gardeners on watering schedules and watering use. In this paper we investigate how expert and novice gardeners respond to advice from this piece of computer software. Do they readily accept it and adapt their activities accordingly, or do they override it with their own local knowledge? We describe the project to develop the simulation, including the design of the user interface, and a study of 20 gardeners using the tool. The focus of the study was to identify factors in the design of the software that influence how well it might intervene in ongoing gardening practice. The findings focus on what brings confidence or a lack of trust in the underlying horticultural model and its application to a particular garden. Finally, we consider how these findings might inform ongoing development of the software.

Designing Health Websites Based on Users’ Web-Based Information-Seeking Behaviors: A Mixed-Method Observational Study

Background Laypeople increasingly use the Internet as a source of health information, but finding and discovering the right information remains problematic. These issues are partially due to the mismatch between the design of consumer health websites and the needs of health information seekers, particularly the lack of support for “exploring” health information. Objective The aim of this research was to create a design for consumer health websites by supporting different health information–seeking behaviors. We created a website called Better Health Explorer with the new design. Through the evaluation of this new design, we derive design implications for future implementations. Methods Better Health Explorer was designed using a user-centered approach. The design was implemented and assessed through a laboratory-based observational study. Participants tried to use Better Health Explorer and another live health website. Both websites contained the same content. A mixed-method approach was adopted to analyze multiple types of data collected in the experiment, including screen recordings, activity logs, Web browsing histories, and audiotaped interviews. Results Overall, 31 participants took part in the observational study. Our new design showed a positive result for improving the experience of health information seeking, by providing a wide range of information and an engaging environment. The results showed better knowledge acquisition, a higher number of page reads, and more query reformulations in both focused and exploratory search tasks. In addition, participants spent more time to discover health information with our design in exploratory search tasks, indicating higher engagement with the website. Finally, we identify 4 design considerations for designing consumer health websites and health information–seeking apps: (1) providing a dynamic information scope; (2) supporting serendipity; (3) considering trust implications; and (4) enhancing interactivity. Conclusions Better Health Explorer provides strong support for the heterogeneous and shifting behaviors of health information seekers and eases the health information–seeking process. Our findings show the importance of understanding different health information–seeking behaviors and highlight the implications for designers of consumer health websites and health information–seeking apps.

Better Health Explorer: Designing for Health Information Seekers

A vast amount of health information has been published online, yet users often report difficulties in locating information in this particular domain. Based on our prior research, we consider four categories of online health information seekers who demonstrate mixed information needs. Although their searching needs are often well satisfied by entering keywords into search engines, their need to explore information is not so well supported, thus affecting their user experience and satisfaction. In this paper, we propose design principles for supporting the exploration of online health information. We present the rationale and the design process of a web app -- Better Health Explorer -- which is a proof-of-concept app tailored to health information exploration. This work contributes to the design of online health information systems as well as exploratory systems in general.