Kwangsu Cho

Scaffolded writing and rewriting in the discipline: A web-based reciprocal peer review system

This paper describes how SWoRD (scaffolded writing and rewriting in the discipline), a web-based reciprocal peer review system, supports writing practice, particularly for large content courses in which writing is considered critical but not feasibly included. To help students gain content knowledge as well as writing and reviewing skills, SWoRD supports the whole cycle of writing, reviews, back-reviews, and rewriting by scaffolding the journal publication process as its authentic practice model. In addition, SWoRD includes algorithms that compute individual reviewers review accuracy, which is in turn used to support the various drawbacks of reciprocal peer reviews (e.g., variation in motivation or ability of reviewers). Finally, this paper describes an empirical evaluation showing that the SWoRD approach is effective in improving writing quality in content classes.

Validity and reliability of scaffolded peer assessment of writing from instructor and student perspectives

Although peer reviewing of writing is a way to create more writing opportunities in college and university settings, the validity and reliability of peer-generated grades are a major concern. This study investigated the validity and reliability of peer-generated writing grades of 708 students across 16 different courses from 4 universities in a particular scaffolded reviewing context: Students were given guidance on peer assessment, used carefully constructed rubrics, and were provided clear incentives to take the assessment task seriously. Distinguishing between instructor and student perspectives of reliability and validity, the analyses suggest that the aggregate ratings of at least 4 peers on a piece of writing are both highly reliable and as valid as instructor ratings while (paradoxically) producing very low estimates of reliability and validity from the student perspective. The results suggest that instructor concerns about peer evaluation reliability and validity should not be a barrier to implementing peer evaluations, at least with appropriate scaffolds. Future research needs to investigate how to address student concerns about reliability and validity and to identify scaffolds that may ensure high levels of reliability and validity.

Commenting on Writing: Typology and Perceived Helpfulness of Comments from Novice Peer Reviewers and Subject Matter Experts.

How do comments on student writing from peers compare to those from subject-matter experts? This study examined the types of comments that reviewers produce as well as their perceived helpfulness. Comments on classmates’ papers were collected from two undergraduate and one graduate-level psychology course. The undergraduate papers in one of the courses were also commented on by an independent psychology instructor experienced in providing feedback to students on similar writing tasks. The comments produced by students at both levels were shorter than the instructor’s. The instructor’s comments were predominantly directive and rarely summative. The undergraduate peers’ comments were more mixed in type; directive and praise comments were the most frequent. Consistently, undergraduate peers found directive and praise comments helpful. The helpfulness of the directive comments was also endorsed by a writing expert.

Peer-based computer-supported knowledge refinement: an empirical investigation

Nonexpert peer-based knowledge refinement, it turns out, is just as helpful as expert-centric knowledge refinement for improving the quality of results.

Learning writing by reviewing in science

We examined a theoretical perspective on reciprocal peer reviewing of writing. As an alternative to the traditional approach, Learning Writing by Writing, focusing on increasing writing opportunities, we proposed and tested a new hypothesis, Learning Writing by Reviewing. Reviewing is defined as a problem solving activity of practicing problem detection, diagnosis, and solution generation in peer writing. The results supported the hypothesis in that peer reviewers improve their own writing by reviewing peer writing.

The SWoRD is mightier than the pen: scaffolded writing and rewriting in the discipline

This paper describes a Web-based reciprocal peer review system called scaffolded writing and rewriting in the discipline (SWoRD), addressing its approaches to writing and rewriting as a sound function of peer reviews. It also provides an empirical evaluation showing that reciprocal peer reviews through SWoRD are very effective in improving writing quality in content classes.