Christian D. Schunn

Scaffolded writing and rewriting in the discipline: A web-based reciprocal peer review system

This paper describes how SWoRD (scaffolded writing and rewriting in the discipline), a web-based reciprocal peer review system, supports writing practice, particularly for large content courses in which writing is considered critical but not feasibly included. To help students gain content knowledge as well as writing and reviewing skills, SWoRD supports the whole cycle of writing, reviews, back-reviews, and rewriting by scaffolding the journal publication process as its authentic practice model. In addition, SWoRD includes algorithms that compute individual reviewers review accuracy, which is in turn used to support the various drawbacks of reciprocal peer reviews (e.g., variation in motivation or ability of reviewers). Finally, this paper describes an empirical evaluation showing that the SWoRD approach is effective in improving writing quality in content classes.

The relationship of analogical distance to analogical function and preinventive structure: The case of engineering design

Analogy was studied in real-world engineering design, using the in vivo method. Analogizing was found to occur frequently, entailing a roughly equal amount of within- and between-domain analogies. In partial support for theories of unconscious plagiarism (Brown & Murphy, 1989; Marsh, Landau, & Hicks, 1996) and Ward’s (1994) path-of-least-resistance model, it was found that the reference to exemplars (in the form of prototypes) significantly reduced the number of between-domain analogies between source and target, as compared with using sketches or no external representational systems. Analogy served three functions in relation to novel design concepts: identifying problems, solving problems, and explaining concepts. Problem identifying analogies were mainly within domain, explanatory analogies were mainly between domain, and problem-solving analogies were a mixture of within- and between-domain analogies.

A mechanistic account of the mirror effect for word frequency: a computational model of remember-know judgments in a continuous recognition paradigm.

A theoretical account of the mirror effect for word frequency and of dissociations in the pattern of responding Remember vs. Know (R vs. K) for low- and high-frequency words was tested both empirically and computationally by comparing predicted with observed data theory in 3 experiments. The SAC (Source of Activation Confusion) theory of memory makes the novel prediction of more K responses for high- than for low-frequency words, for both old and new items. Two experiments used a continuous presentation and judgment paradigm that presented words up to 10 times. The computer simulation closely modeled the pattern of results, fitting new Know and Remember patterns of responding at each level of experimental presentation and for both levels of word frequency for each participant. Experiment 3 required list discrimination after each R response (Group 1) or after an R or K response (Group 2). List accuracy was better following R responses. All experiments were modeled using the same parameter values.

Validity and reliability of scaffolded peer assessment of writing from instructor and student perspectives

Although peer reviewing of writing is a way to create more writing opportunities in college and university settings, the validity and reliability of peer-generated grades are a major concern. This study investigated the validity and reliability of peer-generated writing grades of 708 students across 16 different courses from 4 universities in a particular scaffolded reviewing context: Students were given guidance on peer assessment, used carefully constructed rubrics, and were provided clear incentives to take the assessment task seriously. Distinguishing between instructor and student perspectives of reliability and validity, the analyses suggest that the aggregate ratings of at least 4 peers on a piece of writing are both highly reliable and as valid as instructor ratings while (paradoxically) producing very low estimates of reliability and validity from the student perspective. The results suggest that instructor concerns about peer evaluation reliability and validity should not be a barrier to implementing peer evaluations, at least with appropriate scaffolds. Future research needs to investigate how to address student concerns about reliability and validity and to identify scaffolds that may ensure high levels of reliability and validity.

A Study of Design Fixation, Its Mitigation and Perception in Engineering Design Faculty

The bridge between engineering design and cognitive science research is critical to understand the effectiveness of design methods as implemented by human designers. The study reported in this paper evaluates the effects of design fixation in a group of engineering design faculty, and also provides evidence for approaches to overcome design fixation. Three conditions are compared, a control, a fixation group whom were provided with an example solution, and a defixation group whom were also given materials to mitigate their design fixation. Measures include indicators of design fixation and participant perceptions. The study demonstrates that the engineering design faculty show statistically significant evidence of design fixation, but only partially perceive its effects. This study also indicates that design fixation can be mitigated. The group of participants in this study, due to their background in engineering design research and experience with student design teams, was expected to have more accurate perceptions or awareness of design fixation than the typical participant. Understanding the incongruities between participant perceptions and quantitative design outcomes are particularly of interest to researchers of design methods. For this study, clear evidence exists that designers, even those that study and teach design on a regular basis, do not know when they are being influenced or fixated by misleading or poor information. DOI: 10.1115/1.4001110

Commenting on Writing: Typology and Perceived Helpfulness of Comments from Novice Peer Reviewers and Subject Matter Experts.

How do comments on student writing from peers compare to those from subject-matter experts? This study examined the types of comments that reviewers produce as well as their perceived helpfulness. Comments on classmates’ papers were collected from two undergraduate and one graduate-level psychology course. The undergraduate papers in one of the courses were also commented on by an independent psychology instructor experienced in providing feedback to students on similar writing tasks. The comments produced by students at both levels were shorter than the instructor’s. The instructor’s comments were predominantly directive and rarely summative. The undergraduate peers’ comments were more mixed in type; directive and praise comments were the most frequent. Consistently, undergraduate peers found directive and praise comments helpful. The helpfulness of the directive comments was also endorsed by a writing expert.

The Generality/Specificity of Expertise in Scientific Reasoning

Previous research on scientific reasoning has shown that it involves a diverse set of skills. Yet, little is known about generality or domain specificity of those skills, an important issue in theories of expertise and in attempts to automate scientific reasoning skills. We present a study designed to test what kinds of skills psychologists actually use in designing and interpreting experiments and contrast expertise within a particular research area with general expertise at designing and interpreting experiments. The results suggest that psychologists use many domain-general skills in their experimentation and that bright and motivated Carnegie Mellon undergraduates are missing many of these skills. We believe that these results have important implications for psychological and artificial intelligence models of expertise, as well as educational implications in the domain of science instruction.

On the Benefits and Pitfalls of Analogies for Innovative Design: Ideation Performance Based on Analogical Distance, Commonness, and Modality of Examples

Drawing inspiration from examples by analogy can be a powerful tool for innovative design during conceptual ideation but also carries the risk of negative design outcomes (e.g., design fixation), depending on key properties of examples. Understanding these properties is critical for effectively harnessing the power of analogy. The current research explores how variations in analogical distance, commonness, and representation modality influence the effects of examples on conceptual ideation. Senior-level engineering students generated solution concepts for an engineering design problem with or without provided examples drawn from the U.S. Patent database. Examples were crossed by analogical distance (near-field vs. far-field), commonness (more vs. less-common), and modality (picture vs. text). A control group that received no examples was included for comparison. Effects were examined on a mixture of ideation process and product variables. Our results show positive effects of far-field and less-common examples on novelty and variability in quality of solution concepts. These effects are not modulated by modality. However, detailed analyses of process variables suggest divergent inspiration pathways for far-field vs. less-common examples. Additionally, the combination of far-field, less-common examples resulted in more novel concepts than in the control group. These findings suggest guidelines for the effective design and implementation of design-by-analogy methods, particularly a focus on far-field, less-common examples during the ideation process.

Integrating perceptual and cognitive modeling for adaptive and intelligent human-computer interaction

This paper describes technology and tools for intelligent human-computer interaction (IHCI) in which human cognitive, perceptual, motor and affective factors are modeled and used to adapt the H-C interface. IHCI emphasizes that human behavior encompasses both apparent human behavior and the hidden mental state behind behavioral performance. IHCI expands on the interpretation of human activities, known as W4 (what, where, when, who). While W4 only addresses the apparent perceptual aspect of human behavior the W5+ technology for IHCI described in this paper addresses also the why and how questions, whose solution requires recognizing specific cognitive states. IHCI integrates parsing and interpretation of nonverbal information with a computational cognitive model of the user which, in turn, feeds into processes that adapt the interface to enhance operator performance and provide for rational decision-making. The technology proposed is based on a general four-stage interactive framework, which moves from parsing the raw sensory-motor input, to interpreting the users motions and emotions, to building an understanding of the users current cognitive state. It then diagnoses various problems in the situation and adapts the interface appropriately. The interactive component of the system improves processing at each stage. Examples of perceptual, behavioral, and cognitive tools are described throughout the paper Adaptive and intelligent HCI are important for novel applications of computing, including ubiquitous and human-centered computing.

Priming, analogy, and awareness in complex reasoning

The mechanisms by which a concept used in solving one complex task can influence performance on another complex task were investigated. We tested the hypothesis that even when subjects do not spontaneously make an analogy between two domains, knowledge of one domain can still spontaneously influence reasoning about the other domain via the mechanism of priming. Four groups of subjects (two experimental and two control) were given a simulated biochemistry problem on Day 1 and a simulated molecular genetics problem on Day 2. For the two experimental groups, the solution to the biochemistry problem involved inhibition. For the two control groups, the solution did not involve inhibition. On Day 2, all subjects received the same version of the molecular genetics problem in which the solution involved the concept of inhibition. Subjects in the experimental conditions were more likely to attain the correct answer, to propose inhibition, and to propose inhibition early in the problemsolving session than were subjects in the control conditions. However, subjects in the experimental conditions made no reference to the biochemistry problem either in their verbal protocols or in a posttask questionnaire. The results are interpreted as demonstrating that an implicit process—priming— can make old knowledge available for current problem solving.