Rod D. Roscoe

What Is Successful Writing? An Investigation Into the Multiple Ways Writers Can Write Successful Essays

This study identifies multiple profiles of successful essays via a cluster analysis approach using linguistic features reported by a variety of natural language processing tools. The findings from the study indicate that there are four profiles of successful writers for the samples analyzed. These four profiles are linguistically distinct from one another and demonstrate that expert human raters examine a number of different linguistic features in a variety of combinations when assessing writing proficiency and assigning high scores to independent essays (regardless of the scoring rubric considered). The writing styles in the four clusters can be described as action and depiction style, academic style, accessible style, and lexical style. The study provides empirical evidence that successful writing cannot be defined simply through a single set of predefined features, but that, rather, successful writing has multiple profiles. While these profiles may overlap, each profile is distinct.

Using Automated Indices of Cohesion to Evaluate an Intelligent Tutoring System and an Automated Writing Evaluation System

We present an evaluation of the Writing Pal (W-Pal) intelligent tutoring system (ITS) and the W-Pal automated writing evaluation (AWE) system through the use ofcomputational indices related to text cohesion. Sixty-four students participated in this study. Each student was assigned to either the W-Pal ITS condition or the W-Pal AWE condition. The W-Pal ITS includes strategy instruction, game-based practice, and essay-based practice with automated feedback. In the ITS condition, students received strategy training and wrote and revised one essay in each of the 8 training sessions. In the AWE condition, students only interacted with the essay writing and feedback tools. These students wrote and revised two essays in each of the 8 sessions. Indices of local and global cohesion reported by the computational tools Coh-Metrix and the Writing Assessment Tool (WAT) were used to investigate pretest and posttest writing gains. For both the ITS and the AWE systems, training led to the increased use of global cohesion features in essay writing. This study demonstrates that automated indices of text cohesion can be used to evaluate the effects of ITSs and AWE systems and further demonstrates how text cohesion develops as a result of instruction, writing, and automated feedback.

Online information search and decision making

A naturalistic online information search exposes individuals to multiple sites and conflicting perspectives. In this study, we evaluated how the holistic stance of a web search toward a product influences purchasing decisions. We recruited 109 participants who completed an initial product choice task regarding bottled water, a brief Internet search, and then a second post-search product choice task. Internet searches were analyzed to identify query terms, site visits, and stance. Results show that query terms influenced the types of sites obtained in a search, which in turn shaped the overall search stance. Participants were more likely to buy bottled water when they visited websites that emphasized environmental, economic, or health benefits for bottled water (i.e., positive stance). Participants who were asked to focus on environmental issues were less likely to buy bottled water unless packaged in recycled plastic. Effects of online search and mixed messages on decision making are investigated.Searches focused on superficial product features rather than key knowledge.Superficial search behaviors resulted in biased web searches.The holistic stance of a brief web search influenced decision making.Goal-oriented messaging influenced both search behaviors and decisions.

Feedback and Revising in an Intelligent Tutoring System for Writing Strategies

This study investigates students’ essay revising in the context of an intelligent tutoring system called Writing Pal (W-Pal), which combines strategy instruction, game-based practice, essay writing practice, and automated formative feedback. We examine how high school students use W-Pal feedback to revise essays in two different contexts: a typical approach that emphasizes intensive writing practice, and an alternative approach that offers less writing practice with more direct strategy instruction. Results indicate that students who wrote fewer essays, but received W-Pal strategy instruction, were more likely to make substantive revisions that implemented specific recommendations conveyed by the automated feedback. Additional analyses consider the role of motivation and perceived learning on students’ revising behaviors.

The hfes diversity committee: Challenges and opportunities for involvement

The Human Factors and Ergonomics Society Diversity Committee met initially in January 2017, and on a regular basis thereafter to assess and improve diversity and inclusion in the society, profession, and discipline. Charged by president Bill Marras in 2016, the Committee replaced the Diversity Task Force established in 1994, and formally became a part of the Society’s Internal Affairs Division. As part of the Committee’s initial outreach, this panel will continue the conversation with society members about diversity and inclusion. Panelists will provide an overview of the Committee’s charter, review historical trends and current status of member diversity, discuss previous successful programming for enhancing diversity and feature perspectives from students and professionals in academia, industry, government/military/aerospace and consulting. Committee members welcome feedback and input from attendees on the above topics, what inclusion and diversity means and how to balance the broader concept of diversity with focused activities for targeted groups.

Automated Writing Instruction and Feedback: Instructional Mode, Attitudes, and Revising

This study evaluates high school students’ perceptions of automated writing feedback, and the influence of these perceptions on revising, as a function of varying modes of computer-based writing instruction. Findings indicate that students’ perceptions of automated feedback accuracy, ease of use, relevance, and understandability were favorable. Immediate perceptions of feedback received on a selected essay were minimally related to how and whether students revised their essays. However, attitudes formed over multiple sessions were significantly related to revising. More importantly, the mode of instruction appeared to influence how feedback perceptions shaped revising behaviors. Students who engaged in traditional writing-based training and practice seemed to focus on their own perceived writing abilities when deciding how to revise. In contrast, students who also received strategy instruction and game-based practice attended more carefully to the perceived quality of the automated feedback.

Human Factors and Ergonomics in Diversity, Inclusion and Social Justice Research

The Human Factors and Ergonomics Society Diversity Committee is entering its second year and continuing to explore ways to increase the diversity of the society. Following last year’s panel on “Challenges and Opportunities for Involvement,” we, and others, recognized that human factors and ergonomics (HFE) professionals are equipped and able to advance diversity, inclusion and social justice issues. This panel will bring together researchers to discuss experiences addressing these issues to highlight existing work, spark excitement about conducting new work and share advice. This panel will discuss and share lessons learned in a range of projects, including an HFE approach to studying diversity in academia and applications of user-centered design to address the intersection of technology and bias. Macroergonomics contributes important tools and approaches; the need for collaboration with other disciplines and intersectional thinking will be considered. Finally, venues for scholarship in this area of work will be highlighted.

A Conceptual Qualitative Framework for Assessing Human Systems Engineering Education Outcomes and Opportunities

Human systems engineering education seeks to infuse principles of applied psychology, cognitive science, human factors, and user-centered design into the engineering curriculum to help students understand the people they are engineering for (e.g., clients) and their own roles as engineers. This paper outlines a conceptual qualitative framework for formative assessment of students’ incorporation of human systems engineering concepts in their projects and documentation. The framework examines potential conceptual dimensions along with applications, sources, and depth of such knowledge, which we argue can begin to evaluate students’ work and inform iterative efforts to improve human-centered engineering education programs. Example applications of the framework based on several data sets are discussed.

Exploring Higher Agency Roles for Learning with Educational Technology and Multimedia

Educational technologies with multimedia content can support effective learning, but these outcomes are moderated by learners’ level of cognitive engagement or self-regulation. As a way to encourage deeper cognitive engagement without redesigning or redeveloping software (e.g., building more prompts, scaffolds, or automated support), this study investigates changing the role of the student user. Specifically, this research considers how instituting a “designer” or “teacher” role may inspire better engagement and learning than the default “learner” role. We present the theoretical background, design, and results of an exploratory study of this hypothesis with college students learning about cohesion in writing.

Contrasting Writing Practice Formats in a Writing Strategy Tutoring System

A critical challenge for computer-based writing instruction is providing appropriate and adaptive practice. The current study examined three modes of computer-based writing practice with the goal of identifying those with the greatest learning and motivational value. High school students learned about writing strategies by studying lessons within the Writing Pal tutoring system and then practiced relevant strategies via essay-based practice, strategy practice, or game-based strategy practice. Students acquired strategy knowledge regarding their assigned topics, but there were no main effects of practice format. Similar findings were observed for students’ beliefs about the value of writing practice. However, the effects of practice format depended on prior literacy ability in subtle ways. Essay-based practice appeared to be more effective for skilled readers, whereas less skilled readers benefitted more from game-based practice. Overall, multiple forms of practice opportunities may optimize benefits, although non-game forms of strategy practice are less preferable to students than game-based formats.