Michael E. Martinez

A Review of Automatically Scorable Constructed-Response Item Types for Large-Scale Assessment

The use of automated scanning of test sheets, beginning in the 1930s, led to widespread use of the multiple-choice format in standardized testing. New forms of automated scoring now hold out the possibility of making a wide range of constructed-response item formats feasible for use on a large-scale basis. We describe new developments in five domains: mathematical reasoning, algebra problem solving, computer science, architecture, and natural language. For each one, we describe the task as presented to the examinee, the methods used to score the response, and the psychometric properties of the item responses. We then highlight general challenges and issues spanning these technologies. We conclude by offering our views on the ways in which such technologies are likely to shape the future of testing.

Human Memory: The Basics:

Since learning and remembering are closely linked, teachers can benefit from understanding the fundamentals of human memory.

Automated Scoring of Paper-and-Pencil Figural Responses

Large-scale testing is dominated by the multiple-choice question format. Widespread use of the format is explained, in part, by the ease with which multiple-choice items can be scored automatically. This article examines automated scoring procedures for an alternative item type: figural response. Figural response items call for the completion or modification of figural material, including illustrations, diagrams, and graphs. Nineteen science items were written in cooperation with the National Assessment of Educational Progress and printed with a special ink, invisible to scanning equipment. The items were answered with pencils; response sheets were then scanned and the resulting data were processed by computer-based scoring algorithms. Implications of this technology for the future of large-scale testing are discussed.

A REVIEW OF AUTOMATICALLY SCORABLE CONSTRUCTED-RESPONSE ITEM TYPES FOR LARGE-SCALE ASSESSMENT

The use of automated scanning of test sheets, beginning in the 1930s, led to widespread use of the multiple-choice format in standardized testing. New forms of automated scoring now hold out the possibility of making a wide range of constructed-response item formats feasible for use on a large-scale basis. We describe new developments in five domains: mathematical reasoning, algebra problem solving, computer science, architecture, and natural language. For each one, we describe the task as presented to the examinee, the methods used to score the response, and the psychometric properties of the item responses. We then highlight general challenges and issues spanning these technologies. We conclude by offering our views on the ways in which such technologies are likely to shape the future of testing.