Dennis Quardt

Three Response Types for Broadening the Conception of Mathematical Problem Solving in Computerized Tests

Three open-ended response types—mathematical expression (ME), generating examples (GE), and graphical modeling (GM)—are described that could broaden the conception of mathematical problem solving used in computerized admissions tests. ME presents single-best-answer problems that call for an algebraic formalism, the correct rendition of which can take an infinite number of surface forms. GE presents loosely structured problems that can have many good answers taking the form of a value, letter pattern, expression, equation, or list. GM asks the examinee to represent a given situation by plotting points on a grid; these items can have a single best answer or multiple correct answers. For the three basic types, sample items are provided, the examinee interfaces and approaches to automated scoring are described, and research results are reported. It is illustrated how ME, GE, and GM can be combined to form extended constructed-response problems, and adescription is offered of how item classes might be used as a basis for creating production-ready scoring keys.

Graphical Modeling: A New Response Type for Measuring the Qualitative Component of Mathematical Reasoning

We investigated the functioning of a new computer-delivered response type for potential use in graduate admissions assessment. This response type, which is open-ended and automatically scorable, presents problems calling for the examinee to draw a graph modeling a given situation. Problem situations can be like the single-best-answer items currently found on the Graduate Record Examinations (GRE) General Test (ETS, 1998) or they can be more loosely defined, allowing for multiple-correct responses. Two graphical modeling (GM) tests differing from one another in the manipulation of specific item features were randomly spiraled among study participants. Results showed that GM scores were very reliable and moderately related to the General Tests quantitative section, suggesting that GM might help broaden the GRE quantitative construct. In exploratory difficulty analyses, 1 of 3 manipulated item features, problem structure, had a dependable effect. No significant gender differences independent of those associated with the GRE quantitative section were detected. Finally, more participants preferred regular multiple-choice graphical reasoning questions to GM items but thought GM was the fairer indicator of their ability to undertake graduate study.

THREE RESPONSE TYPES FOR BROADENING THE CONCEPTION OF MATHEMATICAL PROBLEM SOLVING IN COMPUTERIZED‐ADAPTIVE TESTS

We describe three open-ended response types that should broaden the conception of mathematical problem solving embodied in computerized-adaptive admissions tests. The Mathematical Expressions response type presents single-best answer problems that call for an algebraic formalism, the correct rendition of which may take an infinite number of surface forms. Generating Examples presents loosely structured problems which may have many good answers, taking the form of a value, letter pattern, expression, equation, or list. Graphical Modeling problems ask the examinee to represent a given situation by plotting points on a grid, where there may be a single-best answer or multiple correct responses. Finally, these three basic types can be combined to form extended constructed-response problems. For each of the three basic types, this paper gives example items, describes the examinee interface and approach to automatic scoring, and reports available findings.