S. Jeanne Horst

Motivation Matters: Using the Student Opinion Scale to Make Valid Inferences About Student Performance

Institutions of higher education are expected to address accountability and transparency of student learning and development efforts. The Student Opinion Scale assesses examinee motivation during testing and asks students to report level of effort invested and perceived importance of tasks. This article evaluates validity evidence and supports widespread instrument use.

Do Examinees Have Similar Test-Taking Effort? A High-Stakes Question for Low-Stakes Testing

Given the prevalence of low-stakes testing internationally (e.g., NAEP, TIMSS, PIRLS), it is crucial to try to better understand examinee motivation in these contexts. In the current study, mixture modeling results supported three different profiles of test-taking effort over the course of five tests. Classes 1 and 2 had varying levels of effort whereas Class 3 had steady levels of effort across the five tests. Examination of relationships between class membership and external variables suggested the classes could be differentiated by achievement goals, personality, and ability. Implications of these results for international assessment and directions for future research are discussed.

The Effect of Propensity Score Matching Method on the Quantity and Quality of Matches

ABSTRACT Often program administrators are interested in knowing how students benefit from participation in programs compared to students who do not participate. Such comparisons may be sullied by the fact that participants self-select into programs, resulting in differences between groups prior to programming. By controlling for researcher-identified–self-selection variables, propensity score matching enables researchers to create comparable matched groups. However, when employing propensity score matching, researchers are faced with a plethora of matching options. In the current study, we compared the quantity and quality of matches obtained when applying common matching techniques to real data. The methods produced matches of varying quantity and quality. Moreover, group comparisons on the outcome led to different conclusions depending on the matching method employed.