Guangming Ling

Using the Standardized Letters of Recommendation in Selection: Results from a Multidimensional Rasch Model.

In an effort to standardize academic application procedures, the authors developed the Standardized Letters of Recommendation (SLR) to capture important cognitive and noncognitive qualities of graduate school candidates. The SLR, which consists of seven scales, is applied to an intern-selection scenario. Both professor ratings (n = 414) during the application process and mentor ratings of the selected students (n = 51) are collected using the SLR. A multidimensional Rasch investigation suggests that the SLR displays satisfactory internal consistency, model fit, and item fit. The two cognitive scales, knowledge and analytical skills, are found to be the best predictors for intern selection. The professor ratings are systematically higher than the mentor ratings. Possible reasons for the rating discrepancies are discussed. Also, implications for how the SLR can be used and improved in other selection situations are suggested.

Examining the Validity of Standards-Based Assessments for Initially Fluent Students and Former English Language Learners

To date, assessment validity research on non-native English speaking students in the United States has focused exclusively on those who are presently English language learners (ELLs). However, little, if any, research has been conducted on two other sizable groups of language minority students: (a) bilingual or multilingual students who were already English proficient when they entered the school system (IFEPs), and (b) former English language learners, those students who were once classified as ELLs but are now reclassified as being English proficient (RFEPs). This study investigated the validity of several standards-based assessments in mathematics and science for these two student groups and found a very high degree of score comparability, when compared with native English speakers, for the IFEPs, whereas a moderate to high degree of score comparability was observed for the RFEPs. Thus, test scores for these two groups on the assessments we studied appear to be valid indicators of their content knowledge, to a degree similar to that of native English speakers.

Is a Computerized Adaptive Test More Motivating Than a Fixed-Item Test?:

Computer adaptive tests provide important measurement advantages over traditional fixed-item tests, but research on the psychological reactions of test takers to adaptive tests is lacking. In particular, it has been suggested that test-taker engagement, and possibly test performance as a consequence, could benefit from the control that adaptive tests have on the number of test items examinees answer correctly. However, previous research on this issue found little support for this possibility. This study expands on previous research by examining this issue in the context of a mathematical ability assessment and by considering the possible effect of immediate feedback of response correctness on test engagement, test anxiety, time on task, and test performance. Middle school students completed a mathematics assessment under one of three test type conditions (fixed, adaptive, or easier adaptive) and either with or without immediate feedback about the correctness of responses. Results showed little evidence for test type effects. The easier adaptive test resulted in higher engagement and lower anxiety than either the adaptive or fixed-item tests; however, no significant differences in performance were found across test types, although performance was significantly higher across all test types when students received immediate feedback. In addition, these effects were not related to ability level, as measured by the state assessment achievement levels. The possibility that test experiences in adaptive tests may not in practice be significantly different than in fixed-item tests is raised and discussed to explain the results of this and previous studies.