Jamal Abedi

The Language Factor in Mathematics Tests

In this study we investigated the importance of language in student test performance on mathematics word problems. Students were given released items from the National Assessment of Educational Progress mathematics assessment, along with parallel items that were modified to reduce their linguistic complexity. In interviews, students typically preferred the revised items over the original counterparts. Paper-and-pencil tests containing original and revised items were administered to 1,174 8th grade students. Students who were English language learners (ELLs) scored lower on the math test than proficient speakers of English. There were also differences in math performance with respect to socioeconomic status (SES) but not gender. Linguistic modification of test items resulted in significant differences in math performance; scores on the linguistically modified version were slightly higher. Some student groups benefited more from the linguistic modification of items-in particular, students in low-level and average math classes, but also ELLs and low SES students.

The No Child Left Behind Act and English Language Learners: Assessment and Accountability Issues:

There are major issues involved with the disaggregated No Child Left Behind (NCLB) Act in terms of its adequate yearly progress reporting for students with limited English proficiency (LEP). Inconsistent LEP classification, as well as the sparse population of LEP students in many states, threatens the validity of adequate yearly progress reporting. The LEP subgroup’s lack of stability also threatens accountability, since students attaining English proficiency move out of the subgroup. The linguistic complexity of assessment tools may lower LEP student performance in areas with greater language demand. Finally, schools with larger numbers of LEP students with lower baselines may require greater gains. Thus, NCLB’s mandates may unintentionally place undue pressure on schools with high numbers of LEP students. Continuing efforts to remedy these issues should bring more fair assessment and accountability

Assessment Accommodations for English Language Learners: Implications for Policy-Based Empirical Research

Increased attention to large-scale assessments, the growing number of English language learners in schools, and recent inclusionary policies have collectively made assessment accommodations a hotly debated issue, especially regarding the validity of test results for English language learners. Decisions about which accommodations to use, for whom, and under what conditions, are based on limited empirical evidence for their effectiveness and validity. Given the potential consequences of test results, it is important that policy-makers and educators understand the empirical base underlying their use. This article reviews test accommodation strategies for English learners, derived from “scientifically based research.” The results caution against a one-size-fits-all approach. The more promising approaches include modified English and customized dictionaries, which can be used for all students, not just English language learners

Reliability and Validity of a State Metacognitive Inventory: Potential for Alternative Assessment.

Abstract An assumed advantage of alternative assessments is that they result in more higher level thinking or metacognitive skills. We believe that this advantage should be measured directly and explicitly. Unfortunately, few standardized measures of metacognitive skills (planning, monitoring, cognitive strategies, and awareness) exist. In our studies, for 12th graders, alpha reliability estimates and factor analysis indicated that our metacognitive subscales are reliable (alpha above .70) and unidimensional (one factor per subscale). Because the subscales have only 5 items each, they meet brevity standards. Construct validity of our state metacognitive inventory is acceptable. Results indicate that our state metacognitive inventory yields useful information about both the assessment and students.

The effects of students' academic, financial, and demographic variables on time to the doctorate

The time taken to earn doctorates is of major importance to administrators and faculty. By using stepwise multiple regression techniques, this study predicted time to doctorate from the available demographic, academic, and financial variables and determined the significance of each variable on time to the doctorate. The data for this study came from the National Research Councils Doctorate Records File extract prepared for UCLA. This institution is particularly appropriate for a study of this kind, since it annually awards over 400 doctorates in over 75 different majors. Its doctoral recipients represent the range of academic fields of study. The results of this study indicated that source of support was the most important variable in predicting time to doctorate. Following source of support were postdoctoral plans, number of dependents, sex, and field of study. Together these variables explained a significant amount of variation in the criterion variable.

Predicting Placement in Learning Disabilities Programs: Do Predictors Vary by Ethnic Group?:

Notwithstanding the historical persistence of the disproportionate representation of ethnic minority students in special education, there is a scarcity of research on factors affecting the placement of minority students in these programs. The purpose of this study was to identify placement predictors in learning disabilities (LD) programs for Latino, African-American, and Anglo students. We used 12 predictor variables from two key domains (student and family) and used placement data from a national database of eighth-grade students. We found that although some factors predicted placement in LD programs for all ethnic groups, placement predictors also varied by student ethnicity. Based on these findings, we provide implications for research and practice from a sociocultural perspective.

Dimensionality and Generalizability of Domain-Independent Performance Assessments.

Abstract Empirical guidance for the design of comparable performance assessments is sorely lacking. A study was conducted to assess the degree to which domain specifications control topic and rater variability, focusing on task generalizability, rater reliability, and scoring rubric dimensionality. Two classes of history students were administered three on-demand, multistep performance tasks a week apart. For each topic, all students completed a Prior Knowledge Test, read primary source materials, and wrote an essay of explanation. Using a theory-based scoring rubric, four trained raters scored all essays. Inter- and intrarater reliabilities and g-study results are reported. Results show relative efficiency for the assessment approach. The dimensionality analysis supported two factors: Deep Understanding and Surface Understanding across the three topics. Prior Knowledge scores and GPA in history courses correlated with the Deep Understanding elements of the scoring rubric. Implications for design and test...

Accommodations for Students With Limited English Proficiency in the National Assessment of Educational Progress

This study examines the effect and validity of accommodations for limited English proficiency (LEP) students in the National Assessment of Educational Progress (NAEP) and the impact of language factors on the assessment and accommodation of these students. Results indicate that accommodations used in NAEP did not reduce the performance gap between LEP and non-LEP students. The results suggest that these accommodations did not affect the construct under measurement, thus including the accommodated results into the NAEP main assessment data seems reasonable. In addition, results suggest that the higher the level of linguistic complexity in a test item, the higher the performance gap between LEP and non-LEP students, regardless of the items content difficulty. Thus, language-related accommodations may be more effective in reducing the performance gap between LEP and non-LEP students. However, because the main purpose of NAEP is assessment, not research, there are major limitations in the NAEP data for examining the effect and validity of accommodations. Thus, the findings of this study should be interpreted with caution.

Measuring Students' Level of English Proficiency: Educational Significance and Assessment Requirements

This article discusses the status of existing English language proficiency (ELP) tests, and compares the content coverage and psychometric characteristics of ELP assessments that existed prior to the implementation of the No Child Left Behind Act (NCLB) with those developed after NCLB Title III guidelines were introduced. The article argues that the newly developed ELP assessments (post-NCLB) exhibit higher assessment quality when compared with most ELP tests developed prior to NCLB (pre-NCLB). However, more work needs to be done if the new ELP assessments are to inform the instruction and content-based assessment of English language learner students. The results of analyses suggest that ELP assessment outcomes can help inform decisions regarding participation of English language learner students in state content-based assessments.

Predicting Graduate Academic Success from Undergraduate Academic Performance: A Canonical Correlation Study

One of the major criteria employed in selecting graduate students is undergraduate GPA; generally, applicants with low GPAs will not be accepted at high quality graduate schools. This study, which uses a canonical correlation technique, examined the efficiency of the undergraduate GPA as a predictor of graduate academic success and compared it with other predictors. The results indicated that undergraduate GPA was not a good predictor of graduate academic success.