James C. Impara

Teacher Assessment Literacy: What Do Teachers Know about Assessment?

Publisher Summary It is estimated that teachers spend up to 50% of their instructional time in assessment-related activities. The chapter discusses the teacher assessment literacy—what teachers actually know about assessment. It has been found that teachers receive little or no formal assessment training in the preparatory programs and often they are ill-prepared to undertake assessment-related activities. With the introduction of “authentic” assessment strategies, it is more important for teachers to be more skilled in assessment. This is so because they often are involved directly in the administration and scoring of these assessments. Some studies have attempted to quantify the level of teacher preparation in educational assessment of students. The chapter highlights the study of a national survey of teacher assessment literacy. Also, it presents a more detailed analysis of teacher performance on the instrument. The results of this study indicate low levels of assessment competency for teachers. These results suggest that it is time for the education community to recognize that teachers are ill-equipped to successfully undertake one of the most prevalent activities of their instructional program: student assessment. This is especially salient due to the current trend in student assessment, involving an increase in assessment strategies such as performance, portfolio, and other types of “authentic assessments.”

Ability of Panelists to Estimate Item Performance for a Target Group of Candidates: An Issue in Judgmental Standard Setting

Recent researchers (Impara & Plake, 1998; National Research Council, 1999; Shepard, 1995) have called into question the ability of judges to make accurate item performance estimates for target subgroups of candidates, such as minimally competent candidates. The purpose of this study was to examine both the reliability and accuracy of item performance estimates from an Angoff (1971) standard setting application. Results provide evidence that item performance estimates were both reasonable and reliable. Factors that might have influenced these results are discussed.

Making the Cut in School Districts: Alternative Methods for Setting Cutscores

School districts are under increasing pressure to demonstrate that students are competent in various skills, such as reading and mathematics. Often, demonstrating competence involves comparing performance on assessments to a standard of performance, as embodied in a test score. These scores, called cutscores, separate competent and noncompetent examinees. Because school districts have varied sources of data to inform cutscore decisions, various methods are available for suggesting cutscores. In 2 studies, we examine a selection of methods for arriving at rational and defensible cutscores in school districts. Methods examined are the Angoff (1971) method; the borderline and contrasting groups methods; and 2 new methods, 1 based on course enrollment and 1 based on expert expectations. In Study 1, the Angoff, borderline group, and course enrollment results were consistent, whereas in Study 2, the Angoff and professional judgment methods yielded suggested cutscores that were lower than the borderline group method. Suggestions for further study include the reaction of teachers to the cutscore-setting methods, the effect of different teacher attributes on the results of cutscore-setting methods, and the efficiency of and most effective order for employing the various methods.

Closing the Loop: Providing Test Developers with Performance Level Descriptors So Standard Setters Can Do Their Job

Standard setting panels are tasked with recommending one or more performance level standards for assessments that are used to classify students into ability categories. These assessments are sometimes developed with the performance level descriptors known and other times without these descriptors. Based on an analysis of 11 state, educational, alternative assessments, this chapter investigates the effects on the standard setting process of developing a test both with and without these descriptors. The results suggest that the standard setting panelists are more consistent with one another and more aligned with empirical data when the items were developed with the descriptors in mind.

Teacher Assessment Literacy

Publisher Summary It is estimated that teachers spend up to 50% of their instructional time in assessment-related activities. The chapter discusses the teacher assessment literacy—what teachers actually know about assessment. It has been found that teachers receive little or no formal assessment training in the preparatory programs and often they are ill-prepared to undertake assessment-related activities. With the introduction of “authentic” assessment strategies, it is more important for teachers to be more skilled in assessment. This is so because they often are involved directly in the administration and scoring of these assessments. Some studies have attempted to quantify the level of teacher preparation in educational assessment of students. The chapter highlights the study of a national survey of teacher assessment literacy. Also, it presents a more detailed analysis of teacher performance on the instrument. The results of this study indicate low levels of assessment competency for teachers. These results suggest that it is time for the education community to recognize that teachers are ill-equipped to successfully undertake one of the most prevalent activities of their instructional program: student assessment. This is especially salient due to the current trend in student assessment, involving an increase in assessment strategies such as performance, portfolio, and other types of “authentic assessments.”

Chapter 3 – Teacher Assessment Literacy: What Do Teachers Know about Assessment?

Publisher Summary It is estimated that teachers spend up to 50% of their instructional time in assessment-related activities. The chapter discusses the teacher assessment literacy—what teachers actually know about assessment. It has been found that teachers receive little or no formal assessment training in the preparatory programs and often they are ill-prepared to undertake assessment-related activities. With the introduction of “authentic” assessment strategies, it is more important for teachers to be more skilled in assessment. This is so because they often are involved directly in the administration and scoring of these assessments. Some studies have attempted to quantify the level of teacher preparation in educational assessment of students. The chapter highlights the study of a national survey of teacher assessment literacy. Also, it presents a more detailed analysis of teacher performance on the instrument. The results of this study indicate low levels of assessment competency for teachers. These results suggest that it is time for the education community to recognize that teachers are ill-equipped to successfully undertake one of the most prevalent activities of their instructional program: student assessment. This is especially salient due to the current trend in student assessment, involving an increase in assessment strategies such as performance, portfolio, and other types of “authentic assessments.”