Christopher DeLuca

Assessment literacy development: identifying gaps in teacher candidates’ learning

As a result of the standards‐based movement in education there is an increased need for teacher competency in the area of student assessment and evaluation. This study examines assessment education at one pre‐service teacher education program in Ontario, Canada. Through a questionnaire administered to 288 teacher candidates, this study identified teacher candidates’ perceived confidence levels in assessment practice, theory, and philosophy. These teacher candidates also provided their views towards assessment topics that it is important to include in a pre‐service educational assessment course. Descriptive statistics, factor analysis, and analysis of variance were used to analyse questionnaire responses. In general, findings support the need for direct instruction in assessment with specific topics identified (e.g. reporting achievement, modifying assessments, developing constructed‐response items, item reliability, validity, articulating a philosophy of assessment, etc.) as important to developing teacher assessment literacy.

Establishing a foundation for valid teacher judgement on student learning: the role of pre-service assessment education

The purpose of this research was to explore changes in teacher candidates’ confidence in assessment as a foundation for promoting valid teacher judgements on student learning. Based on a sample of 48 teacher candidates enrolled in a semester-long measurement course at one teacher education programme in Florida, this survey-based study provided evidence that teacher candidates can develop confidence in assessment whilst deepening their conceptions of assessment. The majority of teacher candidates expanded their conception of assessment from one based solely on testing to one that recognised multiple forms of assessment that serve varying purposes. Simultaneously, teacher candidates expressed greater confidence in practical assessment approaches and factors related to assessment praxis. Based on study findings, we argue that pre-service teacher education has a critical role to play in promoting assessment literacy in beginning teachers and providing a foundation for teachers’ continued learning about assessment throughout their careers. The paper concludes with implications for future research in the area of assessment education.

Rethinking validation in complex high-stakes assessment contexts

In this article we rethink validation within the complex contexts of high-stakes assessment. We begin by considering the utility of existing models for validation and argue that these models tend to overlook some of the complexities inherent to assessment use, including the multiple interpretations of assessment purposes and the potential interaction of assessment uses. We respond to these limitations by proposing an interpretive approach to validation that we call validation as narrative case description. This approach uses a case-based framework and hermeneutic methodology to construct and analyse validity evidence and suggests narrative as a representational format for communicating validation claims. We illustrate this approach by considering a case study of a secondary mathematics assessment administered in Ontario, Canada. In introducing this approach, we are contributing to the ongoing dialogue on reconceptualising validation as an interpretive process that serves a generative function.

Voices From Test-Takers: Further Evidence for Language Assessment Validation and Use

Test-takers interpretations of validity as related to test constructs and test use have been widely debated in large-scale language assessment. This study contributes further evidence to this debate by examining 59 test-takers perspectives in writing large-scale English language tests. Participants wrote about their test-taking experiences in 300 to 500 words, focusing on their perceptions of test validity and test use. A standard thematic coding process and logical cross-analysis were used to analyze test-takers experiences. Codes were deductively generated and related to both experiential (i.e., testing conditions and consequences) and psychometric (i.e., test construction, format, and administration) aspects of testing. These findings offer test-takers voices on fundamental aspects of language assessment, which bear implications for test developers, test administrators, and test users. The study also demonstrated the need for obtaining additional evidence from test-takers for validating large-scale language tests.

Interpretive validity theory: mapping a methodology for validating educational assessments

Background:u2003Validity theory has evolved significantly over the past 30 years in response to the increased use of assessments across scientific, social and educational settings. The overarching trajectory of this evolution reflects a shift from a purely quantitative, positivistic approach to a conception of validity reliant on the interpretation of multiple evidence sources integrated into validity arguments. Moreover, within contemporary validity, interpretation has been emphasised as a central process; however, despite this emphasis, there have been few explicit articulations of specific interpretive methodologies applicable to the practice of validation. Purpose:u2003To link contemporary theoretical foundations in validity to practical methods and structures to help guide the collection and analysis of interpretive validity evidence. By building upon existing validity theory, this paper aims to provide greater clarity on the practice of validation and contribute toward the larger developing framework for the validation of educational assessments. Source of evidence:u2003An interdisciplinary, integrative review of over 60 research articles and sources related to the theory and practice of educational validation and interpretive inquiry approaches. Sources include literature from the fields of educational assessment and more broadly social scientific research. Main argument:u2003As assessments in education increasingly aim to measure complex constructs that are value-laden and socially dependant, validity theory must keep pace and evolve in ways that address the inherent complexities associated with contemporary educational assessment. Through this paper, I assert that a greater understanding of interpretive methodologies represents one of the most promising areas for development of validation theory and practice. Specifically, I argue that dialectic, hermeneutic and transgressive forms of inquiry can be integrated within current argument-based structures for the collection, analysis and representation of validity evidence in several useful ways. Conclusions:u2003Interpretive inquiry processes, namely dialectic, hermeneutic and transgressive forms of interpretation, serve to expand validation practice to include diverse evidences for the generation of multiple-perspective validity arguments. The paper concludes with specific implications for future research and practice within the field of interpretive validity theory.

Leveraging Technology to Promote Assessment for Learning in Higher Education

Assessment for learning (AFL) is a highly effective strategy for promoting student learning, development and achievement in higher education (Falchikov, 2003; Kirby & Downs, 2007; Nicol & Macfarlane-Dick, 2006; Rust, Price, & O’Donovan, 2003; Vermunt, 2005). However, since AFL relies on continuous monitoring of student progress through instructor feedback, peer collaboration, and student self-assessment, enacting AFL within large-group learning formats is challenging. This paper considers how technology can be leveraged to promote AFL in higher education. Drawing on data from students and instructors and recommendations from an external instructional design consultant, this paper documents the process of pairing technology and AFL within a large-group pre-service teacher education course at one Canadian institution. Recommendations for the improvement of the web-based component of the course are highlighted to provide practical suggestions for instructors to evaluate their own web-based platforms and improve their use of technology in support of AFL. The paper concludes with a discussion of areas for continued research related to the effectiveness of this pairing between assessment theory and technology.