Don A. Klinger

Assessment literacy development: identifying gaps in teacher candidates’ learning

As a result of the standards‐based movement in education there is an increased need for teacher competency in the area of student assessment and evaluation. This study examines assessment education at one pre‐service teacher education program in Ontario, Canada. Through a questionnaire administered to 288 teacher candidates, this study identified teacher candidates’ perceived confidence levels in assessment practice, theory, and philosophy. These teacher candidates also provided their views towards assessment topics that it is important to include in a pre‐service educational assessment course. Descriptive statistics, factor analysis, and analysis of variance were used to analyse questionnaire responses. In general, findings support the need for direct instruction in assessment with specific topics identified (e.g. reporting achievement, modifying assessments, developing constructed‐response items, item reliability, validity, articulating a philosophy of assessment, etc.) as important to developing teacher assessment literacy.

The relationship of schools to emotional health and bullying

Objectives:To examine the extent to which school climate and school pressure could predict other aspects of adolescents’ lives, most particularly their emotional health and bullying. Furthermore, the study sought to investigate if these relationships were consistent across countries.Methods:Participants were 11-, 13-, and 15-year-olds from 26 European countries/regions, Canada, the United States, and Israel. Participants completed surveys focusing on health behaviours and lifestyles, using a contextual framework. Using cluster analytic techniques, three clusters were created varying on school pressure and perceived school climate. These clusters were then examined using variables not used in the clustering.Results:Students in the cluster having the most positive relationships to school outcomes, including academic achievement, truancy, teacher and peer support, also had the most positive emotional health and the lowest incidence of bullying. Similarly, those in the poorest cluster in terms of school also had the poorest outcomes in terms of emotional health and bullying.Conclusions:These relatively small but significant associations suggest that schools may have a small role in supporting children’s emotional well-being and ameliorate the presence of bullying.

Psychometric properties of the multiple mini-interview used for medical admissions: findings from generalizability and Rasch analyses

The multiple mini-interview (MMI) has become an increasingly popular admissions method for selecting prospective students into professional programs (e.g., medical school). The MMI uses a series of short, labour intensive simulation stations and scenario interviews to more effectively assess applicants’ non-cognitive qualities such as empathy, critical thinking, integrity, and communication. MMI data from 455 medical school applicants were analyzed using: (1) Generalizability Theory to estimate the generalizability of the MMI and identify sources of error; and (2) the Many-Facet Rasch Model, to identify misfitting examinees, items and raters. Consistent with previous research, our results support the reliability of MMI process. However, it appears that the non-cognitive qualities are not being measured as unique constructs across stations.

CONTEXTUAL AND SCHOOL FACTORS ASSOCIATED WITH ACHIEVEMENT ON A HIGH‐STAKES EXAMINATION

This study identified student and school ‐ level factors associated with student achievement on the Ontario Secondary School Literacy Test (OSSLT), an examination that includes a student questionnaire examining home literacy practices. Linked student and school contextual data enabled the use of hierarchical linear modeling to complete the analyses and examine both student and school level effects. Fourteen student and three school level variables were found to be associated with students’ reading achievement and twelve student and two school level variables were associated with writing. Significant variations between schools were also found for students with individualized education plans, English as a second language, or previous eligibility on the OSSLT. Key words: hierarchical linear modeling, literacy testing, achievement, education factors Dans cette etude, les auteurs ont identifie les facteurs eleves et niveaux scolaires associes au rendement scolaire dans le Test de competences linguistiques de l’Ontario, un examen qui comprend un questionnaire pour les eleves au sujet des habitudes en matiere de litteratie a la maison. Des donnees reliees aux eleves et au contexte scolaire ont permis l’utilisation d’une modelisation lineaire hierarchique en vue de completer les analyses et d’etudier les effets quant aux eleves et aux niveaux scolaires. Les auteurs ont etabli que quatorze variables ayant trait aux eleves et trois ayant trait aux niveaux scolaires etaient associees au rendement des eleves en matiere de lecture et que douze variables ayant trait aux eleves et deux ayant trait aux niveaux scolaires etaient associees a l’ecriture. Des ecarts importants entre les ecoles ont egalement ete notes chez les eleves ayant des programmes pedagogiques sur mesure, inscrits en anglais langue seconde ou deja admissibles au Test de competences linguistiques des ecoles secondaires de l’Ontario. Mots cles : modelisation lineaire hierarchique, test de competences linguistiques, rendement scolaire, facteurs pedagogiques

Teachers' Perceptions of Large-Scale Assessment Programs within Low-Stakes Accountability Frameworks.

The intent of this study was to examine the views of teachers regarding the appropriateness of the purposes and uses of the provincial assessments in Alberta and Ontario and the seriousness of the concerns raised about these assessments. These provinces represent educational jurisdictions that use large-scale assessments within a low-stakes accountability framework in which the results are intended to be used to support school-based improvement efforts. Despite being implemented at different times (1982–Alberta; 1996–Ontario), teachers in both provinces appear to hold relatively similar views about the testing programs in their own provinces and the issues associated with these programs. Teachers’ concerns regarding the use of these assessments for accountability purposes and the potential misuses of the results appear to be a dominant influence on teachers’ generally low ratings of other purposes and uses of the assessments in both provinces, including those related to improving instruction and learning.

Collaborative inquiry as a professional learning structure for educators: a scoping review

Collaborative inquiry (CI) has emerged as a dominant structure for educator professional learning in the twenty-first century. CI engages educators in collaboratively investigating focused aspects of their professional practice by exploring student responses to instruction, leading to new understandings and changes in classroom teaching. However, despite the increased presence of CI, research on CI frameworks has yet to be consolidated and synthesized. The purpose of this systematic, scoping review was to examine literature on the structure, challenges, and benefits of CI as a professional learning structure for educators (i.e. teachers, principals, school district leaders). In total, 42 sources were identified and analysed in relation to characteristics of CI, supports and resources for CI, empirically supported benefits of CI, and enactment challenges. The review found that the majority of texts and research in this field are highly practical, describing CI steps or case-study examples. Accordingly, the current literature reviewed in this paper largely serves a how to function for engaging in CI projects with empirical data collected from primarily case-study work. The literature also provides preliminary theoretical articulations for CI as a professional learning structure for educators. The paper concludes with identified areas for future CI research related to: clarifying the focus of CI initiatives, articulating what ‘inquiry’ means in CI, and sustaining CI within the profession of teaching.

STUDENT AND SCHOOL CORRELATES OF MATHEMATICS ACHIEVEMENT: MODELS OF SCHOOL PERFORMANCE BASED ON PAN‐ CANADIAN STUDENT ASSESSMENT

This study explored the relationships between student achievement and student, school and home variables from the pan ‐ Canadian assessment program administered by the Council of Ministers of Education Canada (CMEC): the School Achievement Indicators Program (SAIP) Mathematics 2001. The study also evaluated the datasets used in relationship to their utility for statistical modeling of school performance. Student beliefs about mathematics are positively related to mathematics achievement for both age groups and for both domains of mathematics. As students’ use of instructional supports (parental assistance with mathematics homework, computers in the mathematics classroom) increases, there is an associated decrease in mathematics scores. Key words: student achievement, school performance, multilevel modeling. Dans cet article, les auteurs explorent les relations entre le rendement scolaire et certaines caracteristiques des eleves, des ecoles et de l’environnement familial a partir des donnees du programme d’evaluation pancanadien du Conseil des ministres de l ʹ Education (Canada) (CMEC), le Programme d’indicateurs du rendement scolaire (PIRS) 2001 en mathematiques. L’etude evalue egalement les bases de donnees au regard de leur pertinence pour la modelisation statistique de la performance scolaire. Les auteurs notent une correlation directe entre les idees que se font les eleves au sujet des mathematiques et le rendement en mathematiques tant pour les groupes d’âge que pour les domaines des mathematiques. Plus les eleves font appel a du soutien pedagogique (aide des parents pour les devoirs de mathematiques, ordinateurs dans les classes de mathematiques), plus les notes en mathematiques diminuent. Mots cles : rendement des eleves, performance scolaire, modelisation a multiples niveaux.

EXAMINATION OF THE INFLUENCE OF SELECTED FACTORS ON PERFORMANCE ON ALBERTA LEARNING ACHIEVEMENT TESTS

In this study , we identified factors that influence the academic performance of grade ‐ 6 students in a large city school system in language arts and mathematics. For language arts, five student variables, seven class variables, and two school level variables accounted for approximately 50 per cent, 75 per cent, and 90 per cent of the initial variability at the corresponding levels. For mathematics, seven student, five class, and three school level variables accounted for approximately 40 per cent, 60 per cent, and 90 per cent of the initial variability at the school level. Taken together, the results reveal that the majority of variability is at the student level, and additional variables need to be identified to better explain the variability at this level. Key words: school effectiveness, hierarchical linear multi ‐ level modeling, large ‐ scale testing Dans cette etude portant sur le systeme scolaire d’une grande ville, les auteurs identifient les facteurs ayant une incidence sur le rendement scolaire d’eleves de 6 e annee en langue (anglais) et en mathematiques. Pour le domaine de la langue, cinq variables ayant trait aux eleves expliquent environ la moitie de la variabilite initiale quant aux eleves, sept variables ayant trait a la classe expliquent environ 75 % de la variabilite initiale quant aux classes et deux variables ayant trait a l’ecole, environ 90 % de la variabilite initiale quant aux ecoles. En mathematiques, sept variables ayant trait aux eleves, cinq ayant trait aux classes et trois aux ecoles expliquent respectivement environ 40 %, 60 % et 90 % de la variabilite initiale. Pris en bloc, les resultats revelent que la majeure partie de la variabilite se situe au niveau des eleves et que d’autres variables doivent etre identifiees afin de mieux expliquer la variabilite a ce niveau. Mots cles : efficacite de l ʹ ecole, regression hierarchique multiple, tests communs.

Instructional Rounds as a professional learning model for systemic implementation of Assessment for Learning

The purpose of this research was to examine the implementation of a professional learning project aimed at building educators’ knowledge and skills in assessment for learning (AfL) within two school districts in Ontario, Canada. Specifically, the research examined the value of a two-tier Instructional Rounds (IR) professional learning model. This professional learning model was unique because it engaged both teachers and principals in collaboratively learning and implementing AfL strategies in order to develop systemic capacity in assessment. In total, 12 principals, 48 teachers, two superintendents and two school district assessment consultants participated in the study. Data were collected through observations of IR sessions, classroom observations, interviews, IR session reflections and a post-project survey. Findings from this study report on positive changes in teachers’ and principals’ conceptions and implementation of AfL as well as on the value and challenges of IR as a professional learning model. The paper concludes with a discussion on developing systemic capacity in AfL through an IR model of professional learning.

School Accountability and Assessment: Should We Put the Roof Up First?

Abstract School accountability and student assessment are closely associated in educational jurisdictions’ attempts to monitor student achievement, focus instruction, and improve subsequent student and school performance. The research reported in this article examines the School Effectiveness Framework in Ontario, Canada, exploring the foundations upon which the framework is based and educators’ efforts to use assessment information to monitor the effectiveness of their reform efforts. Through document analysis and interviews, the researchers highlight the challenges and opportunities surrounding the accountability and assessment partnership.