Development and Preliminary Content Validation of ENGAGE as an Assessment of Preschool Interactions and Active Engagement

Abstract

ENGAGE is in\ndevelopment as a web-based observation system with core features we believe\nwill facilitate its use as a scalable assessment-to-action coaching and\ninstructional support system in preschool classrooms. ENGAGE assesses adult interaction\npractices and children’s active engagement such that classroom teams receive\ndata to inform their intentional design and delivery of embedded instruction\nthat can be made more effective for children’s developmental needs through\ndifferentiation and intensification. For this study, we describe important\nprecursors to evaluating other psychometric properties within an argument-based\napproach to validation. We used iterative cycles to gather evidence to refine\nand validate the content of our measurement targets (i.e., adult interaction practices\nand child active engagement) as well as constructs (i.e., groupings of practices\nby theorized mechanisms). Following two iterative rounds using online\nquestionnaires, responses from participants representing intended users of\nENGAGE generally demonstrated consistency with our definitions and\nconceptualizations, with more varied responses for child engagement. We discuss\nour procedures and results in terms of reducing measurement error that may be\nattributable to measurement targets in advance of future examinations of observation\nconditions (e.g., observer training, session duration) and situational variance\n(e.g., instructional pedagogy, adult-child ratios) that are often influential\nwithin observation-based assessments.