Donald B. Young

Lessons Learned from the Process of Curriculum Developers' and Assessment Developers' Collaboration on the Development of Embedded Formative Assessments

Our project to embed formative student assessments in the Foundational Approaches in Science Teaching curriculum required a close collaboration between curriculum developers at the Curriculum Research & Development Group (CRDG) and assessment developers at the Stanford Educational Assessment Laboratory (SEAL). This was a new endeavor for each organization, and throughout the project, many lessons were learned about embedding assessments and about the collaboration process. In this article, we discuss what we learned about the strengths and weaknesses of the collaboration up to the beginning of the randomized experiment. What we found comported with the literature on research collaborations. For example, past collaborations between CRDG and SEAL facilitated moving the project forward and sustained the collaboration. That said, the physical distance between the groups gave rise to some misunderstandings and led to a commitment to meet face-to-face on a regular basis; we found that conferencing software did not suffice. Moreover, in our zeal to implement formative assessments, the voices of teachers and teacher trainers got muffled until a pilot study confirmed their advice.

The Complexity of Measuring the Quality of Program Implementation With Observations: The Case of Middle School Inquiry-Based Science

In the growing literature on the evaluation of program implementation, less has been said about evaluating program quality than about evaluating other aspects of program implementation. Furthermore, most articles and reports in the program-implementation evaluation literature have presented only brief descriptions of how implementation instruments have been developed. In this article, the authors describe a method for evaluating the quality of implementation of middle school inquiry-based science using data from observations scaled with paired comparison judgments. The authors show the complexities of developing and applying the method, describe how they tried it out, present the results of validity and reliability analyses, and describe the methods strengths and weaknesses.