Carlos C. Ayala

Reasoning Dimensions Underlying Science Achievement: The Case of Performance Assessment.

Snow argued for multidimensional science achievement in the National Education Longitudinal Study of 1988 (NELS:88) along dimensions of basic knowledge and reasoning, spatial-mechanical reasoning, and quantitative science. We focused the generality of these reasoning dimensions in other multiple-choice tests and performance assessments. Confirmatory factor analyses retrieved the 3 dimensions for a test composed of NELS:88, the Third International Mathematics and Science Study (TIMSS) and the National Assessment of Educational Progress (NAEP) multiple-choice items, and the NELS:88 items alone. We used the latter because factor correlations were lower. We administered 3 reasoning-dimension-linked performance assessments to a subsample of 35 students from the main study. Performance assessments correlated moderately with each other and NELS:88 reasoning scores; the 2 methods partially converged on the dimensions. Performance scores scattered across multiple-choice scores due to the broad reasoning and knowledge spectrum tapped. Findings are tentative; larger samples and cognitive studies of reasoning and knowledge might shed light on convergence.

Proposition 227: Tales from the Schoolhouse.

Abstract This article explores the impact of Proposition 227 on students and teachers based on interviews with parents, teachers, and administrators of a school in the Bay Area. We discuss four themes that emerge from the data: parent involvement, academic impact on students, the instructional challenges posed by Proposition 227, and the emotional impact on teachers and students. Connecting these themes is an overemphasis on language of instruction, which we found to overshadow other issues critical to the education of language minority students.

Lessons Learned from the Process of Curriculum Developers' and Assessment Developers' Collaboration on the Development of Embedded Formative Assessments

Our project to embed formative student assessments in the Foundational Approaches in Science Teaching curriculum required a close collaboration between curriculum developers at the Curriculum Research & Development Group (CRDG) and assessment developers at the Stanford Educational Assessment Laboratory (SEAL). This was a new endeavor for each organization, and throughout the project, many lessons were learned about embedding assessments and about the collaboration process. In this article, we discuss what we learned about the strengths and weaknesses of the collaboration up to the beginning of the randomized experiment. What we found comported with the literature on research collaborations. For example, past collaborations between CRDG and SEAL facilitated moving the project forward and sustained the collaboration. That said, the physical distance between the groups gave rise to some misunderstandings and led to a commitment to meet face-to-face on a regular basis; we found that conferencing software did not suffice. Moreover, in our zeal to implement formative assessments, the voices of teachers and teacher trainers got muffled until a pilot study confirmed their advice.

The Concept of Aptitude and Multidimensional Validity Revisited.

Richard E. Snow, reasoning from his new conception of aptitude, advocated a multidimensional approach to validating the construct of academic achievement. We briefly overview Snows approach and then summarize evidence from this special issue in 3 themes: (a) the multidimensional structure of science achievement, (b) the incremental predictive validity provided when both cognitive and motivational (affective and conative) constructs are used to model individual differences in achievement, and (c) the co-contributions of ability, motivational orientations, and characteristics of achievement test situations to performance differences. Overall, our studies confirmed or established (a) a multidimensional structure of science achievement scores, (b) the validity of several key motivational constructs for predicting science achievement among high school students, (c) systematic variation in relations between motivational and general-ability constructs and science reasoning dimensions, assessment type, and achievement behavior (performance and anticipated choice), and (d) how alternative assessment methods (constructed response and performance assessment) shed light on the notion of multidimensional validity.