Nicole Shechtman

DESIGNING FORMATIVE ASSESSMENT SOFTWARE WITH TEACHERS: AN ANALYSIS OF THE CO-DESIGN PROCESS

Researchers in the learning sciences have explored a collaborative approach to developing innovations that fit into real classroom contexts. The co-design process relies on teachers’ ongoing involvement with the design of educational innovations, which typically employ technology as a critical support for practice. To date, investigators have described the application and results of co-design, but they have not defined the process nor explored how it plays out over time. In this paper, we define co-design as a highly-facilitated, teambased process in which teachers, researchers, and developers work together in defined roles to design an educational innovation, realize the design in one or more prototypes, and evaluate each prototype’s significance for addressing a concrete educational need. We suggest seven key process components and use data from a systematic set of interviews to illustrate the roles of teachers and researchers in co-design and describe how tensions in the process can unfold and be resolved over time.

Integration of Technology, Curriculum, and Professional Development for Advancing Middle School Mathematics Three Large-Scale Studies

The authors present three studies (two randomized controlled experiments and one embedded quasi-experiment) designed to evaluate the impact of replacement units targeting student learning of advanced middle school mathematics. The studies evaluated the SimCalc approach, which integrates an interactive representational technology, paper curriculum, and teacher professional development. Each study addressed both replicability of findings and robustness across Texas settings, with varied teacher characteristics (backgrounds, knowledge, attitudes) and student characteristics (demographics, levels of prior mathematics knowledge). Analyses revealed statistically significant main effects, with student-level effect sizes of .63, .50, and .56. These consistent gains support the conclusion that SimCalc is effective in enabling a wide variety of teachers in a diversity of settings to extend student learning to more advanced mathematics.

Investigating Links from Teacher Knowledge, to Classroom Practice, to Student Learning in the Instructional System of the Middle-School Mathematics Classroom

Using data collected in 125 seventh-grade and 56 eighth-grade Texas classrooms in the context of the “Scaling Up SimCalc” research project in 2005–07, we examined relationships between teachers’ mathematics knowledge, teachers’ classroom decision making, and student achievement outcomes on topics of rate, proportionality, and linear function—three important and cognitively demanding prealgebra topics. We found that teachers’ mathematical knowledge was correlated with student achievement in only one study out of three. We also found a lack of correlations between teachers’ mathematical knowledge and critical aspects of instructional decision making. Curriculum and other learning resources (e.g., technology, student–student interactions) are clearly important factors for student learning in addition to, and in interaction with, teachers’ mathematical knowledge. Our results suggest that mathematics knowledge for teaching may have a nonlinear relationship with student learning, that those effects may be heavily mediated by other instructional factors, and that short-term content knowledge gains in teacher workshops may not persist in classroom instruction. We discuss a need in the field for richer models of how “mathematical knowledge for teaching” works in the context of complete instructional systems.

Scaling up Innovative Technology-Based Mathematics

We report on the initial attempts at evaluating at scale a particular technological/curricular innovation that enables more students to develop deeper knowledge. The methods, issues, and findings of the current pilot experiment speak not only to the success of SimCalc MathWorlds, the focus of our research program, but also to the evaluation at scale of a broad class of representationally innovative technologies and to the merit of long-term investment in design-based research. In particular, we present conditions and findings from a completed pilot experiment involving 21 seventh-grade mathematics teachers from Texas. Pilot outcomes suggest that (a) innovative representational technologies can have an important impact on student learning, (b) considerable impact can be found across a wide range of teachers and conditions, and (c) these gains can be detected even in the absence of other desirable conditions. In particular, detection of student gains does not, in our case, depend on having a long-term context of learning, long-term teacher professional development, or a shift to learner-centered constructivist pedagogy. The full experiment will replicate and extend our experimental design with a wider range of teachers and schools, model the factors that contribute to classroom success with such technology, and explore what happens as research support fades away.

SimCalc at Scale: Three Studies Examine the Integration of Technology, Curriculum, and Professional Development for Advancing Middle School Mathematics

We discuss three large-scale studies—two randomized controlled experiments and one embedded quasi-experiment—designed to evaluate the impact of SimCalc replacement units targeting student learning of advanced middle school mathematics. Each study addressed both replicability of findings and robustness across Texas settings, with varied teacher characteristics—backgrounds, knowledge, attitudes—and student characteristics—demographics, levels of prior mathematics knowledge. Analyses revealed statistically significant main effects, with student-level effect sizes of 0.63, 0.50, and 0.56. These consistent gains support the conclusion that SimCalc is effective in enabling a wide variety of teachers in a diversity of settings to extend student learning to more advanced mathematics.

A Principal Components Model of Simcalc Mathworlds

This study examines changes in student knowledge through a pretest/posttest assessment using data from an experimental research project on the effects of dynamic software on student outcomes in 7th-grade classrooms. Through a principal components analysis, we found that students who used the software had test results that clustered concepts differently than students who did not, grouping together test questions involving various representations of rate and proportionality. Students who did not use the software had test results that grouped test questions together based on surface features and proximity. We present a model showing how increased access to dynamic software leads to an increased ability to fuse concepts, where students add to their mental models and link various representations of the same mathematical concepts together.

Development of Student and Teacher Assessments in the Scaling Up SimCalc Project

As SimCalc targets mathematics achievement goals that lie beyond what many schools today focus on, new assessments are needed to measure what students learn and what teachers must know to support their learning. We provide an overview of how we developed four assessments for the Scaling Up SimCalc project and describe each of the processes we used to document the technical qualities of the assessments. This methodological approach can be used to measure the effectiveness of dynamic mathematics approaches at scale.