Douglas Lyman Corey

Quality control mechanisms for crowdsourcing: peer review, arbitration, & expertise at familysearch indexing

The FamilySearch Indexing project has enabled hundreds of thousands of volunteers to transcribe billions of records, making it one of the largest crowdsourcing initiatives in the world. Assuring high quality transcriptions (i.e., indexes) with a reasonable amount of volunteer effort is essential to keep pace with the mounds of newly digitized documents. Using historical data, we show the relationship between prior experience and native language on transcriber agreement. We then present a field experiment comparing the effectiveness (accuracy) and efficiency (time) of two quality control mechanisms: (1) Arbitration -- the existing mechanism wherein two volunteers independently transcribe records and disagreements go to an arbitrator, and (2) Peer Review -- a mechanism wherein one volunteers work is reviewed by another volunteer. Peer Review is significantly more efficient, though not as effective for certain fields as Arbitration. Design suggestions for FamilySearch Indexing and related crowdsourcing initiatives are provided.

Initial Progress of Children Identified with Disabilities in Michigan's Reading First Schools

This 2-year longitudinal study examined initial evidence of progress in reading for 1,512 children with and without identified speech-language and/or learning disabilities (LD-SLD) in the context of the explicit literacy instruction provided in Michigans Reading First (RF) schools. The findings suggested that children with LD-SLD labels demonstrated significantly slower growth compared to children without LD-SLD labels. Children considered more at risk also demonstrated slower progress in oral reading fluency (but not reading comprehension) compared to children considered less at risk. Implications are discussed in relationship to the extent of instructional support needed by children identified as LD-SLD in the mainstream, and in terms of the specific dimension of reading skills as a criterion.

Explaining Variation in Instructional Time: An Application of Quantile Regression

This research is conducted in the context of a large-scale study of three nationally disseminated comprehensive school reform projects (CSRs) and examines how school- and classroom-level factors contribute to variation in instructional time in English language arts and mathematics. When using mean-based OLS regression techniques such as Hierarchical Linear Models (HLM), we found that CSR programs did not have the expected effects on instructional time. However, when using Quantile Regression to estimate the effects at the lower end of the distribution of instructional time, we found substantial effects. These effects were strongest for the subjects that were the focus of the school interventions.

How Much English Language Arts and Mathematics Instruction Do Students Receive? Investigating Variation in Instructional Time

The amount of instruction students receive has long been viewed as a foundational educational resource. This article presents an analysis of the time students spend in elementary English language arts (ELA) and mathematics instruction. In mathematics, the average student received about 140 hr of instruction, but students in the top sixth of classrooms in this distribution can expect to receive between 80 and 160 hr more instruction over the school year than students assigned to the bottom sixth of classrooms. We found similar magnitudes of variation in ELA. Although most of the variation was due to differences among classrooms, there was also substantial variation among schools. Some variation in instructional time is expected and probably favorable. However, we argue that the large variation demonstrated by these results represent substantial inequity in students’ opportunity to learn ELA and mathematics.

The Instructional Quality of Mathematics Student Teachers in the United States and Japan: The Possible Impact of the Structure of Student Teaching

In this chapter we explore the instructional quality of four US student teachers in a novel student teaching structure. To overcome some of the common problems associated with student teaching documented in the research literature, we adapted a student teaching structure commonly used in Japan. We evaluate the instructional quality of the lessons by using the Mathematical Quality of Instruction (MQI) video coding protocol. We compare the instructional quality to a sample of Japanese student teachers and to a large sample of lessons from six large US school districts, utilizing the Measures of Effective Teaching (MET) study. We also illustrate the quantitative findings with vignettes from US and Japanese student teaching. The results show that given the right support and structure, student teachers in the USA can implement lessons that are similar in quality to Japanese student teachers and much richer than typical US mathematics instruction.