M. Straatemeier

Computer adaptive practice of Maths ability using a new item response model for on the fly ability and difficulty estimation

In this paper we present a model for computerized adaptive practice and monitoring. This model is used in the Maths Garden, a web-based monitoring system, which includes a challenging web environment for children to practice arithmetic. Using a new item response model based on the Elo (1978) rating system and an explicit scoring rule, estimates of the ability of persons and the difficulty of items are updated with every answered item, allowing for on the fly item calibration. In the scoring rule both accuracy and response time are accounted for. Items are sampled with a mean success probability of .75, making the tasks challenging yet not too difficult. In a period of ten months our sample of 3648 children completed over 3.5 million arithmetic problems. The children completed about 33% of these problems outside school hours. Results show better measurement precision, high validity and reliability, high pupil satisfaction, and many interesting options for monitoring progress, diagnosing errors and analyzing development.

Children's knowledge of the earth : A new methodological and statistical approach

In the field of childrens knowledge of the earth, much debate has concerned the question of whether childrens naive knowledge-that is, their knowledge before they acquire the standard scientific theory-is coherent (i.e., theory-like) or fragmented. We conducted two studies with large samples (N=328 and N=381) using a new paper-and-pencil test, denoted the EARTH (EArth Representation Test for cHildren), to discriminate between these two alternatives. We performed latent class analyses on the responses to the EARTH to test mental models associated with these alternatives. The naive mental models, as formulated by Vosniadou and Brewer, were not supported by the results. The results indicated that childrens knowledge of the earth becomes more consistent as children grow older. These findings support the view that childrens naive knowledge is fragmented.

How to detect cognitive strategies: Commentary on 'Differentiation and integration: Guiding principles for analyzing cognitive change'

We have to warn our readers: This is a commentary about measurement and statistics. As developmental psychologists, we would like to discuss more interesting topics like cognitive strategy construction, learning, and development. The contribution of Siegler and Chen (2008) is clearly exciting in this respect. We fully agree with Siegler and Chen about the importance of differentiation and integration in the process of cognitive change. We also greatly appreciate the results on self-explanations of both correct and incorrect answers. These findings may be of great practical use in education. However, we reluctantly set all this aside, in favor of a commentary about the use of statistical techniques to classify subjects into categories. In the present case, the categories are rules or strategies for solving water displacement items. The task of accurately classifying children in this way is a crucial step in many developmental studies. To this end, Siegler and Chen use the Rule Assessment Methodology (RAM), originally proposed by Siegler (1976, 1981) for proportional reasoning tasks. We contend that this classification procedure is suboptimal, because it lacks a sound statistical basis. We also contend that there is a better method, which can handle the model implied by RAM, and which is well developed both statistically and computationally. Our message is simple. Psychologists are intensive users of statistical tests. In almost every paper, they apply advanced multivariate techniques such as structural equation modeling, analysis of variance, mixed effect and loglinear analysis, and sequential multinomial logit analysis (see Siegler & Chen, 2008). Given the general reliance on statistical procedures in our empirical studies, it is remarkable that, when it concerns classification, we tend to ignore well-developed statistical procedures, and resort to methods based on ad hoc criteria for classification. In classifying subjects into unobserved categories (types, rules, strategies, etc.) based on observed responses, we should use appropriate statistical techniques, such as finite mixture modeling or categorical latent variable techniques. From a theoretical point of view, we view Siegler’s introduction of Rule Assessment Methodology (RAM) in cognitive task analysis as one of the milestones in the study of cognitive development. The combination of psychological theory (here differentiation and integration) and the analysis of patterns of responses to a set of well-chosen items constitutes a major improvement on traditional interviewing methods, which suffer from problems of objectivity, falsifiability, and replicability. However, from a statistical modeling point of view, we judge RAM to be suboptimal. This is not to say that RAM, in the hands of experts like Siegler and Chen, necessarily produces incorrect results. But RAM as a statistical method can be improved greatly by using adequate statistical techniques based on latent structure and finite mixture modeling (Clogg, 1995). Two basic techniques are latent class analysis (LCA) and binomial mixture analysis (BMA). These techniques (a) prevent us from accepting categories that are not actually supported by the data; (b) enable us to detect unanticipated categories; (c) allow us to estimate model parameters (and their standard errors) that characterize and help us to interpret the categories; (d) allow us to compare different classifications using statistical goodness of fit measures; (e) allow us to classify subjects optimally into the established categories (or latent classes); (f ) allow us to generalize our results to the population. This commentary is not the place to explain the details of LCA or BMA. Fortunately, for both many clear introductions exist (Clogg, 1995; Everitt & Hand, 1981; McCutcheon, 1987; Hagenaars & McCutcheon, 2002), and various commercial and free software packages are available. Most popular packages nowadays are Latent Gold, Panmark, and Mplus. For the analysis presented here we used the free R-package poLCA (Linzer & Lewis, 2007). In cognitive developmental psychology, Thomas and colleagues introduced and extensively applied the

The role of pattern recognition in children's exact enumeration of small numbers

Enumeration can be accomplished by subitizing, counting, estimation, and combinations of these processes. We investigated whether the dissociation between subitizing and counting can be observed in 4- to 6-year-olds and studied whether the maximum number of elements that can be subitized changes with age. To detect a dissociation between subitizing and counting, it is tested whether task manipulations have different effects in the subitizing than in the counting range. Task manipulations concerned duration of presentation of elements (limited, unlimited) and configuration of elements (random, line, dice). In Study 1, forty-nine 4- and 5-year-olds were tested with a computerized enumeration task. Study 2 concerned data from 4-, 5-, and 6-year-olds, collected with Math Garden, a computer-adaptive application to practice math. Both task manipulations affected performance in the counting, but not the subitizing range, supporting the conclusion that children use two distinct enumeration processes in the two ranges. In all age groups, the maximum number of elements that could be subitized was three. The strong effect of configuration of elements suggests that subitizing might be based on a general ability of pattern recognition.