Jörg-Tobias Kuhn

Evaluating cognitive theory: A joint modeling approach using responses and response times

In current psychological research, the analysis of data from computer-based assessments or experiments is often confined to accuracy scores. Response times, although being an important source of additional information, are either neglected or analyzed separately. In this article, a new model is developed that allows the simultaneous analysis of accuracy scores and response times of cognitive tests with a rule-based design. The model is capable of simultaneously estimating ability and speed on the person side as well as difficulty and time intensity on the task side, thus dissociating information that is often confounded in current analysis procedures. Further, by integrating design matrices on the task side, it becomes possible to assess the effects of design parameters (e.g., cognitive processes) on both task difficulty and time intensity, offering deeper insights into the task structure. A Bayesian approach, using Markov Chain Monte Carlo methods, has been developed to estimate the model. An application of the model in the context of educational assessment is illustrated using a large-scale investigation of figural reasoning ability.

Clinical trial on tonal tinnitus with tailor-made notched music training

BackgroundTinnitus is a result of hyper-activity/hyper-synchrony of auditory neurons coding the tinnitus frequency, which has developed due to synchronous mass activity owing to the lack of inhibition. We assume that removal of exactly these frequencies from a complex auditory stimulus will cause the brain to reorganize around tonotopic regions coding the tinnitus frequency through inhibition-induced plasticity. Based on this assumption, a novel treatment for tonal tinnitus - tailor-made notched music training (TMNMT) - has been introduced and was tested in this clinical trial.MethodsA randomized controlled trial in parallel group design was performed in a double-blinded manner. We included 100 participants with chronic, tonal tinnitus who listened to tailor-made notched music for two hours a day for three consecutive months. Our primary outcome measures were the Tinnitus Handicap Questionnaire and Visual Analog Scales measuring perceived tinnitus loudness, awareness, distress and handicap. Participants rated their tinnitus before and after the training as well as one month after cessation of the training.ResultsWhile no effect was found for the primary outcome measures, tinnitus distress, as measured by the Tinnitus Questionnaire, a secondary outcome measure, developed differently in the two groups. The treatment group showed higher distress scores while the placebo group revealed lower distress scores after the training. However, this effect did not reach significance in post-hoc analysis and disappeared at follow-up measurements. At follow-up, tinnitus loudness in the treatment group was significantly reduced as compared to the control group. Post hoc analysis, accounting for low reliability scores in the Visual Analog Scales, showed a significant reduction of the overall Visual Analog Scale mean score in the treatment group even at the post measurement.ConclusionThis is the first study on TMNMT that was planned and conducted following the CONSORT statement standards for clinical trials. The current work is one more step towards a final evaluation of TMNMT. Already after three months the effect of training with tailor-made notched music is observable in the most direct rating of tinnitus perception – the tinnitus loudness, while more global measures of tinnitus distress do not show relevant changes.Trial registrationCurrent Controlled Trials ISRCTN04840953; Trial registration date: 17.07.2013

Number sense or working memory? The effect of two computer-based trainings on mathematical skills in elementary school

Research on the improvement of elementary school mathematics has shown that computer-based training of number sense (e.g., processing magnitudes or locating numbers on the number line) can lead to substantial achievement gains in arithmetic skills. Recent studies, however, have highlighted that training domain-general cognitive abilities (e.g., working memory [WM]) may also improve mathematical achievement. This study addressed the question of whether a training of domain-specific number sense skills or domain-general WM abilities is more appropriate for improving mathematical abilities in elementary school. Fifty-nine children (Mage = 9 years, 32 girls and 27 boys) received either a computer-based, adaptive training of number sense (n = 20), WM skills (n = 19), or served as a control group (n = 20). The training duration was 20 min per day for 15 days. Before and after training, we measured mathematical ability using a curriculum-based math test, as well as spatial WM. For both training groups, we observed substantial increases in the math posttest compared to the control group (d = .54 for number sense skills training, d = .57 for WM training, respectively). Whereas the number sense group showed significant gains in arithmetical skills, the WM training group exhibited marginally significant gains in word problem solving. However, no training group showed significant posttest gains on the spatial WM task. Results indicate that a short training of either domain-specific or domain-general skills may result in reliable short-term training gains in math performance, although no stable training effects were found in the spatial WM task.

Measurement Invariance of Divergent Thinking Across Gender, Age, and School Forms

The present study explores the factorial structure and the degree of measurement invariance of 12 divergent thinking tests. In a large sample of German students (N = 1328), a three-factor model representing verbal, figural, and numerical divergent thinking was supported. Multigroup confirmatory factor analyses revealed that partial strong measurement invariance was tenable across gender and age groups as well as school forms. Latent mean comparisons resulted in significantly higher divergent thinking skills for females and students in schools with higher mean IQ. Older students exhibited higher latent means on the verbal and figural factor, but not on the numerical factor. These results suggest that a domain-specific model of divergent thinking may be assumed, although further research is needed to elucidate the sources that negatively affect measurement invariance.

A Race Model for Responses and Response Times in Tests

Latent trait models for responses and response times in tests are often pure statistical models without a close connection to features of the assumed response process. In the present paper, a new model is presented that is more closely related to assumptions about the response process. The model is based on two increasing stochastic processes. Each stochastic process represents the accumulation of knowledge with respect to one of two response options, the correct and incorrect response. Both accumulators compete and the accumulator that first exceeds a critical level determines the response. General assumptions about the accumulators result in a race between two response times that follow a bivariate Birnbaum Saunders distribution. The model can be calibrated with marginal maximum likelihood estimation. Feasibility of the estimation approach is demonstrated in a simulation study. Additionally, a test of model fit is proposed. Finally, the model will be used for the analysis of an empirical data set.

An accumulator model for responses and response times in tests based on the proportional hazards model.

Latent trait models for responses and response times in tests often lack a substantial interpretation in terms of a cognitive process model. This is a drawback because process models are helpful in clarifying the meaning of the latent traits. In the present paper, a new model for responses and response times in tests is presented. The model is based on the proportional hazards model for competing risks. Two processes are assumed, one reflecting the increase in knowledge and the second the tendency to discontinue. The processes can be characterized by two proportional hazards models whose baseline hazard functions correspond to the temporary increase in knowledge and discouragement. The model can be calibrated with marginal maximum likelihood estimation and an application of the ECM algorithm. Two tests of model fit are proposed. The amenability of the proposed approaches to model calibration and model evaluation is demonstrated in a simulation study. Finally, the model is used for the analysis of two empirical data sets.

Improving Item Response Theory Model Calibration by Considering Response Times in Psychological Tests

Research findings indicate that response times in personality scales are related to the trait level according to the so-called speed–distance hypothesis. Against this background, Ferrando and Lorenzo-Seva proposed a latent trait model for the responses and response times in a test. The model consists of two components, a standard item response model and a supplemental response time model. The authors could demonstrate that the consideration of response time increases the precision of trait estimation. However, the use of the model is not limited to trait estimation. The consideration of response time additionally enhances the calibration of the item response model. Due to limitations of the suggested estimation method for model calibration, Ferrando and Lorenzo-Seva were not able to exploit this additional benefit of response time modeling. In the article, an improved estimation method is proposed that is faster and more efficient than the original approach. It can be shown that the standard deviation of the parameter estimates of the item response model can be reduced up to 25% when applying the new method.

Comorbidity of Arithmetic and Reading Disorder: Basic Number Processing and Calculation in Children with Learning Impairments.

The aim of the present study was to investigate the cognitive profiles of primary school children (age 82–133 months) on a battery of basic number processing and calculation tasks. The sample consisted of four groups matched for age and IQ: arithmetic disorder only (AD; n = 20), reading disorder only (RD; n = 40), a comorbid group (n = 27), and an unimpaired control group (n = 40). Multiple 2 (RD vs. No RD) × 2 (AD vs. No AD) factorial ANCOVAs showed that children with RD had selective impairments in counting and number transcoding efficiency. In contrast, children with AD performed poorly in most tasks, including symbolic and nonsymbolic magnitude comparisons, subitizing, number line estimation, number sets, number transcoding accuracy, and calculation. These findings provide further support that AD is characterized by multiple, heterogeneous underlying deficits. In contrast, RD is associated with specific number processing impairments only if tasks require verbal processing. Taken together, the results fully support the assumption of comorbid additivity of AD and RD.

Begabungsdiagnostik mit dem Grundintelligenztest (CFT 20-R)

Zusammenfassung. Der CFT 20-R ist ein in der psychologischen Praxis haufig eingesetzter Test zur Diagnostik der fluiden Intelligenz. Er dient damit auch zur Diagnose von Hochbegabung. Gegenstand dieses Artikels ist die psychometrische Uberprufung der Kurzform 1 des Grundintelligenztests (CFT 20-R; Weis, 2006) hinsichtlich der Messaquivalenz des Verfahrens zwischen normal- und hoher begabten Schulern. Berichtet werden Ergebnisse auf Item- sowie Subtestebene. Eine Stichprobe von insgesamt N = 1886 Haupt-, Real- sowie Gymnasialschulern beiderlei Geschlechts bearbeitete die Kurzform 1 des CFT 20-R. Mittels DIF-Analysen (Itemebene) kann beim Vergleich von hoher (IQ ≥ 120) sowie normal begabten Schulern (IQ < 120) gezeigt werden, dass grostenteils Messaquivalenz hinsichtlich der Itemschwierigkeiten, weniger hinsichtlich der Trennscharfen vorliegt. Die Messaquivalenz auf der Ebene der Subtests ist hoch. Das Verfahren kann damit im Rahmen eines ersten (Hoch-)Begabungsscreenings gut eingesetzt werden.

A Mixture Proportional Hazards Model With Random Effects for Response Times in Tests

In this article, a new model for test response times is proposed that combines latent class analysis and the proportional hazards model with random effects in a similar vein as the mixture factor model. The model assumes the existence of different latent classes. In each latent class, the response times are distributed according to a class-specific proportional hazards model. The class-specific proportional hazards models relate the response times of each subject to his or her work pace, which is considered as a random effect. The latent class extension of the proportional hazards model allows for differences in response strategies between subjects. The differences can be captured in the hazard functions, which trace the progress individuals make over time when working on an item. The model can be calibrated with marginal maximum likelihood estimation. The fit of the model can either be assessed with information criteria or with a test of model fit. In a simulation study, the performance of the proposed approaches to model calibration and model evaluation is investigated. Finally, the model is used for a real data set.