Bor-Chen Kuo

Cognitive diagnostic models for tests with multiple-choice and constructed-response items

Traditionally, teachers evaluate students’ abilities via their total test scores. Recently, cognitive diagnostic models (CDMs) have begun to provide information about the presence or absence of students’ skills or misconceptions. Nevertheless, CDMs are typically applied to tests with multiple-choice (MC) items, which provide less diagnostic information than constructed-response (CR) items. This paper introduces new CDMs for tests with both MC and CR items, and illustrates how to use them to analyse MC and CR data, and thus, identify students’ skills and misconceptions in a mathematics domain. Analyses of real data, the responses of 497 sixth-grade students randomly selected from four Taiwanese primary schools to eight direct proportion items, were conducted to demonstrate the application of the new models. The results show that the new models can better determine students’ skills and misconceptions, in that they have higher inter-rater agreement rates than traditional CDMs.

Complex Problem Solving in Assessments of Collaborative Problem Solving

Collaborative problem solving (ColPS) proficiency was developed as a new assessment for the Programme for International Student Assessment (PISA) in the 2015 international evaluation of student skills and knowledge. The assessment framework defined by the PISA ColPS 2015 expert group crossed three major collaboration processes with four problem solving processes that were adopted from the PISA 2012 individual problem solving assessment to form a matrix of 12 specific skills. The three major collaboration processes are (1) establishing and maintaining shared understanding; (2) taking appropriate action; and (3) establishing and maintaining team organization. The four problem solving processes are exploring and understanding the problem, representing and formulating the problem, planning and executing strategies, and monitoring and reflecting on the problem-solving activities. This article discusses how the problem-solving dimension was integrated with the collaboration dimension. We also discuss how computer agents were involved in the PISA ColPS 2015 assessment in order to ensure a satisfactory assessment of collaborative problem solving. Examples of the use of agents to assess ColPS are provided in the context of a released PISA item and a project conducted in Taiwan.

Exploratory and Confirmatory Factor Analyses in Reading-Related Cognitive Component among Grade Four Students in Thailand.

Abstract This study aimed to investigate the structure and the validity of the cognitive components of reading in Thai, which is a language with a high degree of grapheme–phoneme correspondence. The participants were 1181 fourth-grade students in 29 schools in Thailand, divided into two subsamples for data analysis. Phoneme isolation, rapid colour naming, rapid number naming, rapid word segmentation, morphological structure and morphological production were developed based on the cognitive theories involved in reading. The reliabilities of the tests ranged from .81 to .98. The structure of the cognitive components of reading in Thai was examined using exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). With subsample 1 (N = 590), the results of EFA support a three-factor structure, measuring morphological awareness, decoding-related and rapid naming skills. With subsample 2 (N = 591), the results of CFA are consistent with the findings of EFA. These findings indicate that the three-factor structure is appropriate to examine the cognitive components of reading in Thai language.