Claus H. Carstensen

Multidimensional Rasch Measurement via Item Component Models and Faceted Designs

A multidimensional Rasch model is presented here, which is a multidimensional extension of item component models. Relations to existing multidimensional item response theory models are discussed. Apart from other applications, it is also suitable for analyzing tests and questionnaires, which are designed according to two or more facets. An application to a 77-item questionnaire on students’ interest in physics with a two-facet structure demonstrates that the model parameters can even be estimated when 17 latent dimensions are to be measured simultaneously (by means of joint maximum likelihood methods).

Sind die Big Five Rasch-skalierbar?1 1 Diese Arbeit entstand im Rahmen des Forschungsprojektes “Mischverteilungsmodelle” am IPN - Institut für die Pädagogik der Naturwissenschaften - Kiel mit Unterstützung der Deutschen Forschungsgemeinschaft unter dem Titel RO 665-4.

Zusammenfassung. Es werden die Originaldaten der Entwicklung der deutschen Ubersetzung des NEO-FFI-Inventars mit Modellen der Item-Response-Theorie reanalysiert. Dabei wird der Frage nachgegangen, inwieweit die 5 Stufen des Antwortformates eine Ordinalskala bilden, die mit dem gemessenen Trait korrespondiert. Es wird gepruft, inwieweit das eindimensionale ordinale Rasch-Modell fur jede der 5 Skalen angemessen ist, und es wird mit dem mixed Rasch-Modell Personenheterogenitat im Sinne einer zwei-Klassen-Struktur nachgewiesen, die jedoch allein auf unterschiedliche response sets der befragten Personen zuruckzufuhren ist. Lediglich die Extraversionsskala erweist sich als substantiell heterogen. Die Verzerrung der Personenmeswerte infolge unterschiedlicher response sets wird durch die Anwendung des mixed Rasch-Modells gleichsam automatisch korrigiert.

The Role of Content and Context in PISA Interest Scales: A study of the embedded interest items in the PISA 2006 science assessment

This paper focuses interest in science as one of the attitudinal aspects of scientific literacy. Large‐scale data from the Programme for International Student Assessment (PISA) 2006 are analysed in order to describe student interest more precisely. So far the analyses have provided a general indicator of interest, aggregated over all contexts and contents in the science test. With its innovative approach PISA embeds interest items within the cognitive test unit and its contents and contexts. The main difference from conventional interest measures is that in most questionnaires, a relatively small number of interest items cover broad fields of contents and contexts. The science units represent a number of systematically differentiated scientific contexts and contents. The units’ stimulus texts allow for concrete descriptions of relevant content aspects, applications, and contexts. In the analyses, multidimensional item response models are applied in order to disentangle student interest. The results indicate that multidimensional models fit the data. A two‐dimensional model separating interest into two different knowledge of science dimensions described in the PISA science framework is further analysed with respect to gender, performance differences, and country. The findings give a comprehensive description of students’ interest in science. The paper deals with methodological problems and describes requirements of the test construction for further assessments. The results are discussed with regard to their significance for science education.

Multidimensional Modeling of Traits and Response Styles

Response styles can influence item responses in addition to a respondent’s latent trait level. A common concern is that comparisons between individuals based on sum scores may be rendered invalid by response style effects. This paper investigates a multidimensional approach to modeling traits and response styles simultaneously. Models incorporating different response styles as well as personality traits (Big Five facets) were compared regarding model fit. Relationships between traits and response styles were investigated and different approaches to modeling extreme response style (ERS) were compared regarding their effects on trait estimates. All multidimensional models showed a better fit than the unidimensional models, indicating that response styles influenced item responses with ERS showing the largest incremental variance explanation. ERS and midpoint response style were mainly trait-independent whereas acquiescence and disacquiescence were strongly related to several personality traits. Expected a posteriori estimates of participants’ trait levels did not differ substantially between two-dimensional and unidimensional models when a set of heterogeneous items was used to model ERS. A minor adjustment of trait estimates occurred when the same items were used to model ERS and the trait, though the ERS dimension in this approach only reflected scale-specific ERS, rather than a general ERS tendency.

Reversed Thresholds in Partial Credit Models: A Reason for Collapsing Categories?

When questionnaire data with an ordered polytomous response format are analyzed in the framework of item response theory using the partial credit model or the generalized partial credit model, reversed thresholds may occur. This led to the discussion of whether reversed thresholds violate model assumptions and indicate disordering of the response categories. Adams, Wu, and Wilson showed that reversed thresholds are merely a consequence of low frequencies in the categories concerned and that they do not affect the order of the rating scale. This article applies an empirical approach to elucidate the topic of reversed thresholds using data from the Revised NEO Personality Inventory as well as a simulation study. It is shown that categories differentiate between participants with different trait levels despite reversed thresholds and that category disordering can be analyzed independently of the ordering of the thresholds. Furthermore, we show that reversed thresholds often only occur in subgroups of participants. Thus, researchers should think more carefully about collapsing categories due to reversed thresholds.

Application of Multivariate Rasch Models in International Large-Scale Educational Assessments

In large-scale educational assessments, such as the Programme for International Student Assessment (PISA) and the Trends in Mathematics and Science Study (TIMSS), a primary concern is with the estimation of the populationlevel characteristics of a number of latent variables and the relationships between latent variables and other variables. Typically these studies are undertaken in contexts in which there are constraints on sample size and individual student response time, yet there are high expectations with regard to the breadth of content coverage. These demands and constraints have resulted in such studies using rotated-booklet designs, with each student responding to a limited number of items on each of a number of scales. This paper describes the techniques that have been employed in such studies to enable the reliable estimation of population characteristics when there is considerable unreliability at the student level. It also discusses the methodology that is used to make the data sets produced in such studies amenable for use by data analysts undertaking secondary analyses using standard analytic tools.

Introduction: Extending the Rasch Model

The present volume is a collection of chapters on research and development work on extensions of the Rasch model (RM; Rasch, 1960) that have focused on relaxing some fundamental constraints of the original RM, while preserving many of the unique features of the model. More specifically, the volume presents extensions of the RM in which certain homogeneity assumptions on the item level and the population level have been relaxed. With these two types of assumption intact, the original RM decomposes the probability of item responses in two independent components: an item-specific difficulty parameter that is constant across all examinees in the population, and one ability parameter for each examinee that is the same across all items in a given assessment.

Untersuchungsgegenstand, Fragestellungen und technische Grundlagen der Studie

PISA steht fur „Programme for International Student Assessment“ — ein Programm der zyklischen Erfassung basaler Kompetenzen der nachwachsenden Generation, das von der Organisation fur wirtschaftliche Zusammenarbeit und Entwicklung (OECD) durchgefuhrt und von allen Mitgliedsstaaten gemeinschaftlich getragen und verantwortet wird. PISA ist Teil des Indikatorenprogramms der OECD, dessen Ziel es ist, den OECD-Mitgliedsstaaten vergleichende Daten uber die Ressourcenausstattung, individuelle Nutzung sowie Funktions- und Leistungsfahigkeit ihrer Bildungssysteme zur Verfugung zu stellen (OECD, 1999; OECD, 2001a). Die Bundesrepublik Deutschland beteiligt sich an diesem Programm gemas einer Vereinbarung zwischen dem Bundesministerium fur Bildung und Forschung und der Standigen Konferenz der Kultusminister der Lander.

Naturwissenschaftliche Grundbildung im Ländervergleich

Die Untersuchung naturwissenschaftlicher Kompetenz im Rahmen von PISA orientiert sich an aktuellen Vorstellungen naturwissenschaftlicher Grundbildung. Die internationale Diskussion zur Scientific Literacy betont die Funktion naturwissenschaftlicher Grundbildung fur die Teilhabe an einer durch Naturwissenschaft und Technik gepragten Kultur (Duit, Hausler & Prenzel, 2000; OECD, 1999). Sie bezieht aber ebenfalls Aspekte naturwissenschaftlicher Grundbildung ein, die aus der Tradition europaischer und deutscher Bildungstheorien stammen. Wie an anderer Stelle ausfuhrlich dargelegt, besteht ein weit reichender Konsens uber die Struktur naturwissenschaftlicher Grundbildung und uber wesentliche Kompetenzmerkmale (vgl. Prenzel u.a., 2001).

Assessment to Improve Learning in Mathematics: the BEAR Assessment System

The Berkeley Evaluation and Assessment Research (BEAR) Center has for the last several years been involved in the development of an assessment system, which we call the BEAR Assessment System. The system consists of four principles, each associated with a practical “building block” [Wilson 2005] as well as an activity that helps integrate the four parts together (see the section starting on p. 325). Its original deployment was as a curriculum-embedded system in science [Wilson et al. 2000], but it has clear and logical extensions to other contexts such as in higher education [Wilson and Scalise 2006], in largescale assessment [Wilson 2005]; and in disciplinary areas, such as chemistry [Claesgens et al. 2002], and the focus of this chapter, mathematics. In this paper, the four principles of the BEAR Assessment System are discussed, and their application to large-scale assessment is described using an example based on a German assessment of mathematical literacy used in conjunction with the Program for the International Student Assessment [PISA 2005a]; see also Chapter 7, this volume). The BEAR Assessment System is based on a conception of a tight inter-relationship between classroom-level and large-scale assessment [Wilson 2004a; Wilson and Draney 2004]. Hence, in the process of discussing this large-scale application, some arguments and examples will be directed towards classroom-level applications, or, more accurately, towards the common framework that binds the two together [Wilson 2004b].