Antonia E. E. Baumeister

Validating the Interpretations of PISA and TIMSS Tasks: A Rating Study

Scholastic tests regard cognitive abilities to be domain-specific competences. However, high correlations between competences indicate either high task similarity or a dependence on common factors. The present rating study examined the validity of 12 Programme for International Student Assessment (PISA) and Third or Trends in International Mathematics and Science Study (TIMSS) tasks. Two tasks per competence (reading, mathematics, science, problem solving) from PISA and TIMSS were assessed by 34 teachers and 33 psychology students on 11 scales: difficulty, curriculum reference, knowledge versus thinking, reading competence, verbal ability, math competence, science competence, problem solving, reasoning, general knowledge, and intelligence. Intraclass correlation between two randomly chosen raters was ric =.59. None of the tasks represented the intended target competence concisely. In five PISA tasks, competences other than those intended were seen as being more relevant. TIMSS tasks were seen as more curriculum-related and requiring more school knowledge than PISA tasks. For solving PISA tasks, thinking/reasoning ability and general intelligence were rated as being more important (d = 0.36). Only small differences were found between students’ and teachers’ ratings.

Cognitive abilities of Emirati and German engineering university students.

According to human capital theory, individual competences and personality attributes are relevant for individual productivity and income. Within human capital, intelligence is crucial. To study engineering and work successfully as an engineer, high cognitive abilities are necessary, especially for work in research and development. In a study of 30 German and 30 Emirati engineering students (mean age: 22 years), both groups were tested with mathematical and figural intelligence scales (CogAT). German engineering students achieved a mean IQ of 116, and Emirati students 104 (in converted UK norms). In both groups male students achieved better results than females (2 to 4 IQ point difference). The results are compared with those from PISA and TIMSS. The possible causes of these results, their consequences and strategies for improvement are discussed.