Samuel Greiff

The Process of Solving Complex Problems

This article is about Complex Problem Solving (CPS), its history in a variety of research domains (e.g., human problem solving, expertise, decision making, and intelligence), and a formal definition and a process theory of CPS applicable to the interdisciplinary field. CPS is portrayed as (a) knowledge acquisition and (b) knowledge application concerning the goal-oriented control of systems that contain many highly interrelated elements (i.e., complex systems). The impact of implicit and explicit knowledge as well as systematic strategy selection on the solution process are discussed, emphasizing the importance of (1) information generation (due to the initial intransparency of the situation), (2) information reduction (due to the overcharging complexity of the problem’s structure), (3) model building (due to the interconnectedness of the variables), (4) dynamic decision making (due to the eigendynamics of the system), and (5) evaluation (due to many, interfering and/or ill-defined goals).

Dynamic Problem Solving: A New Assessment Perspective

This article addresses two unsolved measurement issues in dynamic problem solving (DPS) research: (a) unsystematic construction of DPS tests making a comparison of results obtained in different studies difficult and (b) use of time-intensive single tasks leading to severe reliability problems. To solve these issues, the MicroDYN approach is presented, which combines (a) the formal framework of linear structural equation models as a systematic way to construct tasks with (b) multiple and independent tasks to increase reliability. Results indicated that the assumed measurement model that comprised three dimensions, information retrieval, model building, and forecasting, fitted the data well (n = 114 students) and could be replicated in another sample (n = 140), showing excellent reliability estimates for all dimensions. Predictive validity of school grades was excellent for model building but nonexistent for the other two MicroDYN dimensions and for an additional measure of DPS. Implications are discussed.

Perspectives on Problem Solving in Educational Assessment: Analytical, Interactive, and Collaborative Problem Solving

Problem solving has received broad public interest as an important competency in modern societies. In educational large-scale assessments paper-pencil based analytical problem solving was included first (e.g., Programme for International Student Assessment, PISA 2003). With growing interest in more complex situations, the focus has shifted to interactive problem solving (e.g., PISA 2012) requiring identification and control of complex systems. In the future, collaborative problem solving represents the next step in assessing problem solving ability (e.g., PISA 2015). This paper describes these different approaches to assessing problem solving ability in large-scale assessments considering theoretical questions as well as assessment issues. For each of the three types of problem solving, the definition and understanding of the construct is explained, items examples are shown together with some empirical results, and limitations of the respective approach are discussed. A final discussion centers on the connection of cognitive and differential psychology within educational research and assessment.

Assessing complex problem-solving skills with multiple complex systems

In this paper we propose the multiple complex systems (MCS) approach for assessing domain-general complex problem-solving (CPS) skills and its processes knowledge acquisition and knowledge application. After defining the construct and the formal frameworks for describing complex problems, we emphasise some of the measurement issues inherent in assessing CPS skills with single tasks (i.e., fixed item difficulty, low or unknown reliability, and a large impact of random errors). With examples of the MicroDYN test and the MicroFIN test (two instances of the MCS approach), we show how to adequately score problem-solving skills by using multiple tasks. We discuss implications for problem-solving research and the assessment of CPS skills in general.

Teacher Evaluation of Student Ability: What Roles Do Teacher Gender, Student Gender, and Their Interaction Play?.

Background: Recent decades have been marked by an extensive movement to analyze bias in people’s thinking, especially in gender-related issues. Studies have addressed the question of gender bias in classrooms on different levels—the use of gender in books, learning opportunities determined by students’ gender, or teachers’ gender preferences. Purpose: In this study, we aim to answer the question of whether and under which circumstances the interaction between teacher gender and student gender positively or negatively influences teachers’ evaluations of students’ performance, while controlling for objective measures of students’ performance. For instance, it could be possible that a teacher with the same gender as a student evaluates the student as better than opposite-gender students, independent of their objective performance. Sample: The sample consisted of n > 1,500 Finnish 6th grade students (Mage= 12.67) and their respective class teachers. Design and methods: Students completed several academic skills tests, including a mathematical thinking test, reading comprehension test, and scientific reasoning test. Furthermore, teachers provided their evaluation of each student, evaluating students’ performance in different school subjects and answering questions regarding their probability of academic success. To test whether the teacher-student gender interaction had an effect on the criterion variable, i.e. teachers’ evaluation of the students’ performance, multilevel analyses accounting for between- and within-class effects were applied. Thereby, the effect of students’ objective performance on teachers’ evaluation of the students and main effects of gender were controlled for as covariates. Results: The main results indicated that the interaction between student and teacher gender did not influence teachers’ evaluation of the students. However, regardless of their gender, teachers tended to evaluate girls as better than boys in first language performance (i.e. Finnish language) and potential for success in school. Teacher gender did not influence the evaluation. Conclusions: The results of the study suggest that the interaction between teacher and student gender is unlikely to be a source of possible bias in the evaluations of students in the Finnish educational system.

The complex route to success: complex problem-solving skills in the prediction of university success

ABSTRACT Successful completion of a university degree is a complex matter. Based on considerations regarding the demands of acquiring a university degree, the aim of this paper was to investigate the utility of complex problem-solving (CPS) skills in the prediction of objective and subjective university success (SUS). The key finding of this study was that CPS was not only strongly correlated to objective university success but also explained university grade point averages (GPA) of N = 78 German business students even after intelligence and high-school GPA were controlled for. In addition, priority setting while working on a complex problem explained self-rated SUS. Given this incremental validity over the most established predictors of objective university success and its versatility, we argue that CPS may be a valuable addition to existing predictors in university applicant selection.

What You See Is What You (Don’t) Get: A Comment on Funke’s (2014) Opinion Paper

In an opinion paper published in this journal, Funke (2014) argues that two different types of assessment instruments for complex problem solving (CPS), computer-simulated microworlds (CSMs), and minimal complex systems (MCSs), might require different types of causal cognition. CPS denotes the ability to successfully deal with new, intransparent, and dynamically changing problem situations (Funke, 2001) and is considered one of the most important skills of the 21st century. Given the recent attention CPS has received from both academic and educational stances, for instance, through the Programme for International Student Assessment, which tested CPS in 15-year-old students across more than 40 countries (OECD, 2014), the topic addressed by Funke (2014) is both timely and of high relevance.

Assessment of Complex Problem Solving: What We Know and What We Don’t Know

ABSTRACT Complex Problem Solving (CPS) is seen as a cross-curricular 21st century skill that has attracted interest in large-scale-assessments. In the Programme for International Student Assessment (PISA) 2012, CPS was assessed all over the world to gain information on students’ skills to acquire and apply knowledge while dealing with nontransparent problem situations. This review article summarizes current developments in the assessment of CPS using multiple complex systems and focuses on four topics: (a) What is CPS and why is it relevant, (b) What do we know about CPS, (c) What do we gain from its assessment, and (d) What there is to come. More specifically, we (a) elaborate on how changes in today’s world call for the assessment of CPS, (b) summarize recent empirical results on CPS, pinpoint how log file analyses of students’ behavioral data (c) help to understand differences in CPS, and (d) may deepen the understanding of CPS.

Dynamic Problem Solving: Multiple-Item Testing Based on Minimally Complex Systems

Problem solving and thinking are important issues in contemporary research. With the advent of educational assessment, problem solving has been identified as a cross-curricular competence that plays an important role in educational and in occupational settings. Our research is connected to previous activities in the field of dynamic problem solving. On the basis of Dorner’s “Theory of Operative Intelligence”, we developed assessment instruments (called MicroDYN and MicroFIN) that allow for psychometrically acceptable measurements in the field of dynamic problem solving. MicroDYN is an approach based on linear structural equation systems and requires from the problem solver the identification of input-output connections in small dynamic systems with varying degrees of complexity. MicroFIN is an approach based on finite state automata and requires from the problem solver the identification of transitions of state in small simulated devices, within a variety of backgrounds. Besides developing of the test instruments, we checked the construct validity in relation to intelligence and working memory in a series of studies with pupils, students, and workers. Also, the internal relations between different facets of the global construct “dynamic problem solving” were analyzed.

Linking complex problem solving to opportunity identification competence within the context of entrepreneurship

Today’s working life is increasingly characterized by entrepreneurial challenges. Entrepreneurial challenges start at an individual level with the identification of entrepreneurial opportunities, which is acknowledged as one of the key competencies for lifelong learning. Since the identification of entrepreneurial opportunities relies heavily on the opportunity identification competence (OIC) of individuals, understanding the meaning of OIC is relevant. Until now, OIC has been explored in the young entrepreneurship research field. However, entrepreneurship researchers until now have not fully explored OIC. According to several authors, the research on complex problem solving (CPS) in the cognitive research field might contribute to understanding OIC. In this paper, we review the link between OIC and CPS by comparing the cognitive and entrepreneurship research fields. We argue that those who excel in identifying opportunities share core characteristics with high-level complex problem-solvers. We propose to conduct empirical research in the future to investigate the relation between OIC and CPS within a work context in order to gain more insight into OIC. We believe that the cognitive research field contributes to the entrepreneurship research field and provides a deeper understanding of the initial steps of the entrepreneurial process.