Daniel V. Holt

Perspectives on Problem Solving in Educational Assessment: Analytical, Interactive, and Collaborative Problem Solving

Problem solving has received broad public interest as an important competency in modern societies. In educational large-scale assessments paper-pencil based analytical problem solving was included first (e.g., Programme for International Student Assessment, PISA 2003). With growing interest in more complex situations, the focus has shifted to interactive problem solving (e.g., PISA 2012) requiring identification and control of complex systems. In the future, collaborative problem solving represents the next step in assessing problem solving ability (e.g., PISA 2015). This paper describes these different approaches to assessing problem solving ability in large-scale assessments considering theoretical questions as well as assessment issues. For each of the three types of problem solving, the definition and understanding of the construct is explained, items examples are shown together with some empirical results, and limitations of the respective approach are discussed. A final discussion centers on the connection of cognitive and differential psychology within educational research and assessment.

Planning impairments in schizophrenia: Specificity, task independence and functional relevance

The present study investigated the specificity of planning impairments in schizophrenia compared to unipolar major depression. Multiple measures of planning ability were employed to assess the task independence of a planning deficit. Furthermore, the predictive power of planning ability with regard to functional outcome was analyzed. A total of 80 participants completed a comprehensive neuropsychological assessment with an emphasis on executive functions and planning ability. The sample consisted of 28 patients with schizophrenia, 28 patients with depression and 24 healthy controls. Both patient groups were impaired on measures of attention, working memory and planning, but only planning ability differentiated between patient groups. The deficit was evident across different measures of planning ability and was the best overall predictor of functional outcome. These results provide evidence for the relative specificity of a planning deficit in schizophrenia and show that the deficit is not task-specific but likely affects central cognitive control processes critical for planned behavior. The observed relation to functional outcome supports the clinical relevance of planning ability.

Planning and problem-solving training for patients with schizophrenia: a randomized controlled trial

BackgroundThe purpose of this study was to assess whether planning and problem-solving training is more effective in improving functional capacity in patients with schizophrenia than a training program addressing basic cognitive functions.MethodsEighty-nine patients with schizophrenia were randomly assigned either to a computer assisted training of planning and problem-solving or a training of basic cognition. Outcome variables included planning and problem-solving ability as well as functional capacity, which represents a proxy measure for functional outcome.ResultsPlanning and problem-solving training improved one measure of planning and problem-solving more strongly than basic cognition training, while two other measures of planning did not show a differential effect. Participants in both groups improved over time in functional capacity. There was no differential effect of the interventions on functional capacity.ConclusionA differential effect of targeting specific cognitive functions on functional capacity could not be established. Small differences on cognitive outcome variables indicate a potential for differential effects. This will have to be addressed in further research including longer treatment programs and other settings.Trial registrationClinicalTrials.gov NCT00507988

Effects of role typicality on processing person information in German: evidence from an ERP study.

The present ERP study investigated how reference resolution is affected by semantic cues to referent gender such as the gender typicality of role names. Participants read general statements about social and occupational groups denoted by a role name (e.g., pilots, florists), which was followed by a co-referring NP with masculine, feminine or neutral lexical gender (e.g., these men/women/people) that could either semantically match, mismatch or be neutral to the role names gender typicality. The study was run in German. Between 300 and 400 ms (N400) responses to all anaphors were more negative after typically male than after typically female antecedents. Between 500 and 700 ms (P600) role name typicality interacted with the lexical gender of the anaphor. Responses to feminine anaphors were more positive in case of incongruent compared to congruent typicality of the antecedent. The effects suggest that participants anticipate a possible semantic incongruity before the anaphor is actually resolved. They, furthermore show that different cues to referent gender are conceptually integrated after 500 ms following the onset of the anaphoric noun. Results are discussed with regard to two-stage models of reference resolution.

When Less Is Less: Solving Multiple Simple Problems Is Not Complex Problem Solving—A comment on Greiff et al. (2015)

In this commentary, we critically review the study of Greiff, Stadler, Sonnleitner, Wolff, and Martin, “Sometimes less is more: Comparing the validity of complex problem solving measures” (Intelligence, 2015, 50, 100–113). The main conclusion of Greiff et al. that the “multiple complex systems” (MCS) approach to measuring complex problem-solving ability possesses superior validity compared to classical microworld scenarios (“less is more”) seems to be an overgeneralization based on inappropriate analysis and selective interpretation of results. In its original form, MCS is a useful tool for investigating specific aspects of problem solving within dynamic systems. However, its value as an instrument for the assessment of complex problem solving ability remains limited.

Shorter identifier names take longer to comprehend

Developers spend the majority of their time reading code, a process in which identifier names play a key role. Although many identifier naming styles exist, they often lack an empirical basis and it is not clear whether short or long identifier names facilitate comprehension. In this paper, we investigate the effect of different identifier naming styles (single letters, abbreviations, and words) on program comprehension. We conducted an experimental study with 72 professional C# developers who had to locate defects in source code snippets. We used a within-subjects design, such that each developer worked with all three versions of identifier naming styles, and we measured the time it took them to find a defect. We found that word identifiers led to a 19% increase in speed to find defects compared to meaningless single letters and abbreviations, but we did not find a difference between letters and abbreviations. The results of our study suggest that code is more difficult to comprehend when it contains only letters and abbreviations as identifier names. Words as identifier names facilitate program comprehension and may help to save costs and improve software quality.

Predictors for Improvement of Problem-Solving during Cognitive Remediation for Patients with Schizophrenia

Cognitive remediation is a promising pathway for ameliorating cognitive impairment of patients with schizophrenia. Here, we investigate predictors of improvement in problem-solving ability for two different types of cognitive remediation - specific problem-solving training and training of basic cognition. For this purpose we conducted a re-analysis of a randomized controlled trial comparing these two training approaches. The main outcome measure was improvement in problem-solving performance. Correlational analyses were used to assess the contribution of clinical, cognitive and training-related predictors. In the problem-solving training group, impaired pre-training planning ability was associated with stronger improvement. In contrast, in the basic cognition training group antipsychotic medication dose emerged as a negative predictor. These results demonstrate that predictors for successful cognitive remediation depend on the specific intervention. Furthermore, our results suggest that at least in the planning domain patients with impaired performance benefit particularly from a specific intervention.

Applying Mathematical Optimization Methods to an ACT-R Instance-Based Learning Model

Computational models of cognition provide an interface to connect advanced mathematical tools and methods to empirically supported theories of behavior in psychology, cognitive science, and neuroscience. In this article, we consider a computational model of instance-based learning, implemented in the ACT-R cognitive architecture. We propose an approach for obtaining mathematical reformulations of such cognitive models that improve their computational tractability. For the well-established Sugar Factory dynamic decision making task, we conduct a simulation study to analyze central model parameters. We show how mathematical optimization techniques can be applied to efficiently identify optimal parameter values with respect to different optimization goals. Beyond these methodological contributions, our analysis reveals the sensitivity of this particular task with respect to initial settings and yields new insights into how average human performance deviates from potential optimal performance. We conclude by discussing possible extensions of our approach as well as future steps towards applying more powerful derivative-based optimization methods.

Model-Based Identification of EEG Markers for Learning Opportunities in an Associative Learning Task with Delayed Feedback

This paper combines a reinforcement learning (RL) model and EEG data analysis to identify learning situations in a associative learning task with delayed feedback. We investigated neural correlates in occipital alpha and prefrontal theta band power of learning opportunities, identified by the RL model. We show that those parameters can also be used to differentiate between learning opportunities which lead to correct learning and those which do not. Finally, we show that learning situations can also be identified on a single trial basis.

TARGETING PLANNING AND PROBLEM SOLVING VERSUS BASIC COGNITION IN COGNITIVE REMEDIATION FOR PATIENTS WITH SCHIZOPHRENIA

Cognitive deficits are important predictors of functional outcome in patients with schizophrenia. Approaches to psychological treatment of cognitive deficits have been subsumed under the term cognitive remediation. There is now converging evidence that cognitive remediation has moderate effects on cognitive performance (McGurk et al., 2007). Importantly, these improvements can generalize to functional outcome, particularly when cognitive remediation is combined with comprehensive rehabilitation. When addressing the question which cognitive functions to target, a major goal is an effect on functional outcome. While an influence of basic cognition on functional outcome has been established, it has been suggested that at least part of this effect is mediated by intermediate functions such as social cognition or learning ability (Green et al., 2000; Vauth et al., 2004). These potential mediators have been proposed as potential targets for training interventions in contrast to the “traditional” training of basic cognition. One cognitive domain of potential clinical relevance is planning and problem-solving (Medalia et al., 2001), which requires the integration of basic cognitive functions. This capacity has been shown to be closely associated with functional outcome.