Damian P. Birney

Cognitive complexity of suppositional reasoning: An application of the relational complexity metric to the Knight-knave task

An application of the Method of Analysis of Relational Complexity (MARC) to suppositional reasoning in the knight—knave task is outlined. The task requires testing suppositions derived from statements made by individuals who either always tell the truth or always lie. Relational complexity (RC) is defined as the number of unique entities that need to be processed in parallel to arrive at a solution. A selection of five ternary and five quaternary items were presented to 53 psychology students using a pencil and paper format. A computer-administered version was presented to 50 students. As predicted, quaternary problems were associated with higher error rates and longer response times than ternary problems. The computer-administered form was more difficult than the pencil and paper version of the test. These differences are discussed in terms of RC theory and alternative processing accounts. Together, they indicate that the relational complexity metric is a useful and parsimonious way to quantify complexity of reasoning tasks.

Measuring the Influence of Complexity on Relational Reasoning: The Development of the Latin Square Task

Relational complexity (RC) theory conceptualizes an individual’s processing capacity and a task’s complexity along a common ordinal metric. The authors describe the development of the Latin Square Task (LST) that assesses the influence of RC on reasoning. The LST minimizes the role of knowledge and storage capacity and thus refines the identification of a processing-capacity-related complexity effect in task performance. The LST is novel with one explicit rule that is easily understood by adults and children. In two studies, a test of 18 items encompassing three RC levels was administered to university (N = 73; 16-33 years) and school (N = 204; 8-19 years) students. Rasch analyses indicate that the LST scores were psychometrically stable across age groups and provides important diagnostic clues for task development. Consistent with RC theory, the LST is sensitive to parallel and serial (via segmentation) processing demands. The LST provides a strong basis for research on working memory and related constructs (fluid intelligence).

Relational processing and working memory capacity in comprehension of relative clause sentences

Previous research has indicated that the cognitive load imposed by tasks in various content domains increases with the complexity of the relational information processed. Sentence comprehension entails processing noun-verb relations to determinewho did what to whom. The difficulty of object-extracted relative clause sentences might stem from the complex noun-verb relations they entail. Across three studies, participants read 16 types of object- and subject-extracted relative clause sentences at their own pace and then responded to a comprehension question for each sentence. Relational processing was assessed using a premise integration task or a Latin square task. These tasks predicted comprehension of object-relatives before and after controlling for subject-relatives. Working memory (WM) capacity was assessed using reading span or forward and backward digit span tests. WM tasks predicted comprehension of object-relatives before but not after controlling for subject-relatives. Comprehension of object-relatives relied more heavily on a domain-general capacity to process complex relations than on WM capacity.

Simulations, Learning and Real World Capabilities

Purpose – The purpose of this paper is to consider how simulations are increasingly used in training programs for the development of skills such as leadership. However, the requirements of leadership development go beyond the development of task specific procedural knowledge or expertise that simulations have typically been used to develop. Leadership requires flexibility in the application of knowledge developed through simulations and the creation of linkages to behavioral execution skills needed to utilize that knowledge effectively in real world settings.Design/methodology/approach – The successful acquisition of flexible expertise and the related execution skills requires instructional techniques that manage cognitive load, delay automatization of responses, and provide diversity in simulated experiences to ensure richness of the mental models developed while working on simulations. The successful transfer of that knowledge to real world settings requires supplemental instructional techniques that li...

Dynamic assessment as a process-oriented assessment in educational settings

The paper reviews current practices in dynamic assessment to address issues in quantifying learning in educational settings. Brief theoretical foundation and historical background on dynamic assessment and its evolution are followed by a more detailed description of recent advances. Dynamic assessment is conceptualized as a process-oriented assessment that provides a wealth of information about individual and group levels of performance as well as about potential performance in any given domain. It is presented as a meaningful addition to standardized assessment. Particular emphasis is placed on dynamic approaches that are manualized and tend to yield reliable and valid data. In addition, capitalizing on the knowledge available in the field, we present an example of successful implementation of dynamic assessment in an academic domain in classroom settings. Strengths and limitations of research and practice in dynamic assessment are also discussed.

Making Instruction and Assessment Responsive to Diverse Students’ Progress: Group-Administered Dynamic Assessment in Teaching Mathematics:

This study entailed a 3 (instructional intervention) × 2 (assessment-type) between-subjects experimental design employing a pretest—intervention—posttest methodology. The instructional interventions were administered between subjects in three conditions: (a) dynamic instruction, (b) triarchic or theory of successful intelligence-control instruction, and (c) standard-control instruction. The assessment-type consisted between subjects of either (a) a group-administered dynamic posttest or (b) the same group-administered posttest interspersed with a control filler activity. Performance in different mathematics content areas taught in fourth grade was investigated. In total, 1,332 students and 63 classroom teachers in 24 schools across six school districts participated in the study. The results indicate the advantages of using dynamic instruction and assessment in regular classrooms while teaching mathematics, especially when the student body is highly ethnically diverse.

A problem shared is learning doubled

We address the issue of underutilisation of learning opportunities in simulations.71 professionals took part in an experiment using a management simulation.Peer interactions were structured to encourage hypothesis-testing strategies.Simple manipulation of how learners interact with the simulation affected learning.Evidence for proximal, distal and deliberation learning effects is presented. Whilst micro-worlds or simulations have increasingly been used in higher education settings, students do not always benefit as expected from these learning opportunities. By using an experimental-control group design we tested the effectiveness of structuring the task environment so as to encourage learners to approach simulations more systematically. Seventy-one professionals who participated in a postgraduate-level management program worked on a management simulation either individually (n=35) or in dyads (n=36) while exploring the simulation (exploration phase). Peer interactions in the shared learning condition were structured so that learners were encouraged to employ hypothesis testing strategies. All participants then completed the simulation again individually so as to demonstrate what they had learned (performance phase). Baseline measures of cognitive ability and personality were also collected. Learners who explored the simulation in the shared learning condition outperformed their counterparts who explored the simulation individually. A simple manipulation of the way learners interacted with the simulation facilitated learning. Improved deliberation is discussed as a potential cause of this effect, preliminary evidence is provided. This study lends further evidence that the effectiveness of learning using simulations is co-determined by characteristics of the learning environment.

From molehill to mountain: The process of scaling up educational interventions (firsthand experience upscaling the theory of successful intelligence)

Contents: M.A. Constas, R.J. Sternberg, Preface. Part I: Translation of Research and Theory in Content Areas and Skill Domains. M.A. Constas, R.J. Sternberg, Part I Commentary. A. Schoenfeld, Notes on the Educational Steeplechase: Hurdles and Jumps in the Development of Research-Based Mathematics Instruction. S. Magnussen, A. Palincsar, The Application of Theory to the Design of Innovative Texts Supporting Science Instruction. J.M. Fletcher, B.R. Foorman, C.A. Denton, S. Vaughn, Scaling Research on Beginning Reading: Consensus and Conflict. R.C. Calfee, R.G. Mille, K. Norman, K. Wilson, G. Trainin, Learning to Do Educational Research. Part II: Translation of Theory and Research Into Practice in Large Scale Reform. M.A. Constas, R.J. Sternberg, Part II Commentary. R.E. Slavin, Translating Research Into Widespread Practice: The Case of Success for All. C. Finnan, H. Levin, Accelerated Schools and the Obstacles to School Reform. J. Comer, E. Joyner, Translating Theory and Research Into Practice Through the Yale Child Study Center School Development Program. E. Zigler, M.F. Stevenson, The School of the 21st Century. Part III: Translation of Theory and Research Into Educational Practice to Build Intellectual Capacity. M.A. Constas, R.J. Sternberg, Part III Commentary. R.J. Sternberg, D. Birney, L. Jarvin, A. Kirlik, S. Stemler, E.L. Grigorenko, From Molehill to Mountain: The Process of Scaling Up Educational Interventions (First-Hand Experience Upscaling the Theory of Successful Intelligence). J. Renzulli, Swimming Upstream in a Small River: Changing Conceptions and Practices About the Development of Giftedness. M. Kornhaber, H. Gardner, Multiple Intelligences: Developments in Implementation and Theory. Appendix: List of Contributors.

Beyond Psychometrics: The Difference between Difficult Problem Solving and Complex Problem Solving

In this paper we argue that a synthesis of findings across the various sub-areas of research in complex problem solving and consequently progress in theory building is hampered by an insufficient differentiation of complexity and difficulty. In the proposed framework of person, task, and situation (PTS), complexity is conceptualized as a quality that is determined by the cognitive demands that the characteristics of the task and the situation impose. Difficulty represents the quantifiable level of a person’s success in dealing with such demands. We use the well-documented “semantic effect” as an exemplar for testing some of the conceptual assumptions derived from the PTS framework. We demonstrate how a differentiation between complexity and difficulty can help take beyond a potentially too narrowly defined psychometric perspective and subsequently gain a better understanding of the cognitive mechanisms behind this effect. In an empirical study a total of 240 university students were randomly allocated to one of four conditions. The four conditions resulted from contrasting the semanticity level of the variable labels used in the CPS system (high vs. low) and two instruction conditions for how to explore the CPS system’s causal structure (starting with the assumption that all relationships between variables existed vs. starting with the assumption that none of the relationships existed). The variation in the instruction aimed at inducing knowledge acquisition processes of either (1) systematic elimination of presumptions, or (2) systematic compilation of a mental representation of the causal structure underpinning the system. Results indicate that (a) it is more complex to adopt a “blank slate” perspective under high semanticity as it requires processes of inhibiting prior assumptions, and (b) it seems more difficult to employ a systematic heuristic when testing against presumptions. In combination, situational characteristics, such as the semanticity of variable labels, have the potential to trigger qualitatively different tasks. Failing to differentiate between ‘task’ and ‘situation’ as independent sources of complexity and treating complexity and difficulty synonymously threaten the validity of performance scores obtained in CPS research.

Relational processing following stroke

The research examined relational processing following stroke. Stroke patients (14 with frontal, 30 with non-frontal lesions) and 41 matched controls completed four relational processing tasks: sentence comprehension, Latin square matrix completion, modified Dimensional Change Card Sorting, and n-back. Each task included items at two or three levels of relational complexity. Relational processing was impaired in the stroke groups. This was due mainly to items at the intermediate ternary-relational level of complexity. Less complex binary-relational items and more complex quaternary-relational items (the latter are difficult for adults generally) were less sensitive to stroke status. Impairment was greater in frontal than non-frontal stroke patients. Positive inter-correlations among measures supported the domain-general nature of relational processing. Implications for assessment and intervention are discussed.