Amanda Jarrell

Modeling Metacognitive Activities in Medical Problem-Solving with BioWorld

Medical diagnostic reasoning is ill-defined and complex, requiring novice physicians to monitor and control their problem-solving efforts. Self-regulation is critical for effective medical problem-solving, helping individuals progress towards a correct diagnosis through a series of actions that informs subsequent ones. BioWorld is a computer-based learning environment designed to support novices in developing medical diagnostic reasoning as they receive feedback in the context of solving virtual cases. The system provides tools that scaffold learners in their requisite cognitive and metacognitive activities. Novices attain higher levels of competence as the system dynamically assesses their performance against expert solution paths. Dynamic assessment in this system relies on a novice-expert overlay and it is used to develop feedback when novices request help. When help-seeking occurs, help is provided by the tutoring module which applies a set of pre-defined rules based on the context of the learner’s activity. The system also provides cumulative feedback by comparing the novice solution with an expert solution following completion of the case. This chapter covers the essential design guidelines of this scaffolding approach to metacognitive activities in problem-solving within the domain of medical education. Specifically, we review recent advances in modeling metacognition through online measures, including concurrent think-aloud protocols, video-screen captures, and log-file entries. Educational data mining techniques are outlined with the goals of capturing metacognitive activities as they unfold throughout problem solving, and guiding the design of scaffolding tools in order to promote higher levels of competence in novices.

Examining the Relationship Between Performance Feedback and Emotions in Diagnostic Reasoning: Toward a Predictive Framework for Emotional Support

The purpose of this research is to understand achievement emotions resulting from performance feedback in a medical education context where 30 first and second year medical students learned to diagnose virtual patients in an intelligent tutoring system (ITS), BioWorld. We found that students could be organized into groups using cluster analyses based on the emotions they reported after receiving performance feedback: a positive emotion cluster, negative emotion cluster, and low overall emotion cluster. Medical students in the positive achievement emotion cluster had the highest performance on the diagnostic reasoning cases; those in the negative achievement emotion cluster had the lowest performance; and students categorized as belonging to the low overall achievement emotion cluster had mean performance levels that fell between the two. From the results we propose critical performance thresholds that can be used to predict emotions following performance feedback.

Supporting diagnostic reasoning by modeling help-seeking

Help-seeking is considered an important metacognitive strategy. For learners to benefit from help-seeking, they need to be engaged in effective search behaviors. Maladaptive search behaviors can lead to negative learning outcomes; thus, if learners are cognizant of such behaviors, they can engage in effective help-seeking leading to achieve better learner outcomes. To foster effective help-seeking, we need to better understand the behaviors learners engage in when seeking help. In this study, we examined help-seeking behaviors of learners, in relation to what learners examined in an online library embedded within BioWorld. The library tool is a resource provides information about diseases, diagnostic tests and medical terms. This paper reports the model search behaviors pertaining to the medical topics searched in the library. The findings have implications for developing scaffolding prompts that can encourage effective search behaviors.

The Regulation of Achievements Emotions: Implications for Research and Practice

This article offers a critical review of several influential emotion theories and emotion regulation models in terms of their utility for explaining how, when, and why students regulate their achievement emotions. Based on this review, we propose a novel framework for the regulation of achievement emotions. This framework is based on the premise that student learning and achievement is influenced by both achievement emotions and efforts to regulate these emotions. The framework further proposes that emotion regulation decisions, namely, the identification, selection, and implementation of regulatory strategies, are shaped by 5 antecedent factors: emotion-outcome expectancies, motives for emotion regulation, implicit beliefs about emotions, emotion regulation self-efficacy, and emotion regulation aptitude. The theoretical and practical implications of this framework are discussed. Résumé Le présent article propose une analyse critique de plusieurs théories influentes sur les émotions et les modèles de régulation des émotions relativement à leur utilité pour expliquer comment, quand et pourquoi les élèves régissent leurs émotions en lien avec l’accomplissement. En fonction de cette analyse, nous proposons un cadre novateur pour la régulation des émotions en lien avec l’accomplissement. Ce cadre part du principe que l’apprentissage et la réussite des élèves sont influencés par les émotions en lien avec l’accomplissement et les efforts déployés pour réguler des émotions. Le cadre propose en outre que les décisions en lien avec la régulation des émotions, notamment l’identification, la sélection et la mise en œuvre de stratégies de régulation, sont façonnées par cinq antécédents : les résultats escomptés des émotions, les motifs de régulation des émotions, les croyances implicites au sujet des émotions, l’auto-efficacité de la régulation des émotions et l’aptitude à réguler ses émotions. Les incidences théoriques et pratiques de ce cadre sont évoquées.

Towards Investigating Performance Differences in Clinical Reasoning in a Technology Rich Learning Environment

Technology Rich Learning Environments (TREs) are increasingly used to support scholastic activities. BioWorld is an example of a TRE designed to support the metacognitive activities of learners tasked with solving virtual patient cases. The present paper aims to examine the performance differences of novice physicians in diagnosing cases in BioWorld. We present an empirically guided line of research concerning the performance differences: (1) across three endocrinology cases, (2) between genders, (3) between goal orientations, and (4) in diagnosis correctness.

Learning to Diagnose a Virtual Patient: an Investigation of Cognitive Errors in Medical Problem Solving

Although cognitive errors (i.e., premature closure, faulty data gathering, and faulty knowledge) are the main reasons for making diagnostic mistakes, the mechanisms by which they occur are difficult to isolate in clinical settings. Computer-based learning environments (CBLE) offer the opportunity to train medical students to avoid cognitive errors by tracking the onset of these errors. The purpose of this study is to explore cognitive errors in a CBLE called BioWorld. A logistic regression was fitted to learner behaviors that characterize premature closure in order to predict diagnostic performance. An ANOVA was used to assess if participants who were highly confident in their wrong diagnosis engaged in more faulty data gathering via confirmation bias. Findings suggest that diagnostic mistakes can be predicted from faulty knowledge and faulty data gathering and indicate poor metacognitive awareness. This study supports the notion that to improve diagnostic performance medical education programs should promote metacognitive skills.

Examining Diagnosis Paths: A Process Mining Approach

This paper is motivated by two observations on computer-supported education: First, there has been growing availability, rapid proliferation, and increased diversity of learner-system educational data. Second, advances in learning analytics and data mining have facilitated and spawned a variety of novel investigations using such data. Driven by these complementary trends, the present work is geared towards exploring knowledge-based discovery approaches in understanding learner-system usage data. More specifically, with an eye toward tracing and comprehending learner behaviors in a medical intelligent tutoring system, we explore the utility of Process Mining, in understanding the problem solving trajectories of students in a medical computer-based learning environment.