Christina St-Onge

The influence of medical students’ self‐explanations on diagnostic performance

Medical Education 2011: 45: 688–695

Accuracy of Person-Fit Statistics: A Monte Carlo Study of the Influence of Aberrance Rates.

Using a Monte Carlo experimental design, this research examined the relationship between answer patterns’ aberrance rates and person-fit statistics (PFS) accuracy. It was observed that as the aberrance rate increased, the detection rates of PFS also increased until, in some situations, a peak was reached and then the detection rates of PFS decreased with increases in aberrance rates. Furthermore, the results suggest that ECI2Z was somewhat more robust to high levels of aberrance than lz , HT , and U3 when cheating was simulated. The results of this study shed light on a limitation of PFS analysis.

Students' self-explanations while solving unfamiliar cases: the role of biomedical knowledge

General guidelines for teaching clinical reasoning have received much attention, despite a paucity of instructional approaches with demonstrated effectiveness. As suggested in a recent experimental study, self‐explanation while solving clinical cases may be an effective strategy to foster reasoning in clinical clerks dealing with less familiar cases. However, the mechanisms that mediate this benefit have not been specifically investigated. The aim of this study was to explore the types of knowledge used by students when solving familiar and less familiar clinical cases with self‐explanation.

Rater cognition: review and integration of research findings

Given the complexity of competency frameworks, associated skills and abilities, and contexts in which they are to be assessed in competency‐based education (CBE), there is an increased reliance on rater judgements when considering trainee performance. This increased dependence on rater‐based assessment has led to the emergence of rater cognition as a field of research in health professions education. The topic, however, is often conceptualised and ultimately investigated using many different perspectives and theoretical frameworks. Critically analysing how researchers think about, study and discuss rater cognition or the judgement processes in assessment frameworks may provide meaningful and efficient directions in how the field continues to explore the topic.

Self‐explanation in learning clinical reasoning: the added value of examples and prompts

Recent studies suggest that self‐explanation (SE) while diagnosing cases fosters the development of clinical reasoning in medical students; however, the conditions that optimise the impact of SE remain unknown. The example‐based learning framework justifies an exploration of students’ use of their own SEs combined with the study of examples. This study aimed to assess the impact on medical students’ diagnostic performance of: (i) combining students’ SEs with their listening to examples of residents’ SEs, and (ii) the addition of prompts (specific questions) while working with examples.

From See One Do One, to See a Good One Do a Better One: Learning Physical Examination Skills Through Peer Observation

Background: Learning and mastering the skills required to execute physical exams is of great importance and should be fostered early during medical training. Observing peers has been shown to positively influence the acquisition of psychomotor skills. Purpose: The current study investigated the influence of peer observation on the acquisition of psychomotor skills required to execute a physical examination. Methods: Second-year medical students (N = 194) learned the neurological physical examination for low back pain in groups of three. Each student learned and performed the physical examination while the other students observed. Analyses compared the impact of the quantity and the quality of observed performances on students’ learning of the physical examination skills. Results: Students benefited from observing peers while they executed their examination. Moreover, observing a high-performing peer increased the acquisition of physical examination skills. Conclusions: Results suggest that group learning activities that allow students to observe their peers during physical examination should be favored.

To observe or not to observe peers when learning physical examination skills; that is the question

BackgroundLearning physical examination skills is an essential element of medical education. Teaching strategies include practicing the skills either alone or in-group. It is unclear whether students benefit more from training these skills individually or in a group, as the latter allows them to observing their peers. The present study, conducted in a naturalistic setting, investigated the effects of peer observation on mastering psychomotor skills necessary for physical examination.MethodsThe study included 185 2nd-year medical students, participating in a regular head-to-toe physical examination learning activity. Students were assigned either to a single-student condition (n = 65), in which participants practiced alone with a patient instructor, or to a multiple-student condition (n = 120), in which participants practiced in triads under patient instructor supervision. The students subsequently carried out a complete examination that was videotaped and subsequently evaluated. Student’s performance was used as a measure of learning.ResultsStudents in the multiple-student condition learned more than those who practiced alone (81% vs 76%, p < 0.004). This result possibly derived from a positive effect of observing peers; students who had the possibility to observe a peer (the second and third students in the groups) performed better than students who did not have this possibility (84% vs 76%, p <. 001). There was no advantage of observing more than one peer (83.7% vs 84.1%, p > .05).ConclusionsThe opportunity to observe a peer during practice seemed to improve the acquisition of physical examination skills. By using small groups instead of individual training to teach physical examination skills, health sciences educational programs may provide students with opportunities to improve their performance by learning from their peers through modelling.

A Monte Carlo Study of the Effect of Item Characteristic Curve Estimation on the Accuracy of Three Person-Fit Statistics

To date, there have been no studies comparing parametric and nonparametric Item Characteristic Curve (ICC) estimation methods on the effectiveness of Person-Fit Statistics (PFS). The primary aim of this study was to determine if the use of ICCs estimated by nonparametric methods would increase the accuracy of item response theory—based PFS for small sample sizes. Using three recognized PFS (lz, ECI2 z, and ECI4z), four estimation methods were compared: two parametric methods (the two-parameter logistic model and three-parameter logistic model) and two nonparametric methods (Nadaraya-Watsons regression and the local logistic regression). Finally, matrices of 100- and 1,000-answer vectors were generated for this Monte Carlo study. For both large and small sample sizes, the accuracy of the PFS was greater when used with the parametric models.

Is there a way for clinical teachers to assist struggling learners? A synthetic review of the literature

Struggling medical trainees pose a challenge to clinical teachers, since these learners warrant closer supervision that is time-consuming and competes with time spent on patient care. Clinical teachers’ perception that they are ill equipped to address learners’ difficulties efficiently may lead to delays or even lack of remediation for these learners. Because of the paucity of evidence to guide best practices in remediation, the best approach to guide clinical teachers in the field remains to be established. We aimed to present a synthetic review of the empirical evidence and theory that may guide clinical teachers in their daily task of supervising struggling learners, reviewing current knowledge on the challenges and solutions that have been identified and explored. A computerized literature search was performed using Medline, Embase, Education Resources Information Center, and Education Source, after which final articles were selected based on relevance. The literature reviewed provided best evidence for clinical teachers to address learners’ difficulties, which is presented in the order of the four steps inherent to the clinical approach: 1) detecting a problem based on a subjective impression, 2) gathering and documenting objective data, 3) assessing data to make a diagnosis, and 4) planning remediation. A synthesized classification of pedagogical diagnoses is also presented. This review provides an outline of practical recommendations regarding the supervision and management of struggling learners up to the remediation phase. Our findings suggest that future research and faculty development endeavors should aim to operationalize remediation strategies further in response to specific diagnoses, and to make these processes more accessible to clinical teachers in the field.

Expectations, Observations, and the Cognitive Processes That Bind Them: Expert Assessment of Examinee Performance.

Performance-based assessment (PBA) is a valued assessment approach in medical education, be it in a clerkship, residency, or practice context. Raters are intrinsic to PBA and the increased use of PBA has lead to an increased interest in rater cognition. Although several researchers have tackled factors that may influence the variability in rater judgment, the critical examination of rater observation of performance and the translation of that data into judgements are being investigated. The purpose of this study was to qualitatively investigate the cognitive processes of raters, and to create a framework that conceptualizes those processes when raters assess a complex performance. We conducted semi-structured interviews with 11 faculty members (nominated as excellent assessors) from a Department of Medicine to investigate how raters observe, interpret, and translate performance into judgments. The transcribed verbal protocols were analyzed using Constructivist Grounded Theory in order to develop a theoretical model of raters’ assessment processes. Several themes emerged from the data and were grouped according to three macro-level themes describing how the raters balance two sources of data [(1) external sources of information and (2) internal/personal sources of information] by relying on specific cognitive processes to assess an examinee performance. The results from our study demonstrate that assessment is a difficult cognitive task that involves nuance using specific cognitive processes to weigh external and internal data against each other. Our data clearly draws attention to the constant struggle between objectivity and subjectivity that is observed in assessment as illustrated by the importance given to nuancing the examinee’s observed performance.