Ellen M. Kok

The challenges of studying visual expertise in medical image diagnosis

Visual expertise is the superior visual skill shown when executing domain‐specific visual tasks. Understanding visual expertise is important in order to understand how the interpretation of medical images may be best learned and taught. In the context of this article, we focus on the visual skill of medical image diagnosis and, more specifically, on the methodological set‐ups routinely used in visual expertise research.

Before your very eyes: the value and limitations of eye tracking in medical education

Medicine is a highly visual discipline. Physicians from many specialties constantly use visual information in diagnosis and treatment. However, they are often unable to explain how they use this information. Consequently, it is unclear how to train medical students in this visual processing. Eye tracking is a research technique that may offer answers to these open questions, as it enables researchers to investigate such visual processes directly by measuring eye movements. This may help researchers understand the processes that support or hinder a particular learning outcome.

AM Last Page: Avoiding Five Common Pitfalls of Experimental Research in Medical Education

References: 1. Cook DA, Beckman TJ. Reflections on experimental research in medical education. Adv Health Sci Educ. 2010;15:455–464. 2. Kirk RE. Experimental design. In: Millsap RE, Maydeu-Olivares A. The SAGE Handbook of Quantitative Methods in Psychology. London, UK: SAGE Publications Ltd; 2009. 3. Issa N, Schuller M, Santacaterina S, et al. Applying multimedia design principles enhances learning in medical education. Med Educ. 2011;45:818–826. 4. Cook DA, Thompson WC, Thomas KG, Thomas MR, Pankratz VS. Impact of self-assessment questions and learning styles in Web-based learning: A randomized, controlled, crossover trial. Acad Med. 2006;81:231–238. 5. Hatala RM, Brooks LR, Norman GR. Practice makes perfect: The critical role of mixed practice in the acquisition of ECG interpretation skills. Adv Health Sci Educ. 2003;8:17–26. 6. Mamede S, van Gog T, Moura AS, et al. Reflection as a strategy to foster medical students’ acquisition of diagnostic competence. Med Educ. 2012;46:464–472. 7. Marquard JL, Henneman PL, He Z, Jo J, Fisher DL, Henneman EA. Nurses’ behaviors and visual scanning patterns may reduce patient identification errors. J Exp App. Psychol 2011;17:247–256.

What We Do and Do Not Know about Teaching Medical Image Interpretation

Educators in medical image interpretation have difficulty finding scientific evidence as to how they should design their instruction. We review and comment on 81 papers that investigated instructional design in medical image interpretation. We distinguish between studies that evaluated complete offline courses and curricula, studies that evaluated e-learning modules, and studies that evaluated specific educational interventions. Twenty-three percent of all studies evaluated the implementation of complete courses or curricula, and 44% of the studies evaluated the implementation of e-learning modules. We argue that these studies have encouraging results but provide little information for educators: too many differences exist between conditions to unambiguously attribute the learning effects to specific instructional techniques. Moreover, concepts are not uniformly defined and methodological weaknesses further limit the usefulness of evidence provided by these studies. Thirty-two percent of the studies evaluated a specific interventional technique. We discuss three theoretical frameworks that informed these studies: diagnostic reasoning, cognitive schemas and study strategies. Research on diagnostic reasoning suggests teaching students to start with non-analytic reasoning and subsequently applying analytic reasoning, but little is known on how to train non-analytic reasoning. Research on cognitive schemas investigated activities that help the development of appropriate cognitive schemas. Finally, research on study strategies supports the effectiveness of practice testing, but more study strategies could be applicable to learning medical image interpretation. Our commentary highlights the value of evaluating specific instructional techniques, but further evidence is required to optimally inform educators in medical image interpretation.

Through rose-coloured glasses : An empirical test of narcissistic overestimation

BACKGROUND Grandiosity is designated as a hallmark trait of narcissism. The current study tested whether narcissistic traits are related to overestimation of a range of agentic performances. METHOD Eighty-five non-patients executed six objective tasks to assess their level of (emotional) intelligence, first impression, attractiveness, social skills and learning performance. They were also asked to estimate their level of functioning in these six areas. This estimation was given on two moments: before and after performing the objective tasks. Our main variable of interest was the degree to which subjective estimation exceeds objective scores. RESULTS Narcissistic traits were related to overestimation of (emotional) intelligence, attractiveness and social skills, particularly at a global level before performing the tasks. LIMITATIONS The use of a homogenous student sample; Mainly agentic performances, were studied. CONCLUSION Overall, the findings provide a theoretical validation of unwarranted feelings of grandiosity as a core narcissistic criterion. Copyright

Even if I showed you where you looked, remembering where you just looked is hard

People know surprisingly little about their own visual behavior, which can be problematic when learning or executing complex visual tasks such as search of medical images. We investigated whether providing observers with online information about their eye position during search would help them recall their own fixations immediately afterwards. Seventeen observers searched for various objects in “Wheres Waldo” images for 3 s. On two-thirds of trials, observers made target present/absent responses. On the other third (critical trials), they were asked to click twelve locations in the scene where they thought they had just fixated. On half of the trials, a gaze-contingent window showed observers their current eye position as a 7.5° diameter “spotlight.” The spotlight “illuminated” everything fixated, while the rest of the display was still visible but dimmer. Performance was quantified as the overlap of circles centered on the actual fixations and centered on the reported fixations. Replicating prior work, this overlap was quite low (26%), far from ceiling (66%) and quite close to chance performance (21%). Performance was only slightly better in the spotlight condition (28%, p = 0.03). Giving observers information about their fixation locations by dimming the periphery improved memory for those fixations modestly, at best.

Radiology education : a radiology curriculum for all medical students?

Abstract Diagnostic errors in radiology are frequent and can cause severe patient harm. Despite large performance differences between radiologists and non-radiology physicians, the latter often interpret medical images because electronic health records make images available throughout the hospital. Some people argue that non-radiologists should not diagnose medical images at all, and that medical school should focus on teaching ordering skills instead of image interpretation skills. We agree that teaching ordering skills is crucial as most physicians will need to order medical images in their professional life. However, we argue that the availability of medical images is so ubiquitous that it is important that non-radiologists are also trained in the basics of medical image interpretation and, additionally in recognizing when radiological consultancy should be sought. In acute situations, basic image interpretations skills can be life-saving. We plead for a radiology curriculum for all medical students. This should include the interpretation of common abnormalities on chest and skeletal radiographs and a basic distinction of normal from abnormal images. Furthermore, substantial attention should be given to the correct ordering of radiological images. Finally, it is critical that students are trained in deciding when to consult a radiologist.

The Neural Implementation of Surgical Expertise Within the Mirror-Neuron System: An fMRI Study

Motor expertise is an important aspect of high-level performance in professional tasks such as surgery. While recently it has been shown that brain activation as measured by functional magnetic resonance imaging (fMRI) within the mirror-neuron system (MNS) is modulated by expertise in sports and music, little is known about the neural underpinnings of professional, e.g., surgical expertise. Here, we investigated whether and (if so) how surgical expertise is implemented in the MNS in medical professionals across three levels of surgical qualification. In order to answer the more specific research question, namely, if the neural implementation of motor expertise develops in a linear or non-linear fashion, the study compares not only brain activation within the MNS related to action observation of novices and experts, but also intermediates. Ten novices (medical students), ten intermediates (residents in orthopedic surgery) and ten experts (orthopedic surgeons) watched 60 video clips (5 s each) of daily-life activities and surgical procedures each while their brain activation was measured using a 3-T fMRI scanner. An established localization procedure was followed to functionally define the MNS for each participant individually. A 2 (video type: daily-life activities, surgical procedures) × 3 (expertise level: novice, intermediate, expert) ANOVA yielded a non-significant interaction. Furthermore, separate analyses of the precentral and parietal part of the MNS also yielded non-significant interactions. However, post hoc comparisons showed that intermediates displayed marginally significantly lower brain activation in response to surgery-related videos within the MNS than novices. No other significant differences were found. We did not find evidence for the hypothesis that the brain-activation level in the MNS evoked by observing surgical videos reflects the level of surgical expertise in the professional task of (orthopedic) surgery. However, the results suggest a potential non-linear relationship between expertise level and MNS-activation level.

Optimizing self-regulation of performance: is mental effort a cue?

Accurate self-regulation of performance is important for trainees. Trainees rely on cues to make monitoring judgments to self-regulate their performance. Ideally, cues and monitoring judgements accurately reflect performance, as measured by cue diagnosticity (the ability of a cue to predict performance) and monitoring accuracy (the ability of a monitoring judgement to predict performance). However, this process is far from perfect, emphasizing the need for more accurate cues and monitoring judgements. Perhaps the mental effort of a task could be a cue used to inform certainty judgements. The purpose of this study is to measure cue utilization and cue diagnosticity of mental effort and monitoring accuracy of certainty for self-regulation of performance. Focused on the task of ECG interpretation, 22 PGY 1-3 Internal Medicine residents at McMaster University provided a diagnosis for 10 ECGs, rating their level of certainty (0–100%) and mental effort (Paas scale, 1–9). 220 ECGs completed by 22 participants were analyzed using path analysis. There was a negative moderate path coefficient between certainty and mental effort (β = − 0.370, p < 0.001), reflecting cue utilization. Regarding cue diagnosticity of mental effort, this was reflected in a small negative path coefficient between mental effort and diagnostic accuracy (β = − 0.170, p = 0.013). Regarding monitoring accuracy, a moderate path coefficient was observed between certainty and diagnostic accuracy (β = 0.343, p < 0.001). Our results support mental effort as a cue and certainty as a monitoring judgement for self-regulated performance. Yet, reported correlations are not very high. Future research is needed to identify additional cues.

Beyond your very eyes: eye movements are necessary, not sufficient

Editor – In a response to our paper ‘Before your very eyes, the values and limitations of eye tracking for medical education’,1 Smith and colleagues discuss two additional critical limitations of eye‐tracking methodology2: (i) looking at something does not equal (consciously) processing it, and (ii) changes in processes do not necessary translate into changes in performance. We argue that these are characteristics instead of limitations of eye tracking. To address the first point, the unique feature of eye tracking is that we can objectively and non‐intrusively measure what information entered the human cognitive system.3 Eye tracking measures the positioning of light on the fovea, which is the only part of the eye that can process details and colour. Such entrance does not guarantee further cognitive processing, even though that is what happens most of the time.4 That is, eye tracking does not directly measure what information is processed, but it indicates what can be processed. Triangulation with other measures, such as verbal protocols, helps to support the interpretation of eye‐tracking data. That said, what we can validly determine from eye tracking alone is what was not looked at and, as a result, has hardly a chance of being processed, falling at best onto the periphery of the eyes retina. Spoken in the terminology of logical empiricism, looking at information is a necessary condition for it being processed cognitively; it is, however, not sufficient. To the second point, we have experienced exactly this issue in our own research: We found that training medical students to systematically inspect radiographs made them ‘look at’ more of an image (measured via eye tracking) but not ‘detect’ (i.e. report) more abnormalities.5 In this study, eye tracking provided us with invaluable information: that training in a systematic viewing strategy does change viewing behaviour but does NOT change diagnostic performance. The implication is that in this specific case, we should focus our efforts on training students in what abnormalities look like rather than focusing on better viewing strategies. Eye tracking provided unique insight into the root cause of the problem, something that a diagnostic performance test alone could not have. Like every research technique, eye tracking has its strengths, but also its limitations. Together with verbal data (how?), eye‐tracking data (what?) can provide unique insights by helping us detect where exactly the challenges lie within a specific task.