Heather Armson

Factors influencing responsiveness to feedback: on the interplay between fear, confidence, and reasoning processes

Self-appraisal has repeatedly been shown to be inadequate as a mechanism for performance improvement. This has placed greater emphasis on understanding the processes through which self-perception and external feedback interact to influence professional development. As feedback is inevitably interpreted through the lens of one’s self-perceptions it is important to understand how learners interpret, accept, and use feedback (or not) and the factors that influence those interpretations. 134 participants from 8 health professional training/continuing competence programs were recruited to participate in focus groups. Analyses were designed to (a) elicit understandings of the processes used by learners and physicians to interpret, accept and use (or not) data to inform their perceptions of their clinical performance, and (b) further understand the factors (internal and external) believed to influence interpretation of feedback. Multiple influences appear to impact upon the interpretation and uptake of feedback. These include confidence, experience, and fear of not appearing knowledgeable. Importantly, however, each could have a paradoxical effect of both increasing and decreasing receptivity. Less prevalent but nonetheless important themes suggested mechanisms through which cognitive reasoning processes might impede growth from formative feedback. Many studies have examined the effectiveness of feedback through variable interventions focused on feedback delivery. This study suggests that it is equally important to consider feedback from the perspective of how it is received. The interplay observed between fear, confidence, and reasoning processes reinforces the notion that there is no simple recipe for the delivery of effective feedback. These factors should be taken into account when trying to understand (a) why self-appraisal can be flawed, (b) why appropriate external feedback is vital (yet can be ineffective), and (c) why we may need to disentangle the goals of performance improvement from the goals of improving self-assessment.

The processes and dimensions of informed self-assessment: a conceptual model.

Purpose To determine how learners and physicians engaged in various structured interventions to inform self-assessment, how they perceived and used self-assessment in clinical learning and practice, and the components and processes comprising informed self-assessment and factors that influence these. Method This was a qualitative study guided by principles of grounded theory. Using purposive sampling, eight programs were selected in Canada, the United States, the United Kingdom, the Netherlands, and Belgium, representing low, medium, and high degrees of structure/rigor in self-assessment activities. In 2008, 17 focus groups were conducted with 134 participants (53 undergraduate learners, 32 postgraduate learners, 49 physicians). Focus-group transcripts were analyzed interactively and iteratively by the research team to identify themes and compare and confirm findings. Results Informed self-assessment appeared as a flexible, dynamic process of accessing, interpreting, and responding to varied external and internal data. It was characterized by multiple tensions arising from complex interactions among competing internal and external data and multiple influencing conditions. The complex process was evident across the continuum of medical education and practice. A conceptual model of informed self-assessment emerged. Conclusions Central challenges to informing self-assessment are the dynamic interrelationships and underlying tensions among the components comprising self-assessment. Realizing this increases understanding of why self-assessment accuracy seems frequently unreliable. Findings suggest the need for attention to the varied influencing conditions and inherent tensions to progress in understanding self-assessment, how it is informed, and its role in self-directed learning and professional self-regulation. Informed self-assessment is a multidimensional, complex construct requiring further research.

Tensions in informed self-assessment: how the desire for feedback and reticence to collect and use it can conflict.

Purpose Informed self-assessment describes the set of processes through which individuals use external and internal data to generate an appraisal of their own abilities. The purpose of this project was to explore the tensions described by learners and professionals when informing their self-assessments of clinical performance. Method This 2008 qualitative study was guided by principles of grounded theory. Eight programs in five countries across undergraduate, postgraduate, and continuing medical education were purposively sampled. Seventeen focus groups were held (134 participants). Detailed analyses were conducted iteratively to understand themes and relationships. Results Participants experienced multiple tensions in informed self-assessment. Three categories of tensions emerged: within people (e.g., wanting feedback, yet fearing disconfirming feedback), between people (e.g., providing genuine feedback yet wanting to preserve relationships), and in the learning/practice environment (e.g., engaging in authentic self-assessment activities versus “playing the evaluation game”). Tensions were ongoing, contextual, and dynamic; they prevailed across participant groups, infusing all components of informed self-assessment. They also were present in varied contexts and at all levels of learners and practicing physicians. Conclusions Multiple tensions, requiring ongoing negotiation and renegotiation, are inherent in informed self-assessment. Tensions are both intraindividual and interindividual and they are culturally situated, reflecting both professional and institutional influences. Social learning theories (social cognitive theory) and sociocultural theories of learning (situated learning and communities of practice) may inform our understanding and interpretation of the study findings. The findings suggest that educational interventions should be directed at individual, collective, and institutional cultural levels. Implications for practice are presented.

Features of assessment learners use to make informed self-assessments of clinical performance.

Medical Education 2011: 45: 636–647

Facilitated Reflective Performance Feedback: Developing an Evidence- and Theory-Based Model That Builds Relationship, Explores Reactions and Content, and Coaches for Performance Change (R2C2)

Purpose To develop and conduct feasibility testing of an evidence-based and theory-informed model for facilitating performance feedback for physicians so as to enhance their acceptance and use of the feedback. Method To develop the feedback model (2011–2013), the authors drew on earlier research which highlights not only the factors that influence giving, receiving, accepting, and using feedback but also the theoretical perspectives which enable the understanding of these influences. The authors undertook an iterative, multistage, qualitative study guided by two recognized research frameworks: the UK Medical Research Council guidelines for studying complex interventions and realist evaluation. Using these frameworks, they conducted the research in four stages: (1) modeling, (2) facilitator preparation, (3) model feasibility testing, and (4) model refinement. They analyzed data, using content and thematic analysis, and used the findings from each stage to inform the subsequent stage. Results Findings support the facilitated feedback model, its four phases—build relationship, explore reactions, explore content, coach for performance change (R2C2)—and the theoretical perspectives informing them. The findings contribute to understanding elements that enhance recipients’ engagement with, acceptance of, and productive use of feedback. Facilitators reported that the model made sense and the phases generally flowed logically. Recipients reported that the feedback process was helpful and that they appreciated the reflection stimulated by the model and the coaching. Conclusions The theory- and evidence-based reflective R2C2 Facilitated Feedback Model appears stable and helpful for physicians in facilitating their reflection on and use of formal performance assessment feedback.

Feedback data sources that inform physician self-assessment

Background: Self-assessment is a process of interpreting data about ones performance and comparing it to explicit or implicit standards. Aim: To examine the external data sources physicians used to monitor themselves. Methods: Focus groups were conducted with physicians who participated in three practice improvement activities: a multisource feedback program; a program providing patient and chart audit data; and practice-based learning groups. We used grounded theory strategies to understand the external sources that stimulated self-assessment and how they worked. Results: Data from seven focus groups (49 physicians) were analyzed. Physicians used information from structured programs, other educational activities, professional colleagues, and patients. Data were of varying quality, often from non-formal sources with implicit (not explicit) standards. Mandatory programs elicited variable responses, whereas data and activities the physicians selected themselves were more likely to be accepted. Physicians used the information to create a reference point against which they could weigh their performance using it variably depending on their personal interpretation of its accuracy, application, and utility. Conclusions: Physicians use and interpret data and standards of varying quality to inform self-assessment. Physicians may benefit from regular and routine feedback and guidance on how to seek out data for self-assessment.

Virtual patient activity patterns for clinical learning

Virtual patients are software tools that present learners with patient case situations and tasks. Some virtual patients take the learner through a guided case scenario, whereas others require learners to make diagnostic and therapeutic decisions. Much attention has been paid to the design of virtual patients and their use as standalone activities, but rather less attention has been paid to their use in broader educational activities. This article describes a series of activity patterns that make use of virtual patients.

Affordances of knowledge translation in medical education: a qualitative exploration of empirical knowledge use among medical educators.

Purpose Little is known about knowledge translation processes within medical education. Specifically, there is scant research on how and whether faculty incorporate empirical medical education knowledge into their educational practices. The authors use the conceptual framework of affordances to examine factors within the medical education practice environment that influence faculty utilization of empirical knowledge. Method In 2012, the authors, using a purposive sampling strategy, recruited medical education leaders in undergraduate medical education from a Canadian university. Recruits all had direct teaching and curricular development roles in either preclinical or clinical courses across the four years of the undergraduate curriculum. Data were collected through individual semistructured interviews on participants’ use of empirical evidence, as well as the factors that influence integration of empirical knowledge into practice. Data were analyzed using thematic analysis. Results Fifteen medical educators participated. The authors identified both constraining and facilitating affordances of empirical medical education knowledge use. Constraining affordances included poor quality and availability of evidence, inadequate knowledge delivery approaches, work and role overload, faculty and student change resistance, and resource limitations. Facilitating affordances included faculty development, peer recommendations, and local involvement in medical education knowledge creation. Conclusions Affordances of the medical education practice environment influence empirical knowledge use. Developing strategies for effective knowledge translation thus requires careful assessment of contextual factors that can enable, constrain, or inhibit evidence use. Empirical knowledge use is most likely to occur among medical educators who are afforded rich, facilitative opportunities for participation in creating, seeking, and implementing knowledge.

Is the Cognitive Complexity of Commitment‐to‐Change Statements Associated With Change in Clinical Practice? An Application of Bloom's Taxonomy

Introduction: This study categorizes 4 practice change options, including commitment‐to‐change (CTC) statements using Blooms taxonomy to explore the relationship between a hierarchy of CTC statements and implementation of changes in practice. Our hypothesis was that deeper learning would be positively associated with implementation of planned practice changes. Methods: Thirty‐five family physicians were recruited from existing practice‐based small learning groups. They were asked to use their usual small‐group process while exploring an educational module on peripheral neuropathy. Part of this process included the completion of a practice reflection tool (PRT) that incorporates CTC statements containing a broader set of practice change options—considering change, confirmation of practice, and not convinced a change is needed (“enhanced” CTC). The statements were categorized using Blooms taxonomy and then compared to reported practice implementation after 3 months. Results: Nearly all participants made a CTC statement and successful practice implementation at 3 months. By using the “enhanced” CTC options, additional components that contribute to practice change were captured. Unanticipated changes accounted for one‐third of all successful changes. Categorizing statements on the PRT using Blooms taxonomy highlighted the progression from knowledge/comprehension to application/analysis to synthesis/evaluation. All PRT statements were classified in the upper 2 levels of the taxonomy, and these higher‐level (deep learning) statements were related to higher levels of practice implementation. Conclusion: The “enhanced” CTC options captured changes that would not otherwise be identified and may be worthy of further exploration in other CME activities. Using Blooms taxonomy to code the PRT statements proved useful in highlighting the progression through increasing levels of cognitive complexity—reflecting deep learning.

Encouraging Reflection and Change in Clinical Practice: Evolution of a Tool

This article describes the systematic development and gradual transformation of a tool to guide participants in a continuing medical education program to reflect on their current practices and to make commitments to change. The continuous improvement of this tool was influenced by evolving needs of the program, reviews of relevant educational literature, feedback from periodic program surveys, interviews with group facilitators, and results from educational research studies. As an integral component of the educational process used in the Practice Based Small Group Learning Program, the current tool is designed to help family physicians think about what has been learned during each educational session and examine issues related to the implementation of evidence-based changes into their clinical practice. Lessons learned will be highlighted. Both the developmental processes employed and the practice reflection tool itself have applicability to other educational environments that focus on continuing professional development.