Valerie Wass

Web-based feedback after summative assessment: how do students engage?

There is little research into how to deliver summative assessment student feedback effectively. The main aims of this study were to clarify how students engage with feedback in this context and to explore the roles of learning‐related characteristics and previous and current performance.

Barriers to the uptake and use of feedback in the context of summative assessment

Despite calls for feedback to be incorporated in all assessments, a dichotomy exists between formative and summative assessments. When feedback is provided in a summative context, it is not always used effectively by learners. In this study we explored the reasons for this. We conducted individual interviews with 17 students who had recently received web based feedback following a summative assessment. Constant comparative analysis was conducted for recurring themes. The summative assessment culture, with a focus on avoiding failure, was a dominant and negative influence on the use of feedback. Strong emotions were prevalent throughout the period of assessment and feedback, which reinforced the focus on the need to pass, rather than excel. These affective factors were heightened by interactions with others. The influence of prior learning experiences affected expectations about achievement and the need to use feedback. The summative assessment and subsequent feedback appeared disconnected from future clinical workplace learning. Socio-cultural influences and barriers to feedback need to be understood before attempting to provide feedback after all assessments. A move away from the summative assessment culture may be needed in order to maximise the learning potential of assessments.

Prevalence, Nature, Severity and Risk Factors for Prescribing Errors in Hospital Inpatients: Prospective Study in 20 UK Hospitals

IntroductionIt has been suggested that doctors in their first year of post-graduate training make a disproportionate number of prescribing errors.ObjectiveThis study aimed to compare the prevalence of prescribing errors made by first-year post-graduate doctors with that of errors by senior doctors and non-medical prescribers and to investigate the predictors of potentially serious prescribing errors.MethodsPharmacists in 20 hospitals over 7 prospectively selected days collected data on the number of medication orders checked, the grade of prescriber and details of any prescribing errors. Logistic regression models (adjusted for clustering by hospital) identified factors predicting the likelihood of prescribing erroneously and the severity of prescribing errors.ResultsPharmacists reviewed 26,019 patients and 124,260 medication orders; 11,235 prescribing errors were detected in 10,986 orders. The mean error rate was 8.8xa0% (95xa0% confidence interval [CI] 8.6–9.1) errors per 100 medication orders. Rates of errors for all doctors in training were significantly higher than rates for medical consultants. Doctors who were 1xa0year (odds ratio [OR] 2.13; 95xa0% CI 1.80–2.52) or 2xa0years in training (OR 2.23; 95xa0% CI 1.89–2.65) were more than twice as likely to prescribe erroneously. Prescribing errors were 70xa0% (OR 1.70; 95xa0% CI 1.61–1.80) more likely to occur at the time of hospital admission than when medication orders were issued during the hospital stay. No significant differences in severity of error were observed between grades of prescriber. Potentially serious errors were more likely to be associated with prescriptions for parenteral administration, especially for cardiovascular or endocrine disorders.ConclusionThe problem of prescribing errors in hospitals is substantial and not solely a problem of the most junior medical prescribers, particularly for those errors most likely to cause significant patient harm. Interventions are needed to target these high-risk errors by all grades of staff and hence improve patient safety.

Factors influencing students’ receptivity to formative feedback emerging from different assessment cultures

IntroductionFeedback after assessment is essential to support the development of optimal performance, but often fails to reach its potential. Although different assessment cultures have been proposed, the impact of these cultures on students’ receptivity to feedback is unclear. This study aimed to explore factors which aid or hinder receptivity to feedback.MethodsUsing a constructivist grounded theory approach, the authors conducted six focus groups in three medical schools, in three separate countries, with different institutional approaches to assessment, ranging from a traditional summative assessment structure to a fully implemented programmatic assessment system. The authors analyzed data iteratively, then identified and clarified key themes.ResultsHelpful and counterproductive elements were identified within each school’s assessment system. Four principal themes emerged. Receptivity to feedback was enhanced by assessment cultures which promoted students’ agency, by the provision of authentic and relevant assessment, and by appropriate scaffolding to aid the interpretation of feedback. Provision of grades and comparative ranking provided a helpful external reference but appeared to hinder the promotion of excellence.ConclusionsThis study has identified important factors emerging from different assessment cultures which, if addressed by programme designers, could enhance the learning potential of feedback following assessments. Students should be enabled to have greater control over assessment and feedback processes, which should be as authentic as possible. Effective long-term mentoring facilitates this process. The trend of curriculum change towards constructivism should now be mirrored in the assessment processes in order to enhance receptivity to feedback.

How we give personalised audio feedback after summative OSCEs.

Abstract Background: Students often receive little feedback after summative objective structured clinical examinations (OSCEs) to enable them to improve their performance. Electronic audio feedback has shown promise in other educational areas. Aim: We investigated the feasibility of electronic audio feedback in OSCEs. Methods: An electronic OSCE system was designed, comprising (1) an application for iPads allowing examiners to mark in the key consultation skill domains, provide “tick-box” feedback identifying strengths and difficulties, and record voice feedback; (2) a feedback website giving students the opportunity to view/listen in multiple ways to the feedback. Acceptability of the audio feedback was investigated, using focus groups with students and questionnaires with both examiners and students. Results: 87 (95%) students accessed the examiners’ audio comments; 83 (90%) found the comments useful and 63 (68%) reported changing the way they perform a skill as a result of the audio feedback. They valued its highly personalised, relevant nature and found it much more useful than written feedback. Eighty-nine per cent of examiners gave audio feedback to all students on their stations. Although many found the method easy, lack of time was a factor. Conclusions: Electronic audio feedback provides timely, personalised feedback to students after a summative OSCE provided enough time is allocated to the process.

Changing the culture of assessment: the dominance of the summative assessment paradigm

BackgroundDespite growing evidence of the benefits of including assessment for learning strategies within programmes of assessment, practical implementation of these approaches is often problematical. Organisational culture change is often hindered by personal and collective beliefs which encourage adherence to the existing organisational paradigm. We aimed to explore how these beliefs influenced proposals to redesign a summative assessment culture in order to improve students’ use of assessment-related feedback.MethodsUsing the principles of participatory design, a mixed group comprising medical students, clinical teachers and senior faculty members was challenged to develop radical solutions to improve the use of post-assessment feedback. Follow-up interviews were conducted with individual members of the group to explore their personal beliefs about the proposed redesign. Data were analysed using a socio-cultural lens.ResultsProposed changes were dominated by a shared belief in the primacy of the summative assessment paradigm, which prevented radical redesign solutions from being accepted by group members. Participants’ prior assessment experiences strongly influenced proposals for change. As participants had largely only experienced a summative assessment culture, they found it difficult to conceptualise radical change in the assessment culture. Although all group members participated, students were less successful at persuading the group to adopt their ideas. Faculty members and clinical teachers often used indirect techniques to close down discussions. The strength of individual beliefs became more apparent in the follow-up interviews.ConclusionsNaïve epistemologies and prior personal experiences were influential in the assessment redesign but were usually not expressed explicitly in a group setting, perhaps because of cultural conventions of politeness. In order to successfully implement a change in assessment culture, firmly-held intuitive beliefs about summative assessment will need to be clearly understood as a first step.

Limits of ‘patient-centredness’: valuing contextually specific communication patterns

Globally, doctor–patient communication is becoming synonymous with high‐quality health care in the 21st century. However, what is meant by ‘good communication’ and whether there is consensus internationally remain unclear.

Sustainability in clinical skills teaching.

The deleterious effects of climate change mean that environmental sustainability is increasingly becoming a moral and economic necessity. Consequently, clinicians will increasingly be called upon to manage the effects of health care on climate change, and they must therefore do as much as is practically possible to limit the negative effects of their practice on the environment. As medical educators we have the opportunity not only to reduce the environmental impact of our own clinical practice, but also that of those who we teach, through innovation. Such novelty can be explored during student‐selected components (SSCs).

Medical students’ action plans are not specific

Action plans have been shown to be important in changing behaviour. In learners, action plans have been proposed as a mechanism by which feedback leads to an increase in expertise: feedback leads to action plans, which lead to changes in learning behaviours and finally to improvement. Little is known about the extent to which students are able to make specific actions plans that relate to the feedback they are given, however. We explored whether medical students created action plans after being given feedback, the quality of those plans and whether the action plans were related to the feedback given.

Very-short-answer questions: reliability, discrimination and acceptability

Single‐best‐answer questions (SBAQs) have been widely used to test knowledge because they are easy to mark and demonstrate high reliability. However, SBAQs have been criticised for being subject to cueing.