Anselme Derese

Factors confounding the assessment of reflection: a critical review

BackgroundReflection on experience is an increasingly critical part of professional development and lifelong learning. There is, however, continuing uncertainty about how best to put principle into practice, particularly as regards assessment. This article explores those uncertainties in order to find practical ways of assessing reflection.DiscussionWe critically review four problems: 1. Inconsistent definitions of reflection; 2. Lack of standards to determine (in)adequate reflection; 3. Factors that complicate assessment; 4. Internal and external contextual factors affecting the assessment of reflection.SummaryTo address the problem of inconsistency, we identified processes that were common to a number of widely quoted theories and synthesised a model, which yielded six indicators that could be used in assessment instruments. We arrived at the conclusion that, until further progress has been made in defining standards, assessment must depend on developing and communicating local consensus between stakeholders (students, practitioners, teachers, supervisors, curriculum developers) about what is expected in exercises and formal tests. Major factors that complicate assessment are the subjective nature of reflections content and the dependency on descriptions by persons being assessed about their reflection process, without any objective means of verification. To counter these validity threats, we suggest that assessment should focus on generic process skills rather than the subjective content of reflection and where possible to consider objective information about the triggering situation to verify described reflections. Finally, internal and externalcontextual factors such as motivation, instruction, character of assessment (formative or summative) and the ability of individual learning environments to stimulate reflection should be considered.

Correlation of a written test of skills and a performance based test: a study in two traditional medical schools

Studies in innovative curricula have shown that a written test of skills is potentially able to predict OSCE scores. In this study we verified the potential of a written test of skills as an alternative to an OSCE in two traditional medical schools. A 12 station OSCE was scored using detailed checklists and global rating scales. The written test of skills consisted of 132 true-false questions. Students were assessed immediately before graduation in 1997. The size of the reported correlations confirms that the written test of skills can be used to predict performance-based test scores in traditional medical schools to some extent. For research purposes, particularly for outcome research to compare curricula of medical schools, a written test of skills is a viable alternative to performance-based testing. A written test of skills is unable to replace the OSCE for assessment of individual students.

Knowledge and willingness to teach cardiopulmonary resuscitation: A survey amongst 4273 teachers

INTRODUCTION Schoolteachers are expected to play a role in teaching cardiopulmonary resuscitation (CPR) to schoolchildren, but little is known about their attitudes, actual knowledge and willingness to do so. We conducted a survey about CPR knowledge, preparedness to perform and teach CPR as well as attitude towards an alternative self-learning strategy amongst Flemish teachers. METHODS A questionnaire was developed consisting of four distinct parts: (1) Demographics; (2) CPR knowledge and skills level; (3) Attitude towards training and (4) Resuscitation experience. Content experts screened the questionnaire in view of content validity. One hundred and seventy-one students in Educational Sciences were each asked to interview 25 different teachers. RESULTS A total of 4273 teachers participated in the study (primary school n=856; secondary school n=2562; higher education n=855). Of all respondents, 59% (2539/4273) had received previous CPR training with the highest proportion observed in primary schoolteachers (69%) and in the age group 21-30 years (68%). Mandatory CPR training at school was supported by 41% (1751/4273) of the teachers and only 36% was aware that CPR is now formally included in the secondary education curriculum. Sixty-one percent (2621/4273) did not feel capable and was not willing to teach CPR, mainly because of a perceived lack of knowledge in 50% (2151/2621). In addition 69% (2927/4273) felt incompetent to perform correct CPR and 73% (3137/4273) wished more training. Feeling incompetent and not willing to teach was related to the absence of previous training. Primary schoolteachers and the age group 21-30 years were most willing to teach CPR. CONCLUSIONS Although many teachers mentioned previous CPR training, only a minority of mostly young and primary schoolteachers felt competent in CPR and was willing to teach it to their students.

Unsatisfactory basic skills performance by students in traditional medical curricula

Effectivity of basic clinical skills among students in their final year at two typically European medical schools was evaluated. The two schools rely heavily on clerkships to train basic clinical skills. Three groups of final-year students from two medical schools were studied in a 12-station, 13-minute OSCE. This OSCE was based on existing skill lists and was constructed in close consultation with faculty of both schools. Observers were senior faculty and experienced general practitioners. Students were rated unsatisfactory in any station when a mean score of 50% was given. Results show poor overall performance, indicating defective skills training, for skills of both the compulsory and the non-compulsory clerkships. Therefore curricula need revision and in the meantime basic clinical skills should receive ample attention in postgraduate education.

Impact on knowledge acquisition of the transition from a conventional to an integrated contextual medical curriculum

Context  This study set out to test the hypotheses that after the implementation of an integrated contextual medical curriculum (ICMC), ICMC students would attain higher levels of knowledge in both the basic and clinical sciences at an earlier stage than conventional medical curriculum (CMC) students, that ICMC students would perform significantly better on knowledge tests at the end of their education and, finally, that ICMC students would show a more linear acquisition of knowledge in the basic and clinical sciences.

Impact of selection strategies on representation of underserved populations and intention to practise: international findings

Socially accountable medical schools aim to reduce health inequalities by training workforces responsive to the priority health needs of underserved communities. One key strategy involves recruiting students from underserved and unequally represented communities on the basis that they may be more likely to return and address local health priorities. This study describes the impacts of different selection strategies of medical schools that aspire to social accountability on the presence of students from underserved communities in their medical education programmes and on student practice intentions.

Impact of three alternative consultation training formats on self-efficacy and consultation skills of medical students

Background: Conducting a consultation is a core competence of medical professionals. Consultation training of medical students centers on clinical, communication, reasoning and reflection skills. The training incorporates practice with a standardized simulated patient and supervising physician, to prepare for real patient encounters. To meet the request for more training, while dealing with an increasing student population and limited staff availability, alternative formats of consultation training were developed and evaluated. Aim: To investigate the impact of three consultation training formats on students’ self-efficacy beliefs and their consultation skills acquisition. The three formats comprised (1) traditional training with supervising physician, (2) autonomous training with feedback from simulated patients and peers, without direct supervision and (3) online training based on video fragments and answering guiding questions. Methods: A quasi-experimental pre/posttest study was set up, with random assignment of students to a training condition. The differential impact was tested on two dependent measures: self-efficacy and consultation performance. Self-efficacy was tested with a nine-item scale and the cognitive component of consultation performance was tested on the base of responses to a standardized video case. Results: The autonomous training has a significant positive effect on students’ self-efficacy (p = 0.016). The traditional training and the online training did only positively influence the cognitive component of the consultation competence (p < 0.001 and p = 0.003). Conclusions: Each consultation training contributes to the learning process in a different way. In order to achieve optimum learning effects, medical educators should be aware of the particular impact of specific trainings on the cognitive and motivational side of skills and pursue a balanced mixture of instructional formats.

Does reflection have an effect upon case-solving abilities of undergraduate medical students?

BackgroundReflection on professional experience is increasingly accepted as a critical attribute for health care practice; however, evidence that it has a positive impact on performance remains scarce. This study investigated whether, after allowing for the effects of knowledge and consultation skills, reflection had an independent effect on students’ ability to solve problem cases.MethodsData was collected from 362 undergraduate medical students at Ghent University solving video cases and reflected on the experience of doing so. For knowledge and consultation skills results on a progress test and a course teaching consultation skills were used respectively. Stepwise multiple linear regression analysis was used to test the relationship between the quality of case-solving (dependent variable) and reflection skills, knowledge, and consultation skills (dependent variables).ResultsOnly students with data on all variables available (n = 270) were included for analysis. The model was significant (Anova F(3,269) = 11.00, p < 0.001, adjusted R square 0.10) with all variables significantly contributing.ConclusionMedical students’ reflection had a small but significant effect on case-solving, which supports reflection as an attribute for performance. These findings suggest that it would be worthwhile testing the effect of reflection skills training on clinical competence.

Using video-cases to assess student reflection: development and validation of an instrument.

BackgroundReflection is a meta-cognitive process, characterized by: 1. Awareness of self and the situation; 2. Critical analysis and understanding of both self and the situation; 3. Development of new perspectives to inform future actions. Assessors can only access reflections indirectly through learners’ verbal and/or written expressions. Being privy to the situation that triggered reflection could place reflective materials into context. Video-cases make that possible and, coupled with a scoring rubric, offer a reliable way of assessing reflection.MethodsFourth and fifth year undergraduate medical students were shown two interactive video-cases and asked to reflect on this experience, guided by six standard questions. The quality of students’ reflections were scored using a specially developed Student Assessment of Reflection Scoring rubric (StARS®). Reflection scores were analyzed concerning interrater reliability and ability to discriminate between students. Further, the intra-rater reliability and case specificity were estimated by means of a generalizability study with rating and case scenario as facets.ResultsReflection scores of 270 students ranged widely and interrater reliability was acceptable (Krippendorff’s alpha = 0.88). The generalizability study suggested 3 or 4 cases were needed to obtain reliable ratings from 4th year students and ≥ 6 cases from 5th year students.ConclusionUse of StARS® to assess student reflections triggered by standardized video-cases had acceptable discriminative ability and reliability. We offer this practical method for assessing reflection summatively, and providing formative feedback in training situations.

Are general practitioners prepared to end life on request in a country where euthanasia is legalised

Background In 2002, Belgium set a legal framework for euthanasia, whereby granting and performing euthanasia is entrusted entirely to physicians, and—as advised by Belgian Medical Deontology—in the context of a trusted patient–physician relationship. Euthanasia is, however, rarely practiced, so the average physician will not attain routine in this matter. Aim To explore how general practitioners in Flanders (Belgium) deal with euthanasia. This was performed via qualitative analysis of semistructured interviews with 52 general practitioners (GPs). Results Although GPs can understand a patients request for euthanasia, their own willingness to perform it is limited, based on their assumption that legal euthanasia equates to an injection that ends life abruptly. Their willingness to perform euthanasia is affected by the demanding nature of a patients request, by their views on what circumstances render euthanasia legitimate and by their own ability to inject a lethal dose. Several GPs prefer increasing opioid dosages and palliative sedation to a lethal injection, which they consider to fall outside the scope of euthanasia legislation. Conclusions Four attitudes can be identified: (1) willing to perform euthanasia; (2) only willing to perform as a last resort; (3) feeling incapable of performing; (4) refusing on principle. The situation where GPs have to consider the request and—if they grant it—to perform the act may result in arbitrary access to euthanasia for the patient. The possibility of installing transparent referral and support strategies for the GPs should be further examined. Further discussion is needed in the medical profession about the exact content of the euthanasia law.