Madalena Patricio

Is the OSCE a feasible tool to assess competencies in undergraduate medical education

Background: The Objective Structured Clinical Examination (OSCE) was introduced by Harden et al. (1975) trying to answer the problems regarding the assessment of clinical competencies. Despite increasingly widespread use of OSCEs, debate continues with arguments as ‘why using such a demanding format if other methods are available?’ Aim: To review and synthesize evidence on technical and economic feasibility of OSCE in undergraduate medical studies. Methods: Best Evidence Medical Education methodology was applied by two independent coders to 1083 studies identified by literature search from 1975 until the end of 2008. Key findings: The OSCE is a feasible approach to the assessment of clinical competence for use in different cultural and geographical contexts; to assess a wide range of learning outcomes; in different specialties and disciplines; for formative and summative purposes; to assess students a curriculum or an educational intervention; in the different phases of education including the early and later years of the undergraduate curriculum; and in different health care professions. Conclusion: Despite being an expensive test format, evidence suggests that the use of OSCE produces reliable results. The study also suggests that one reason for the wide-scale adoption of the OSCE and the feasibility of its use in different contexts and situations is its inherent flexibility in terms of the number of students that can be assessed, the number of examiners included, the type of patients represented and the format of the examination itself, including the length of the examination, the number and duration of stations.

A comprehensive checklist for reporting the use of OSCEs.

Background: The Objective Structured Clinical Examination (OSCE) has experienced an explosion of use which has rarely been accompanied by systematic investigations on its validity, reliability and feasibility. A systematic review of OSCE was undertaken as part of Best Evidence Medical Education at the Centre for Evidence Based Medicine of the Faculty of Medicine of the University of Lisbon. Several problems were identified with published papers relating to completeness of information presented, methodological issues or the use of terminology. Aim: To identify a need for standardization within the reporting of OSCE studies in medical education based in the first 104 papers of the aforementioned review. Method: Two independent reviewers coded each paper. Results: The most important problem identified was the lack of information, followed by the degree of inconsistency when reporting on OSCEs (papers with missing data and papers where data was given in a way that interpretation is difficult or impossible in terms of evidence; heterogeneity in reporting, lack of a standardized vocabulary, statistical errors and lack of structure within reporting). Conclusions: The authors present a ‘Comprehensive Checklist for those describing the use of OSCEs in the report of educational literature’ as an attempt to encourage better report standards.

The Bologna Process – A global vision for the future of medical education

What has become known as the Bologna Process has evolved over a series of ministerial conferences with the last meeting in Leuven and Louvain-la-Neuve in 2009. There has been a move towards recognition of the benefits to be gained from greater transparency, a general recognition of degrees across Europe, cooperation with regard to quality assurance, an emphasis on more flexible learning paths and lifelong learning, and the promotion of mobility. This paper highlights the ambitious objectives underpinning the Bologna Declaration and Process and the developments since the 1999 Declaration and the current position in particular with regard to medicine. The paper describes common myths and misunderstandings about the Process relating to the two cycle model, the progress of students after the first cycle and the concept of harmonisation rather than uniformity. It is argued that the Bologna Process can serve as a catalyst for the necessary change in medical education. With careful management and imaginative implementation and the necessary vision, creativity and enthusiasm, any problems can be circumnavigated and rich rewards achieved. The Bologna Process is constantly evolving and its dynamic nature is one of its strengths. Medicine has much to contribute and should be part of this Process.

Implementation of the Bologna two-cycle system in medical education: Where do we stand in 2007?–Results of an AMEE-MEDINE survey

Background: The Bologna Declaration aims to harmonize European higher education. At workshops held at AMEE Conferences (2001/2007), it was observed that medical educators seem unaware of Bologna Declaration policies in their own countries. Specifically the objective to structure higher education in two cycles evokes strong opinions, but an overview on the implementation progress is lacking. Method: To determine the present state of implementation of the Bologna two-cycle system in medical education, an AMEE–MEDINE survey was sent to all forty-six signatory countries, inquiring about legislative decisions. Results: Not all answers were unequivocal, but it appears that only seven countries decided for adoption and nineteen decided not to adopt it. The remainder fifteen have not decided or leaves the decision to their medical schools. Non-European countries seem to reject the system more often than European countries. Discussion: We found that very few persons are well informed about national policies and harmonization of medical education does not seem to be enhanced by the Bologna Declaration. Our findings point in the direction of a diversification regarding curricula structure. There is a need for clarity and dialogue on many aspects of Medical Education. The Bologna process could serve as a vehicle to reach this goal.

Bologna in Medicine Anno 2012: Experiences of European medical schools that implemented a Bologna two-cycle curriculum – An AMEE-MEDINE2 survey

Background: The 1999 Bologna Agreement implies a European harmonization of higher education using three cycles: bachelor and master before doctorate. Undergraduate medical programmes were restructured in only seven of the 47 countries. Aim: Given the debate about a two-cycle system in undergraduate medical education, providing an overview of experiences in medical schools that applied this structure was the purpose of this investigation. Methods: In 2009, an AMEE-MEDINE2 survey was carried out among all the 32 medical schools that applied the two-cycle system in medicine. At the end of 2011, a member-check validation using a draft manuscript was carried out to complete an accurate up-to-date impression. Results: All the 32 schools responded initially; 26 schools responded to the second round. All schools had implemented the two-cycle system (all but one in a 3 + 3 year model) with hardly any problems. All reported smaller or larger curriculum improvements, often triggered, but not caused, by the two-cycle system. No school reported that introducing the system interfered with any desired curriculum development, particularly horizontal or vertical integration. Conclusion: In 32 of the 442 medical schools in Bologna signatory countries, introducing a two-cycle model for basic medical education was successfully completed. However, harmonization of medical training in Europe requires further international collaboration.

Systematic reviews of evidence in medical education and clinical medicine: Is the nature of evidence similar?

Background: It is accepted worldwide that clinical and educational decisions should be informed by the best available evidence, not individual opinion only. Aims: This article discusses the epistemological basis of educational evidence, as compared with clinical evidence, looking at the different nature of the science behind each one. Method: Two models – BEME Reviews in medical education and Cochrane Reviews in clinical medicine – based on our own experience of a soon to be published BEME Review (BEMER) and several systematic reviews our group has published in clinical medicine – were used to identify similarities and differences between the two approaches. Key findings: The evidence to support clinical as well as educational decision making is different in its nature, as well as in its quality. However, their approach is similar in its fundamental steps (design a question, select evidence, critically appraise it, synthesize and apply), so the differences between BEME and Cochrane are perhaps more a matter of degree, than the existence of fundamental differences. Conclusions: Two fundamental principles – decision making should be supported by a hierarchy of evidence and evidence alone is never sufficient for sound practice – apply to BEME and Cochrane reviews. The capacity to transfer their results into practice is the most important factor in terms of success of both approaches.

Best Evidence Medical and Health Professional Education (BEME) collaboration: A moving spotlight

Abstract Early this year, a Massachusetts Institute of Technology professor of philosophy presented a new theory of time that caught our interest. One of the key concepts that Professor Skow presented is the “moving spotlight” theory. As Skow asserts, the experiences had a year ago or 10 years ago are still just as real, they’re just “inaccessible” because they are now in a different part of spacetime. This was felt to be an elegant notion to inform this ‘spotlight’ piece on BEME (www.bemecollaboration.org).

Early Interventions and Pediatric Practice

An intervention study using the Brazelton Neonatal Behaviour Assessment Scale (BNBAS) was carried out on 60 primiparous low to medium-class Portuguese mothers. Its aim was to reinforce the mothers’ skills through active learning of some of their babies’ competences. Intervention in the experimental group was made on the third day after delivery and consisted of the assessment of items in the BNBAS such as visual and auditory orientation, consolability with intervention, and cuddliness. The intervention took the form of seven minutes of demonstration to the mothers allied with their active participation. Maternal adjustment and perception of their babies were assessed through specific questionnaires. BNBAS assessment of the babies was performed on the third and twenty-eighth days. Development assessments, through the Bayley Scales of Infant Development (BSID), took place at the third and sixth months. Better results for the experimental group were obtained, especially on aspects concerned with the BNBAS items used in the intervention and those related with mental infant development. A longitudinal perspective of the assessment of babies’ development and behavior is discussed with particular reference to the first six months although this research is designed to examine the intervention effects during the first two years of life.

Enhancing the impact of BEME systematic reviews on educational practice

As medical educators, we believe that the synthesis of our field’s rapidly expanding, kaleidoscopic streams of research is as vital as the production of novel primary research. Clearly, imparting the skills to achieve this to the next generation of medical educators is vital in achieving this goal, hence our intense interest in Ahmadi et al. (2015) timely BEME review. The goals of this review are appropriate and consistent with contemporaneous systematic reviews in the field, focussing on the effectiveness of Evidence-Based Medicine (EBM) teaching. Indeed, the team go further by breaking up the elements of EBM education in an attempt to ascertain their relative importance. The team exemplify how far the BEME collaboration has supported the evolution of health education evidence synthesis, particularly in the area of methodology. Ahmadi et al. (2015) use robust, transparent methods that demonstrate to the reader that reviewer bias has been minimised and that the evidence base presented is reliable and complete. This is at the heart of the BEME vision to systematically review evidence in a manner that supports the shift to evidence-based medical education. Despite these significant strengths, when considering the final synthesised piece, we were left considering if one element is missing. The authors do not describe the various teaching strategies used in the primary studies (e.g. learning outcomes, pedagogy, resources required or lesson plans). This is possibly a shift in perspective that authors may feel is outside the scope of secondary evidence synthesis. Authors may argue that the aim of their reviews are to focus on ‘‘justification’’ of education, namely whether interventions work. Indeed, authors may go on to argue that, given the nature of much primary research reporting medical education, it is highly unlikely that such an attempt would have yielded very much in the way of extracted data. From our perspective, there are two reasons why such an addition may enhance a BEME review. Firstly, when synthesising any groupings of primary research, heterogeneity must always be considered. In the context of reviews of educational interventions, it is important to consider ‘‘educational heterogeneity’’ and we would propose that it is extremely difficult to consider this without having details of the primary education that was delivered. Of course, it may be that such data is not presented and obviously this lack of data presents a risk of bias that should be considered within the discourse of the review, but this possibility does not negate the potential benefits of attempting to extract such data. Secondly, when considering impact, a greater question outside of the context of this review is raised – what do readers want medical education review to offer them? (Gordon et al. 2014). In medical education systematic reviews focussed on a type of teaching or learning, focussing on whether teaching works (justification) and completely ignoring what works, for whom, in what circumstances (description of factors associated to positive or negative outcomes) (Cook et al. 2008) risks under-utilising the significant investment of time and resource needed to complete such reviews. When extracting data, adding such outcomes as well as considering associated factors, can again add a meaningful dimension to the findings, despite the challenges that confounding variables may present. As authors of primary studies are often contacted to clarify methodological data, asking them to also give details of their educational interventions could easily be incorporated. A recent study of non-pharmacological interventional studies in major medical journals found that while reporting of interventions was low, response from the authors after contact was positive in two-thirds of the cases (Hoffmann et al. 2013). This suggestion is not a proposal as to how such data should be utilised, as authors could report anything from a simple summary of the types of teaching used, to a qualitative synthesis of primary education to clarify (Cook et al. 2008) appropriate theoretical elements pertinent in the setting. Rather, we seek to highlight an issue that we often consider when reading health education systematic reviews. Addition of descriptive elements of teaching and learning can only enhance the final product of systematic review, both by enhancing the rigour of the methodology and by offering greater utility for those delivering and innovating medical education. Presentation of the content of the interventions under scrutiny can support dissemination and replication, as well as allowing a deeper level of synthesis to consider issues such as context and learner characteristics. While such activity will add more demands to an already demanding process, we would propose that the potential gains merit consideration of such works in future reviews. While the focus of many BEME reviews appears still to be on ‘‘whether’’ a particular

The Bologna Process – From futility to utility

Davis (2010) highlights in his Commentary in this issue of Medical Teacher , the Bologna Process with its aims of putting in place a system of easily readable and comparable degrees, a two/three-cycle qualification system, a credit transfer and accumulation mechanism, student mobility, quality assurance, a comprehensive system of lifelong learning, wider access and equality of opportunity and measures to promote the attractiveness of European higher education. The Bologna Process envisages a first or Bachelor cycle anticipated as usually of 2–3 years duration and a second or Master cycle, again of 2–3 years duration. ‘A beautiful futility’, however, was the term used by Gordon et al. (2009) to describe the twocycle model and the Bologna Process. Problems and disagreements over the Bachelor Master two-cycle model, regrettably they believe, have dominated the discussions about the implementation of the Bologna Process in Medicine and have distracted from other important aspects of the process. They argue that the two-cycle model is ‘irrelevant, backward looking and arbitrary’ and has been ‘a regrettable waste of time and other resources’. They believe that this results from the fact that the needs of medical education have not been fully taken into account in the development of the Bologna Process. This, however, is a misunderstanding of the Bologna Process. It is not that Medicine has been excluded but rather that the Bologna action lines have been defined at a system level independent of academic discipline, as described by Davis (2010). Gordon and co-authors are concerned that the two-cycle model imposes an arbitrary divide or partition in medical education courses that do not need to be divided. In an article in this issue of Medical Teacher , Cumming (2010) argues a contrary view – that the two- or three-cycle Bologna system adopted in medical education can bring clarity to what is a historically confused area and can encourage the integration of clinical learning with medical sciences at all stages of the student’s journey. The idea that the two-cycle model is in some way a retrograde step returning medical education to the basic science/clinical divide is one of the common myths about the Bologna Process refuted by Patro´cio and Harden (2010) in a further article in this issue. They argue that the two-cycle model, rather than partitioning the medical curriculum, presents an opportunity to develop a spiral curriculum (Harden & Stamper 1999) that reflects the students’ progression as they pass through the different phases of the undergraduate education programme, increasing their mastery and capabilities as they pass from the first to the second phase, with the basic sciences and clinical medicine closely integrated and appropriate learning outcomes described for each curricular cycle or phase. The Bologna Process envisages that students on the completion of the first cycle may choose to leave their studies and seek employment in another field. Gordon et al. (2009) believe that the employment opportunities in other sectors following completion of the first cycle are illusory. Experience, however, in Switzerland has shown that this is not the case and that, for the small number of students who choose to leave their studies after the completion of the first or Bachelor cycle, job opportunities can exist in areas such as medical communication systems, medico-legal work, the pharmaceutical industry and other health-related occupations. Gordon and co-authors also raise thespectre of a cadreof ‘barefoot doctors’. This is a second myth about the Bologna Process highlighted by Patro´cio and Harden (2010). There is no expectation inherent in the twocyclemodelthatstudentsleavingafterthefirstcyclewillpractise as some sort of second-rate doctor.