Deirdre Bennett

A psychometric appraisal of the DREEM

BackgroundThe quality of the Educational environment is a key determinant of a student centred curriculum. Evaluation of the educational environment is an important component of programme appraisal. In order to conduct such evaluation use of a comprehensive, valid and reliable instrument is essential. One of most widely used contemporary tools for evaluation of the learning environment is the Dundee Ready Education Environment Measure (DREEM). Apart from the initial psychometric evaluation of the DREEM, few published studies report its psychometric properties in detail. The aim of this study was to examine the psychometric quality of the DREEM measure in the context of medical education in Ireland and to explore the construct validity of the device.Methods239 final year medical students were asked to complete the DREEM inventory. Anonymised responses were entered into a database. Data analysis was performed using PASW 18 and confirmatory factor analysis performed.ResultsWhilst the total DREEM score had an acceptable level of internal consistency (alpha 0.89), subscale analysis shows that two subscales had sub-optimal internal consistency. Multiple group confirmatory factor analysis (using Flemings indices) shows an overall fit of 0.76, representing a weak but acceptable level of fit. 17 of the 50 items manifest fit indices less than 0.70. We sought the best fitting oblique solution to the 5-subscale structure, which showed large correlations, suggesting that the independence of the separate scales is open to question.ConclusionsThere has perhaps been an inadequate focus on establishing and maintaining the psychometric credentials of the DREEM. The present study highlights two concerns. Firstly, the internal consistency of the 5 scales is quite variable and, in our sample, appears rather low. Secondly, the construct validity is not well supported. We suggest that users of the DREEM will provide basic psychometric appraisal of the device in future published reports.

Anatomy education for the YouTube generation

Anatomy remains a cornerstone of medical education despite challenges that have seen a significant reduction in contact hours over recent decades; however, the rise of the “YouTube Generation” or “Generation Connected” (Gen C), offers new possibilities for anatomy education. Gen C, which consists of 80% Millennials, actively interact with social media and integrate it into their education experience. Most are willing to merge their online presence with their degree programs by engaging with course materials and sharing their knowledge freely using these platforms. This integration of social media into undergraduate learning, and the attitudes and mindset of Gen C, who routinely creates and publishes blogs, podcasts, and videos online, has changed traditional learning approaches and the student/teacher relationship. To gauge this, second year undergraduate medical and radiation therapy students (n = 73) were surveyed regarding their use of online social media in relation to anatomy learning. The vast majority of students had employed web‐based platforms to source information with 78% using YouTube as their primary source of anatomy‐related video clips. These findings suggest that the academic anatomy community may find value in the integration of social media into blended learning approaches in anatomy programs. This will ensure continued connection with the YouTube generation of students while also allowing for academic and ethical oversight regarding the use of online video clips whose provenance may not otherwise be known. Anat Sci Educ.

In the eye of the beholder: student perspectives on professional roles in practice

Context  Learning about professional roles in clinical settings is confounded by the gap between espoused theory and the professional practice of the workplace. Workplace learning is grounded in that which is afforded to learners and individuals’ engagement with those affordances. The meaning students make of the real‐world performance of professional roles and how this relates to formal professionalism frameworks remain unclear. Construal of experience is individual. Professional roles are enacted in the eye of the beholder. In their reflections, student subjectivities, intentionalities and engagement with workplace affordances are revealed. Our research question was: How do students’ perspectives of professional roles in practice, revealed through written reflections, relate to the formal professionalism curriculum?

Are the bigger hospitals better: DREEM on?

BackgroundClinical teaching in the undergraduate medical curriculum occurs at diverse sites. Analysis of the quality or effectiveness of such sites centres on student feedback. The Dundee Ready Education Environment Measure (DREEM) is a standardised instrument used internationally to analyse the educational environment.AimOur objective was to analyse and compare the student experience at a range of hospital clinical sites in the year 3 undergraduate medical programme in the School of Medicine in University College Cork using the DREEM.MethodsAll year 3 students were asked to complete the DREEM at the end of each hospital rotation.ResultsAll hospital sites performed well but smaller hospitals perform particularly well.ConclusionsIn the current climate, schools are compelled to make strategic decisions regarding the configuration of clinical teaching in a context where healthcare is also increasingly centralised. However, it is clear that smaller hospitals are preferred by students.

Convergence and translation: attitudes to inter-professional learning and teaching of creative problem-solving among medical and engineering students and staff.

BackgroundHealthcare worldwide needs translation of basic ideas from engineering into the clinic. Consequently, there is increasing demand for graduates equipped with the knowledge and skills to apply interdisciplinary medicine/engineering approaches to the development of novel solutions for healthcare. The literature provides little guidance regarding barriers to, and facilitators of, effective interdisciplinary learning for engineering and medical students in a team-based project context.MethodsA quantitative survey was distributed to engineering and medical students and staff in two universities, one in Ireland and one in Belgium, to chart knowledge and practice in interdisciplinary learning and teaching, and of the teaching of innovation.ResultsWe report important differences for staff and students between the disciplines regarding attitudes towards, and perceptions of, the relevance of interdisciplinary learning opportunities, and the role of creativity and innovation. There was agreement across groups concerning preferred learning, instructional styles, and module content. Medical students showed greater resistance to the use of structured creativity tools and interdisciplinary teams.ConclusionsThe results of this international survey will help to define the optimal learning conditions under which undergraduate engineering and medicine students can learn to consider the diverse factors which determine the success or failure of a healthcare engineering solution.

One week with the experts: A short course improves musculoskeletal undergraduate medical education

BACKGROUND Musculoskeletal problems constitute a considerable workload across all medical and surgical disciplines. There is a mismatch between the burden of musculoskeletal medicine seen by non-orthopaedists clinically and the amount of time afforded it in undergraduate training. Recent initiatives to address this include the United States Bone and Joint Decade and curricular innovations that demonstrate a benefit from improved instruction. Such curricular interventions are usually situated within a wider program reform and last a short time. Gaining institutional support and securing curricular time are challenging. This article shows the positive evaluation of a brief, intense course on musculoskeletal medicine. METHODS A one-week course was offered to 154 medical students. The study took place in Ireland, where the student body comprises a mix of graduate students and undergraduate students, who enter medical school directly from second-level education. This course comprised brief didactic talks, case-based small group work, and physical examination skills demonstration. Attitudes toward musculoskeletal medicine prior to the course were elicited. The course was evaluated using pre-course and post-course standardized cognitive tests. Long-term retention was evaluated by end-of-year extended matching questions and an objective standardized clinical examination station. The test results between undergraduate and graduate students and student rating of musculoskeletal medicine as important or less important were compared. RESULTS Complete data were available for 125 students (81%). Seventy-four percent of students rated musculoskeletal medicine to be of major or critical importance to their career. There was a significant difference (p < 0.001, r = 0.678) in the mean score of the standardized cognitive test between the pre-course test and the post-course test; the mean performance score (and standard deviation) was 48.2% ± 14.2% (range, 17% to 79%), with a pass rate of 3.3%, for the pre-course test and 75.3% ± 15.02% (range, 32% to 100%), with a pass rate of 61%, for the post-course test. At the end of the year, 69.9% of students passed the extended matching questions and 96.7% passed an objective standardized clinical examination station. Graduate students performed better on the post-course standardized cognitive test score (p < 0.001) and objective standardized clinical examination (p < 0.05). Students who rated musculoskeletal medicine as important did not perform better than those who rated it as less important (p = 0.334). CONCLUSIONS We report a favorable evaluation of a short, intense course on musculoskeletal medicine and suggest that the introduction of basic concepts of musculoskeletal medicine is feasible within established curricula.

A picture tells 1000 words: learning teamwork in primary care

Background:  Teamwork and patient centredness are frequently articulated concepts in medical education, but are not always explicit in the curriculum. In Ireland, recent government policy emphasises the importance of a primary care team approach to health care. We report on an appraisal of a newly introduced community‐based student attachment, which focused on teamwork.

Can less be more? Comparison of an 8-item placement quality measure with the 50-item Dundee Ready Educational Environment Measure (DREEM)

Clinical clerks learn more than they are taught and not all they learn can be measured. As a result, curriculum leaders evaluate clinical educational environments. The quantitative Dundee Ready Environment Measure (DREEM) is a de facto standard for that purpose. Its 50 items and 5 subscales were developed by consensus. Reasoning that an instrument would perform best if it were underpinned by a clearly conceptualized link between environment and learning as well as psychometric evidence, we developed the mixed methods Manchester Clinical Placement Index (MCPI), eliminated redundant items, and published validity evidence for its 8 item and 2 subscale structure. Here, we set out to compare MCPI with DREEM. 104 students on full-time clinical placements completed both measures three times during a single academic year. There was good agreement and at least as good discrimination between placements with the smaller MCPI. Total MCPI scores and the mean score of its 5-item learning environment subscale allowed ten raters to distinguish between the quality of educational environments. Twenty raters were needed for the 3-item MCPI training subscale and the DREEM scale and its subscales. MCPI compares favourably with DREEM in that one-sixth the number of items perform at least as well psychometrically, it provides formative free text data, and it is founded on the widely shared assumption that communities of practice make good learning environments.

Framework for feedback: the peer mini-clinical examination as a formative assessment tool

What problems were addressed? Our students wanted more feedback on their performance with patients in the clinical setting. We wanted to develop students’ ability to deliver and accept useful feedback. The mini-clinical examination (mini-CEX) is part of our assessment programme and is familiar to students. It is a workplace-based assessment in which student performance during a focused clinical encounter is evaluated. Immediate feedback on performance and the teaching opportunity provided have been identified as strengths of the process by students. We wondered if the mini-CEX format could serve as a useful framework for a process of reciprocal peer feedback on clinical placements. What was tried? We ran a pilot study to assess the utility and acceptability to students of the mini-CEX as a framework for feedback. Forty students, assigned in pairs, undertook two peer mini-CEX evaluations whilst on clinical attachments. In this way, each student acted as both learner and assessor. The National Health Service mini-CEX form, with minor modifications for the undergraduate setting, was used. The assessor assigned a mark and identified areas for improvement in his or her peer’s performance. All students were asked to complete a qualitative survey evaluating the process. Responses were coded and grouped into themes independently by the authors. The marks and written feedback on the mini-CEX forms completed by the student assessors were also examined. What lessons were learned? The mini-CEX provided a useful framework for formative peer feedback in the undergraduate clinical setting. Peer assessors were able to identify the areas in which students had performed well and those that required improvement, and noted these, along with actions for improvement, on the mini-CEX forms. However, no student scored less than 4 out of 5. This is unlikely to accurately reflect the range of student performance and suggests that students may have found quantitative evaluation more challenging. Qualitative feedback revealed that students liked benchmarking their own performances against those of their peers, receiving useful feedback, gaining experience for other examinations, the reallife context of the mini-CEX and the opportunity to gain a different perspective by examining a peer’s performance. Negative comments focused on the validity of the feedback in view of the lack of qualification of peers, logistical issues concerning the organisation of the assessment, and desire for more guidance regarding standards to apply in assessing peers. Although the mini-CEX was effective in scaffolding peer feedback, students appeared to lack confidence in the process. Although they recognised that the feedback was useful, only half of the students felt that it was appropriate to be assessed by a peer. Building a culture of peer-assisted learning might improve this. More guidance from faculty staff on standards and metrics underpinning judgements is needed and we have piloted a more structured mini-CEX form in response to this need. Our students are trained in giving and receiving feedback; however, their confidence in these skills did not transfer to a real-life setting. Graduated real-life application of feedback skills facilitated by faculty members may help to bridge this gap.

Possibility and agency in Figured Worlds: Becoming a ‘good doctor’

Figured Worlds is a socio‐cultural theory drawing on Vygotskian and Bakhtinian traditions, which has been applied in research into the development of identities of both learners and teachers in the wider education literature. It is now being adopted in medical education.