Sue Roff

The Dundee Ready Educational Environment Measure (DREEM)—a generic instrument for measuring students’ perceptions of undergraduate health professions curricula

Students’ perceptions of their educational environment have been studied at all levels of the education system, from primary through post-secondary education. Recent imperatives towards enhanced quality assessment monitoring at a time when health professions education is increasingly committed to student-centred teaching and learning have stimulated a revival of interest in this field. This paper reports a body of research in health professions institutions around the world based on the Dundee Ready Educational Environment Measure (DREEM), a reliable, validated inventory that claims to be generic to undergraduate health professions education and non-culturally specific.

Development and validation of an instrument to measure the postgraduate clinical learning and teaching educational environment for hospital-based junior doctors in the UK

This paper describes the development and validation of a 40-item inventory, the Postgraduate Hospital Educational Environment Measure (PHEEM), by researchers in Scotland and the West Midlands using a combination of grounded theory and Delphi process. The instrument has since returned an alpha reliability >0.91 in two administrations in England and may be a useful instrument in the quality assurance process for postgraduate medical education and training.

Students' perceptions of educational environment: a comparison of academic achievers and under-achievers at kasturba medical college, India.

CONTEXT No country, least of all poorly resourced countries such as India, can afford to lose too many medical students in their undergraduate years. It would be useful to have an instrument to identify those students who are vulnerable to academic failure at this level of training and to identify the features of the educational environment that they perceive differently from students who are succeeding academically in order to design intervention strategies. Gender differences in perceptions of the educational environment might well emerge in particular academic or cultural contexts, with particular curricula. The present study was motivated by this concern and focused on comparisons between academic achievers and under-achievers and male and female students of Kasturba Medical College, India. OBJECTIVES (1) To compare the perceptions of the educational environment of academic achievers and under-achievers and to identify problem areas that should be remediated. (2) To identify whether there is any gender difference in the perceptions. METHODS The Dundee Ready Educational Environment Measure (DREEM) was administered to 508 medical students studying in the clinical years at the Kasturba Medical College in India. Item as well as scale scores were compared between academic achievers and under-achievers. FINDINGS Overall sample of the present study rated educational environment in this institution as average. The overall mean DREEM score was significantly higher for academic achievers. Compared to under-achievers, academic achievers scored significantly higher on perceptions regarding teachers, academic atmosphere and social self-perceptions. In addition to this, the overall rating (total DREEM score) of female students was significantly less compared to males in the academically vulnerable group. CONCLUSIONS Perceptions of poor performers are significantly different from those of better performers in the same institution. More importance should be given to the perceptions of students to improve the educational environment, as perceptions are associated positively with learning outcome, learning approach and attitude toward studying. Use of the DREEM as a monitoring tool might permit timely interventions to remediate problematic educational environments.

WHAT IS EDUCATIONAL CLIMATE

When students enter a new learning institution, they become aware of the curriculum they will follow through various explicit means such as the course syllabi, the classes they attend, the examinations they prepare for. The teachers, of course, should be well aware of the curriculum they are expected to teach through the course documentation and through faculty meetings and discussions. In some cases the curriculum is developed by the institution itself; in others it is mandated by external authorities, or even is taught as part of a national curriculum. In addition to the documented curriculum, students and teachers both become aware of the ‘educational environment’ or ‘climate’ of the institution. Is the teaching and learning environment very competitive? Is it authoritarian? Is the atmosphere in classes and field placements relaxed or is it in various ways stressful, perhaps even intimidating? These are all key questions in determining the nature of the learning experience. What some researchers refer to as the ‘press’ in the academic and social environment of the institution can vary from course to course within the curriculum, even from class to class. These expectations can be perceived as either formal or informal components of the educational experience—and they can vary from individual to individual. They can be motivating—or demotivating. Individual students may respond differently to these subtle elements in their learning experience. If we can identify the elements operating in the educational environment or climate of a given institution or course, and evaluate how they are perceived by students and teachers, we have the basis for modifying them to enhance the learning experience in relation to our teaching goals. Several research groups over the years have attempted to identify and quantify the presence and impact of rather intangible aspects of a learning environment: the climate, or atmosphere, or ethos, tone or ambience, the culture or personality of the institution. Just as there is a ‘sick building syndrome’ there could be a ‘sick learning environment’. Jack Genn surveys these and many other important aspects in his paper in this issue. Perhaps one reason why the study of learning environments in health professions such as medicine has been of increasing interest in recent years is the growing diversity of the student population. We can no longer assume that we are teaching young people just out of secondary education. The move to mature entry in several countries may mean that we have to teach differently. Kick et al. (2000) have begun to study the perceived learning environment needs of mature students at the University of Colorado Health Sciences Center. There may also be gender differences in preferred learning environments that will become increasingly apparent now that many institutions have a more equitable gender balance in their student populations. Carol Gilligan and Susan Pollak (1988) commented more than a decade ago that “The increasing number of women entering the medical profession prompts a rethinking of medical education. Like the canaries taken into mines to reveal the presence of unseen dangers, women medical students in their heightened sensitivity to detachment and isolation often reveal the places in medical training and practice where human connection has become dangerously thin”. Perhaps the study of learning environments can help us to begin to explore those places. And although we have seen many studies of the learning needs of socially disadvantaged minorities who seek access to medical schools and other health professions institutions, we have not yet really looked at their needs from the educational environment once they succeed in getting admitted. Another variable that is apparent from the work that has been done recently, e.g. Pololi and Price’s (2000) development of an instrument to measure the learning environment as perceived by undergraduates in an American medical school and similar work done by the present writers with the help of postgraduate research students (Roff et al., 1997) is the level or stage that the learner is at in the curriculum. Firstand second-year students have different perceptions of the desirable learning environment than do those in the clinical years. Undergraduate students’ perceived needs are different from those who are newly graduated and in their first year of practice. Even within the first 2 or 3 years of postgraduate training, we begin to see that the mature learner wants progressively different types of teaching and learning—and environments. Data that we have begun to collect from Scottish postgraduates suggest that the new practitioner wants progressive degrees of autonomy in his or her learning environment, but at the same time wants to be able to reach out to more competent and experienced supervisors when he/she needs to. We have also looked at the question of whether it is possible to develop a universal ‘culture free’ inventory to measure the quality of the educational environment for the health professions. Our Dundee Ready Education Environment Measure (DREEM) (Roff et al., 1997) is being used in a dozen countries in medical schools, nursing colleges

Perceptions of the educational environment of the medical school in king Abdul Aziz university,Saudi Arabia

The present paper reports data from the undergraduate medical school at King Abdul Aziz University in Saudi Arabia using the 50-item Dundee Ready Education Environment Measure (DREEM), which has been validated and found to be reliable in a range of cultural contexts. The mean total score was 102/200. The mean total score for Perception of Learning was 22/48; for Perceptions of Teaching 24/44; for Academic Self-perceptions 17/32; for Perceptions of Atmosphere 23/48 and for Social Self Perceptions 15/28. The mean score of 0.9 for Item 3—There is a good support system for students who get stressed—was the lowest and is indicative of the pressures felt by the students. These scores give a clear indication of the priorities for reform of the curriculum. These data can also serve as a baseline for a longitudinal quality assessment of students’ perceptions of the changes planned for the medical school at King Abdul Aziz University, Saudi Arabia. Further studies are needed to analyse educational environment down to the year levels and to detect any gender differences.

Assessment of the undergraduate medical education environment in a large UK medical school

Objective To assess the undergraduate educational environment in a large UK medical school. Method Prospective study using the already validated Dundee Ready Education Environment (DREEM) questionnaire (Appendix 1). Setting A large UK medical school. Participants All medical students enrolled in the academic year 2002/2003. Main outcome measures Perception of the overall educational environment. In addition the impact of year of study and gender were examined. Results The DREEM questionnaire consisting of 50 items scored on a 0-4 Likert scale was completed by 749 medical students (55 per cent) in the academic year 2002/2003. The mean total score was 124 (out of a maximum of 200) indicating relative satisfaction with the perceived environment. There were no individual areas of excellence (that is no item scored >3.5). Some items scored consistently badly indicating cause for concern, for example lack of a support system for stressed students, school time-tabling, feedback from teachers and memorisation of facts. Clinical students perceived the environment to be significantly more positive than preclinical students (127 v 119, p<0.05). Female students perceived the environment to be significantly more positive compared to male students (126 v 123, p<0.05). Conclusions This tool identified areas of concern within a large UK medical school. Further use of the DREEM as a monitoring tool would be useful to re-evaluate the environment following appropriate intervention.

Education environment: a bibliography

1. AL-HAZIMI, A., AL-HYIANI, A. & ROFF, S. (2004) Perceptions of the educational environment at the medical school in King Abdul Aziz University, Saudi Arabia, Medical Teacher, 26, pp. 570–573. 2. AL-HAZIMI, A., ZAINI, R., AL-HYIANI, A., HASSAN, N., GUNAID, A., PONNAMPERUMA, G., KARUNATHILAKE, I., ROFF, S., McAleer, S. & DAVIS, M.D. (2004) Educational environment in traditional and innovative medical schools: a study in four undergraduate medical schools, Education for Health, 17, pp. 192–203. 3. AL-QAHTANI, M.F. (1999) Approaches to study and learning environment in medical schools with special reference to the Gulf countries, thesis presented for the PhD, Faculty of Medicine, Dentistry and Nursing, University of Dundee, Scotland. 4. AL-ZIDGALI, L. (1999) Students’ approaches to studying at the Institute of Health Sciences, Sultanate of Oman, Masters of Medical Education dissertation, University of Dundee, Scotland. 5. BASSAW, B., ROFF, S., MCALEER, S., ROOPNARINESINGH, S., DE LISLE, J., TEELUCKSINGH, S. & GOPAUL, S. (2003) Students’ perspectives of the educational environment, Faculty of Medical Sciences, Trinidad, Medical Teacher, 25, pp. 522–526. 6. BIDDLE, W.B., SMITH, D.U. & TREMONTI, L. (1985) Congruence between curriculum goals and students’ perceptions of learning environment, Journal of Medical Education, 60, pp. 627–634. 7. DUNNE, F., RAMDIN, L., POPOVIC, C., MERRICKS, B., POLLARD, D., MARTIN, U., HEATH, C., BRAMHALL, S., MCALEER, S. & ROFF, S. (2003) Assessment of the educational environment at Birmingham Medical School using the DREEM questionnaire. Oral presentation, ASME Conference Edinburgh, September 2003. 8. FELETTI, G.I. & CLARKE, R.M. (1981) Review of psychometric features of the medical school learning environment survey, Medical Education, 15, pp. 92–96. 9. GENERAL MEDICAL COUNCIL (1993) Tomorrow’s Doctors. Recommendations on Undergraduate Medical Education (London, GMC). 10. GENN, J. & HARDEN, R.M. (1986) What is medical education here really like? Suggestions for action research studies of climates of medical education environments, Medical Teacher, 8, pp. 111–124. 11. GENN, J.M. (2001) AMEE Medical Education Guide No.23 (Part 1), Curriculum, environment, climate, quality and change in medical education – a unifying perspective, Medical Teacher, 23, pp. 337–344. 12. HARDEN, R.M. (1986a) Ten questions to ask when planning a course or curriculum. ASME Medical Education Booklet No 2, Medical Education, 20, pp. 356–365. 13. HARDEN, R.M. (1986b) Approaches to research in medical education. ASME Medical Education Research Booklet No 2, Medical Education, 20, pp. 522–531. 14. HARDEN, R.M., SOWDEN, S. & DUNN, W. (1984) Some educational strategies in curriculum development: the SPICES model. ASME Medical Education Booklet No 18, Medical Education, 18, pp. 284–297. 15. HARDEN, R.M. (2001) The learning environment and the curriculum, Medical Teacher, 23, pp. 335–336. 16. HARTH, S.C., BAVANANDAN, S., THOMAS, K.E., LAI, M.Y. & THONG, Y.H. (1992) The quality of student–tutor interactions in the clinical learning environment, Medical Education, 26, pp. 321–326. 17. HUTCHINS, E.B. (1961) The 1960 medical school graduate: his perception of his faculty, peers and environment, Journal of Medical Education, 36, pp. 322–329. 18. HUTCHINSON, L. (2003) The ABC of learning and teaching: educational environment, British Medical Journal, 326, pp. 810–812. 19. JIFFRY, M.T.M., MCALEER, S., FERNANDO, S. & MARASINGHE, R.B. (2005) Using the DREEM questionnaire to gather baseline information on an evolving medical school in Sri Lanka, Medical Teacher, 27, pp. 348–352. 20. KOVATZ, S., NOTZER, N., BEILBERG, I. & SHENKMAN, L. (2004) Cultural perception of harassment in two groups of medical students: American and Israeli, Medical Teacher, 26, 349–352. 21. LEMPP, H. & SEALE, C. (2004) The hidden curriculum in undergraduate medical education: qualitative study of medical students’ perceptions of teaching, British Medical Journal, 329, pp. 770–773. 22. LEVY, M., MORSE, P.K., LIEBELT, R.A., DALLMAN, J.J., MCDONALD, T.H. (1973) Use of the learning environment questionnaire to assess curriculum change, Journal of Medical Education, 48, pp. 840–844.

Is PHEEM a multi-dimensional instrument? An international perspective

Aim: To look at the characteristics of Postgraduate Hospital Educational Environment Measure (PHEEM) using data from the UK, Brazil, Chile and the Netherlands, and to examine the reliability and characteristics of PHEEM, especially how the three PHEEM subscales fitted with factors derived statistically from the data sets. Methods: Statistical analysis of PHEEM scores from 1563 sets of data, using reliability analysis, exploratory factor analysis and correlations of factors derived with the three defined PHEEM subscales. Results: PHEEM was very reliable with an overall Cronbachs alpha of 0.928. Three factors were derived by exploratory factor analysis. Factor One correlated most strongly with the teaching subscale (R = 0.802), Factor Two correlated most strongly with the role autonomy subscale (R = 0.623) and Factor Three correlated most strongly with the social support subscale (R = 0.538). Conclusions: PHEEM is a multi-dimensional instrument. Overall, it is very reliable. There is a good fit of the three defined subscales, derived by qualitative methods, with the three principal factors derived from the data by exploratory factor analysis.

Reconsidering the “decline” of medical student empathy as reported in studies using the Jefferson Scale of Physician Empathy-Student version (JSPE-S)

Abstract Introduction: The suggestion that empathy “declines” or “erodes” as students progress through medical school has largely rested on observations reported from Jefferson Medical College in the United States using the Jefferson Scale of Physician Empathy (JSPE) developed by Hojat and colleagues. Now that the student version of JSPE has been administered to medical students in more than a dozen countries, it is timely to consider whether or not the Jefferson “case study” and the conclusions drawn from it are generalisable. Methods: A literature research was conducted on MEDLINE in mid-2014 to identify studies reporting administrations of the Student version of JPSE (JSPE-S) to cohorts of medical students and the means for studies and their sub-parts conducted in Japan, South Korea, China, Kuwait, India, Iran, UK, USA, Australia, Brazil, Colombia, the Dominican Republic and Portugal. Results: The means of these studies from a dozen countries outside the USA consistently cluster round 75% out of the possible maximum of 140 unlike the early Jefferson studies (although the later Jefferson means are also <120). Conclusions: These observations may support Costa et al.’s contention that “a latent growth model suggests that empathy of medical students does not decline over time” (p. 509) – or at least not significantly. But in order to understand the maturation process of medical students and trainees we need to develop more sophisticated, integrated models that combine culturally-sensitive concepts of emotional intelligence and moral reasoning with far more refined understandings of the nature of empathy required for the safe practice of patient-centred medicine.

New resources for measuring educational environment

The educational environment in which postgraduate doctors are taught and trained in the UK has increasingly come under scrutiny with the acceptance of learner-centred principles of adult teaching and learning in medical education (Rotem et al., 1996; Hutchinson, 2003; Spencer, 2003). The new UK Postgraduate Medical Education and Training Board (PMETB)—on which the writer serves—is including trainees’ perceptions of their training experience as a part of the quality assurance system for accrediting programmes. While some degree of competitiveness and both intellectual and interpersonal challenges are usually productive for the trainee (Lempp et al., 2004), there can be environments which are dysfunctional for learning and practising medicine. UK deaneries are increasingly responsive to problems presented by bullying of junior staff both as trainees and service providers. Hicks (2003), for instance, recently reported that a third of the 2400 trainee doctors at all grades in the deanery for Kent, Sussex and Surrey claimed that they had been subjected to bullying in the previous year, 7% reporting persistent bullying. Similar findings were reported by Paice et al. (2004) among London trainees and more widely throughout the UK by Quine (2002; 2003). McManus & Sproston (2000) have investigated the existence of ‘glass ceilings’ for women in UK medicine and gender discrimination. Racial discrimination (Beagan, 2003; Singh, 2003; Goldacre et al., 2004) is known to exist in many, if not most, health care systems. However, as well as the negative features, investigation (Paice et al., 2003) of the educational environment in which the young doctor is learning and practising frequently points to the importance of good role models. While there is extensive literature on the qualities that are perceived by trainees to make a good clinical teacher, there is also evidence of the presence of poor role models among clinical teaching faculty (Skeff & Mutha, 1998; Wright et al. 1998). Hicks et al. (2001), for instance, reported that nearly half (47%) of the 108 final year clinical students at a Toronto medical school whom they interviewed reported witnessing a clinical teacher acting unethically. There has been a growing body of research on the importance of undergraduates’ perceptions of their teaching and training environments, as indicated by the accompanying bibliography. One of the inventories available is the Dundee Ready Education Environment Measure (DREEM) (Roff et al., 1997; 2001; Primparyon et al., 2000) and some of the reports of the range of applications to which it has been put in more than a dozen countries around the world are published in this issue (de Oliveira Filho & Schonhorst, 2005, de Oliveira Filho et al., 2005; Jiffry et al., 2005; Shoham & Shenkman, 2005; Till, 2005; Zaini, 2005). Roff et al. (2005) and Skinner (2005) report the development of the Postgraduate Hospital Educational Environment Measure which is now being used in training sites throughout the UK and has been piloted in The Netherlands. Holt (2002) used the same methodology to adapt the PHEEM for the measurement of the anaesthetic theatre teaching environment for English trainees. The development of the Anaesthetic Theatre Educational Environment Measure (ATEEM) is described in Holt & Roff (2004). Cassar (2004) used the same methodology to develop and validate the Surgical Theatre Educational Environment Measure (STEEM) and Mulrooney (2005) reports the development of similar inventories for general practice training in Ireland in this issue. With little adaptation, these inventories can probably be used in other countries since the items included mirror research findings on the perceived attributes of good clinical teachers in the USA (Irby, 1978; 1994; Irby & Rakestraw, 1981; Woverton & Bosworth, 1985; Fallon et al., 1987; Irby et al., 1987; 1991; Bing-you & Greenberg, 1990; Gjerde & Coble, 1994; Ramsbottom-Lucier et al., 1994; Blount & Jolissaint, 1996; Schum & Yindra, 1996; Jolissaint & Blount, 1997; Xu et al., 1998; Litzelman et al., 1999; Copeland & Hewson, 2000; Keitz et al., 2003); in Canada (Patel & Dauphinee, 1985; Cleave-Hogg & Benedict, 1997; Shysh & Eagle, 1997; Rotenberg et al., 2000) and in Australia (Harth et al., 1992; Price & Mitchell, 1993; Rotem et al., 1995). Research is currently being conducted to test their transferability. It is hoped that they will be useful tools in the quality assessment processes of junior doctor training, since it is now recognized as Hoff et al. (2004) have expressed it, that ‘the establishment of a supportive, learning-oriented culture is of utmost significance in creating competent physicians’. The newly constituted Postgraduate Medical Education and Training Board in the UK convened a national workshop on developing standards for educational environment as part of the quality assurance process for future approval of training programmes in April 2005.