Johanna Schönrock-Adema

Effects of conventional and problem‐based learning on clinical and general competencies and career development

Objective  To test hypotheses regarding the longitudinal effects of problem‐based learning (PBL) and conventional learning relating to students’ appreciation of the curriculum, self‐assessment of general competencies, summative assessment of clinical competence and indicators of career development.

Necessary steps in factor analysis : Enhancing validation studies of educational instruments. The PHEEM applied to clerks as an example

Background: The validation of educational instruments, in particular the employment of factor analysis, can be improved in many instances. Aims: To demonstrate the superiority of a sophisticated method of factor analysis, implying an integration of recommendations described in the factor analysis literature, over often employed limited applications of factor analysis. We demonstrate the essential steps, focusing on the Postgraduate Hospital Educational Environment Measure (PHEEM). Method: The PHEEM was completed by 279 clerks. We performed Principal Component Analysis (PCA) with varimax rotation. A combination of three psychometric criteria was applied: scree plot, eigenvalues >1.5 and a minimum percentage of additionally explained variance of approximately 5%. Furthermore, four interpretability criteria were used. Confirmatory factor analysis was performed to verify the original scale structure. Results: Our method yielded three interpretable and practically useful dimensions: learning content and coaching, beneficial affective climate and external regulation. Additionally, combining several criteria reduced the risk of overfactoring and underfactoring. Furthermore, the resulting dimensions corresponded with three learning functions essential to high-quality learning, thus strengthening our findings. Confirmatory factor analysis disproved the original scale structure. Conclusions: Our sophisticated approach yielded several advantages over methods applied in previous validation studies. Therefore, we recommend this method in validation studies to achieve best practice.

Assessment of professional behaviour in undergraduate medical education: peer assessment enhances performance

Objectives  To examine whether peer assessment can enhance scores on professional behaviour, with the expectation that students who assess peers score more highly on professional behaviour than students who do not assess peers.

Development of a student rating scale to evaluate teachers’ competencies for facilitating reflective learning

Medical Education 2011: 45: 155–165

Transition to clinical training : influence of pre-clinical knowledge and skills, and consequences for clinical performance

Context  Many students experience a tough transition from pre‐clinical to clinical training and previous studies suggest that this may constrict students’ progress. However, clear empirical evidence of this is lacking. The aim of this study was to determine: whether the perceived difficulty of transition influences student performance during the first 2 weeks of clerkships; whether it influences students’ overall performance in their first clerkship, and the degree to which the difficulty of transition is influenced by students’ pre‐clinical knowledge and skills levels.

Key Elements in Assessing the Educational Environment: Where Is the Theory?.

The educational environment has been increasingly acknowledged as vital for high-quality medical education. As a result, several instruments have been developed to measure medical educational environment quality. However, there appears to be no consensus about which concepts should be measured. The absence of a theoretical framework may explain this lack of consensus. Therefore, we aimed to (1) find a comprehensive theoretical framework defining the essential concepts, and (2) test its applicability. An initial review of the medical educational environment literature indicated that such frameworks are lacking. Therefore, we chose an alternative approach to lead us to relevant frameworks from outside the medical educational field; that is, we applied a snowballing technique to find educational environment instruments used to build the contents of the medical ones and investigated their theoretical underpinnings (Study 1). We found two frameworks, one of which was described as incomplete and one of which defines three domains as the key elements of human environments (personal development/goal direction, relationships, and system maintenance and system change) and has been validated in different contexts. To test its applicability, we investigated whether the items of nine medical educational environment instruments could be mapped unto the framework (Study 2). Of 374 items, 94% could: 256 (68%) pertained to a single domain, 94 (25%) to more than one domain. In our context, these domains were found to concern goal orientation, relationships and organization/regulation. We conclude that this framework is applicable and comprehensive, and recommend using it as theoretical underpinning for medical educational environment measures.

Which characteristics of written feedback are perceived as stimulating students' reflective competence: an exploratory study

BackgroundTeacher feedback on student reflective writing is recommended to improve learners’ reflective competence. To be able to improve teacher feedback on reflective writing, it is essential to gain insight into which characteristics of written feedback stimulate students’ reflection processes. Therefore, we investigated (1) which characteristics can be distinguished in written feedback comments on reflective writing and (2) which of these characteristics are perceived to stimulate students’ reflection processes.MethodsWe investigated written feedback comments from forty-three teachers on their students’ reflective essays. In Study 1, twenty-three medical educators grouped the comments into distinct categories. We used Multiple Correspondence Analysis to determine dimensions in the set of comments. In Study 2, another group of twenty-one medical educators individually judged whether the comments stimulated reflection by rating them on a five-point scale. We used t-tests to investigate whether comments classified as stimulating and not stimulating reflection differed in their scores on the dimensions.ResultsOur results showed that characteristics of written feedback comments can be described in three dimensions: format of the feedback (phrased as statement versus question), focus of the feedback (related to the levels of students’ reflections) and tone of the feedback (positive versus negative). Furthermore, comments phrased as a question and in a positive tone were judged as stimulating reflection more than comments at the opposite side of those dimensions (t = (14.5) = 6.48; p = < .001 and t = (15) = −1.80; p < .10 respectively). The effect sizes were large for format of the feedback comment (r = .86) and medium for tone of the feedback comment (r = .42).ConclusionsThis study suggests that written feedback comments on students’ reflective essays should be formulated as a question, positive in tone and tailored to the individual student’s reflective level in order to stimulate students to reflect on a slightly higher level. Further research is needed to examine whether incorporating these characteristics into teacher training helps to improve the quality of written feedback comments on reflective writing.

One-year transitional programme increases knowledge to level sufficient for entry into the fourth year of the medical curriculum

Background: To cope with a lack of doctors and in anticipation of the Bachelor-Master structure for Medicine, several Dutch universities offer graduate entry programmes for students with degrees in areas related to Medicine. The graduate entry programme is a four-year programme: after a transition period of one year students enrol in the fourth year of the regular six-year training programme. Aim: The research questions in this study were (1) whether and when graduate entry students’ knowledge reached a level comparable to that of regular medical students and (2) whether there were differences in knowledge levels between graduate entry students with a university or HBO (college) degree. Methods: The progress test results of ninety graduate entry students who were enrolled in the transitional programme between 2002 and 2004 were compared to those of regular third-year students. Results: Initially, graduate entry students scored significantly lower on the progress tests, but differences disappeared within a year. No differences were found between graduate entry students with a university or HBO degree. Conclusions: The results of this study indicate that the increase in knowledge after a one-year transitional period is sufficient to enrol students with related degrees in the fourth year of the regular medical training programme.

‘What would my classmates say?’ An international study of the prediction-based method of course evaluation

Objectives  Traditional student feedback questionnaires are imperfect course evaluation tools, largely because they generate low response rates and are susceptible to response bias. Preliminary research suggests that prediction‐based methods of course evaluation ‐ in which students estimate their peers’ opinions rather than provide their own personal opinions ‐ require significantly fewer respondents to achieve comparable results and are less subject to biasing influences. This international study seeks further support for the validity of these findings by investigating: (i) the performance of the prediction‐based method, and (ii) its potential for bias.

Instruments for measuring quality of educational environments: Validation not required any longer?

Clerkships are generally viewed as a favourable learning environment for final year students. During this period, it is necessary to completely integrate final year students into ward procedures in order to improve their proficiency in independent patient management. Since on-ward supervision has been shown to be rare (Howley & Wilson 2004), we introduced an innovative model of a supplementary, structured, on-ward supervision for final year students, with experienced physicians who were not simultaneously the current ward physician serving as supervisors. Learning goals were defined based on a search of the literature (Schrauth et al. 2009) and comprised patient history taking and clinical examination, conducting ward rounds including chart documentation, the presentation of patient cases, and writing medical reports. A cohort of final year students (n1⁄4 16; 9 female; mean age 25.0 years) agreed to participate in the on-ward supervision programme during their clinical rotation. Final-year-student supervisors (n1⁄4 2; 1 female) were third-year internal medicine residents and received intensive training in teaching methodology as well as in providing qualified feedback. Evaluation of the programme included post-intervention ratings on a seven-point Likert-scale ranging from 6 (very good) to 1 (unsatisfactory) and a focus group analysis. Quantitative analysis revealed that final year students considered the programme to be extremely helpful (5.75; 0.44), to improve their clinical skills (4.90; 1.40), and to support independent working (4.80; 1.01). They expressed the wish for the programme to be expanded (5.62; 0.62). Final-year-student supervisors were seen as good teachers (5.81; 0.40) that were sufficiently qualified (5.87; 0.50) and were considered to represent an important attachment figure (5.80; 0.30). A supplementary focus-group analysis based on a sub-sample of participants (n1⁄4 4) affirmed that delegated medical tasks normally remain unsupervised, and that finalyear-students benefited enormously from the programme, mainly due to the close one-to-one monitoring by the supervisors. In conclusion, an on-ward supervision programme is a feasible and well-accepted tool in final year medical education which assures supervision and conveys skills in independent patient management. M. Eden N. Köhl-Hackert M. Krautter J. Jünger C. Nikendei Department of General Internal and Psychosomatic Medicine University of Heidelberg Medical Hospital Im Neuenheimer Feld 410 69120 Heidelberg Germany Tel: þ49-6221-56-3-8663 Fax: þ49-6221-56-5749 E-mail: christoph_nikendei@med.uni-heidelberg.de