Marianne Giesler

Development of a questionnaire to assess medical competencies: Reliability and validity of the Questionnaire.

Introduction: While preparing a graduate survey for medical education in 2008 we realized that no instrument existed that would be suitable to evaluate whether the learning outcomes outlined in the Medical Licensure Act (ÄAppO) would be met. Therefore we developed the Freiburg Questionnaire to Assess Competencies in Medicine (Freiburger Fragebogen zur Erfassung von Kompetenzen in der Medizin, FKM)1 which has been revised and extended several times since then. Currently the FKM includes 45 items which are assigned to nine domains that correspond to the CanMEDS roles: medical expertise, communication, team-work, health and prevention, management, professionalism, learning, scholarship, and personal competencies. Methods: In order to test the reliability and validity of the questionnaire we have repeatedly surveyed medical students and residents since May 2008. In this article we report on the results of a cross-sectional study with 698 medical students from the preclinical and clinical years. In addition, we report the results of a survey of 514 residents who were up to two years into their residency. Results and conclusions: In summary, results show that the scales of the FKM are reliable (Cronbach’s α between .68 and .97). Significant differences in means between selected groups of students support the measure’s construct validity. Furthermore, there is evidence that the FKM might be used as a screening tool e.g. in graduate surveys to identify weaknesses in the medical education curriculum.

Importance and benefits of the doctoral thesis for medical graduates

Introduction: The majority of medical graduates in Germany complete a doctorate, even though a doctoral degree is not necessary for the practice of medicine. So far, little is known about doctoral candidates’ view on the individual benefit a doctoral thesis has for them. Consequently, this is the subject of the present investigation. Method: Data from surveys with graduates of the five medical faculties of Baden-Württemberg from the graduation years 2007/2008 (N=514) and 2010/2011 (N=598) were analysed. Results: One and a half years after graduating 53% of those interviewed had completed their doctorate. When asked about their motivation for writing a doctoral thesis, participants answered most frequently “a doctorate is usual” (85%) and “improvement of job opportunities” (75%), 36% said that an academic career has been their primary motive. Less than 10% responded that they used their doctoral thesis as a means to apply for a job. The proportion of graduates working in health care is equally large among those who have completed a thesis and those who have not. Graduates who pursued a thesis due to scientific interest are also currently more interested in an academic career and recognise more opportunities for research. An implicit benefit of a medical thesis emerged with regard to the self-assessment of scientific competences as those who completed a doctorate rated their scientific competencies higher than those who have not. Discussion: Although for the majority of physicians research interest is not the primary motivation for completing a doctorate, they might nevertheless achieve some academic competencies. For graduates pursuing an academic career the benefit of completing a medical thesis is more obvious.

Novice medical students: individual patterns in the use of learning strategies and how they change during the first academic year.

Background: Adequate use of different learning strategies is one of the most important prerequisites of academic success. The actual use of learning strategies is the result of an interaction between individual and situational variables. Against this background we conducted a longitudinal study with first year medical students to investigate whether individuals show different patterns in their use of learning strategies and whether these patterns change during the first academic year. Methods: Medical students (N=175, 58% female) were surveyed three times in their first academic year regarding their use of learning strategies. A hierarchical cluster analysis (Ward) was conducted in order to identify groups of students with different patterns of learning strategies. Results: We identified four different patterns in approaches to learning among novice medical students (“easy-going”, “flexible”, “problematic” and “hardworking” learners). Compared to their peers, the problematic learners had the worst final school grades. In addition changes in the use of learning strategies were identified, most of them occurred during the first term. Conclusion: Students start their academic studies with different patterns of learning strategies; the characteristics of these patterns change during the first academic year. Further research is necessary to better understand how individual and situational variables determine students’ learning.

Benchmarking for research-related competencies – a curricular mapping approach at medical faculties in Germany

Abstract Objectives: Internationally, scientific and research-related competencies need to be sufficiently targeted as core outcomes in many undergraduate medical curricula. Since 2015, standards have been recommended for Germany in the National Competency-based Learning Objective Catalogue in Medicine (NKLM). The aim of this study is to develop a multi-center mapping approach for curricular benchmarking against national standards and against other medical faculties. Method: A total of 277 faculty members from four German medical faculties have mapped the local curriculum against the scientific and research-related NKLM objectives, using consented procedures, metrics, and tools. The amount of mapping citations of each objective is used as indicator for its weighting in the local curriculum. Achieved competency levels after five-year education are compared. Results: All four programs fulfill the NKLM standards, with each emphasizing different sub-competencies explicitly in writing (Scholar: 17–41% of all courses; Medical Scientific Skills: 14–37% of all courses). Faculties show major or full agreement in objective weighting: Scholar 44%, scientific skills 79%. The given NKLM competency level is met or even outperformed in 78–100% of the courses. Conclusions: The multi-center mapping approach provides an informative dataset allowing curricular diagnosis by external benchmarking and guidance for optimization of local curricula.

Confirmatory factor analysis of the German Readiness for Interprofessional Learning Scale (RIPLS-D)

ABSTRACT Over the past five years, the development of interprofessional education programmes has been gaining momentum in Germany fostering the need to evaluate these with appropriate instruments. Instead of developing a new instrument for evaluation purposes, the Readiness for Interprofessional Learning Scale (RIPLS) was chosen, as it is a widespread instrument that has been used in a variety of different educational settings and countries. The German version of the RIPLS was administered in two sites to health professional students in Heidelberg and Freiburg, Germany. Cronbach’s alpha was used to examine internal consistency. Confirmatory factor analysis (CFA) was performed for confirmation of the underlying factor structure of the RIPLS-D. In total, 531 questionnaires were analysed. The instrument showed overall reliability (0.81) and low reliability (< 0.7) in the subscales. The underlying factor structure could not be confirmed. These results contribute further evidence on deficits with the RIPLS. Despite known issues, the RIPLS continues to be translated and applied. This paper highlights the problematic issues in the RIPLS-D and does not recommend its use.

Impact of the Medical Faculty on Study Success in Freiburg: Results from Graduate Surveys.

Aim: Using the data from graduate surveys, this study aims to analyze which factors related to teaching and learning at the Freiburg Faculty of Medicine can influence study success. Background: Study success and the factors influencing it have long been the subject of investigation, with study success being measured in terms of easily quantifiable indicators (final grades, student satisfaction, etc.). In recent years, it has also frequently been assessed in terms of graduate competency levels. Graduate surveys are considered suitable instruments for measuring these dimensions of study success. Method: Data from three Freiburg graduate surveys conducted one and a half years after graduation were drawn upon for the analysis. Study success was operationalized using four indicators: results on the written section of the M2 exam, self-assessment of medical expertise and scientific expertise, and student satisfaction. Using multiple regression analyses, the predictive power was calculated for selected variables, also measured by the graduate surveys, for the different study success indicators. Results: It was possible to identify models that contribute slightly or moderately to the prediction of study success. The score earned on the university entrance qualification demonstrated itself to be the strongest predictor for forecasting the M2 written exam: R2 is between 0.08 and 0.22 for the three surveys. Different variables specific to degree program structure and teaching are helpful for predicting medical expertise (R2=0.04-0.32) and student satisfaction (R2=0.12-0.35). The two variables, structure and curricular sequencing of the degree program and combination of theory and practice, show themselves to be significant, sample-invariant predictors (β-weightStructure=0.21-0.58, β-weightCombination=0.27-0.56). For scientific expertise, no sample-independent predictors could be determined. Conclusion: Factors describing teaching hardly provide any assistance when predicting the written M2 exam score, which makes sense to the extent that teaching goes far beyond the heavily knowledge-based content of the written M2 exam. The lack of predictability for scientific expertise is most likely explained in that these have been only rarely included in the curriculum and often inexplicitly so. The variable combination of theory and practice appears to be significant for imparting medical expertise and the development of student satisfaction. The extent to which these relationships are practically relevant needs to be explored in further studies. A specific limitation is that the measurement of expertise and skill is based solely on self-assessments.

Audience-response systems for evaluation of pediatric lectures--comparison with a classic end-of-term online-based evaluation.

Aim: Course evaluations are often conducted and analyzed well after the course has taken place. By using a digital audience response system (ARS), it is possible to collect, view and discuss feedback during or directly following a course or lecture session. This paper analyzes a student evaluation of a lecture course with ARS to determine if significant differences exist between the results of the ARS lecture evaluation and those of the online evaluation at the end of the semester. In terms of the overall evaluation, consideration is given to the level of students’ prior knowledge, the presentation of the lecture material by the lecturers and the relevance of the lecture topic for students. Method: During the 2011-12 winter semester, the lecture on Pediatrics at the Freiburg Center for Pediatric and Adolescent Medicine (Zentrum für Kinder- und Jugendmedizin (ZKJ) Freiburg) was evaluated using ARS. Thirty-four lectures were evaluated by an average of 22 (range 8-44) students, who responded to four questions each time an evaluation took place. Results: On a 6-point Likert scale (1=very good to 6=deficient), the students rated their level of preparedness with a mean of 3.18, the presentation of the lecture with 2.44, and the relevance of the lecture topic with 2.19. The overall evaluation of the lecture course by means of ARS resulted in 2.31. The online evaluation conducted at the end of the semester yielded a score of 2.45. Highly significant correlations were seen between the results of the ARS for the overall evaluation, assessment of prior knowledge, lecture presentation, and the estimated relevance of the lecture topic. Conclusion: The use of ARS is suitable for immediate evaluation of lectures, in particular regarding timely feedback for the individual lecturerlecturers. In comparison with an end-of-term evaluation, ARS yielded a better assessment.