D.H.J.M. Dolmans

The need for evidence in education

In this article a plea is made to use evidence in education. A remarkable difference in attitude is noted between university staff in their role as scientists in their discipline and in their role as teachers. Whereas evidence is the key to guide scientists in the development of their discipline, evidence on teaching and learning hardly affects their role as teachers. Teaching is, rather, dominated by intuition and tradition. However, particularly in education, intuitions and traditions are not always correct when they are submitted to empirical verification. It even often turns out that our intuitions are not justified or that assumed relations are far more complex. To illustrate the fallacy of our (implicit) intuitions and beliefs, a few of these assumptions are held against the available evidence. Two assumptions related to the learning of students and two assumptions related to the assessment of student achievement are discussed. The illustrations make clear that we do need to use evidence in educatio...In this article a plea is made to use evidence in education. A remarkable difference in attitude is noted between university staff in their role as scientists in their discipline and in their role as teachers. Whereas evidence is the key to guide scientists in the development of their discipline, evidence on teaching and learning hardly affects their role as teachers. Teaching is, rather, dominated by intuition and tradition. However, particularly in education, intuitions and traditions are not always correct when they are submitted to empirical verification. It even often turns out that our intuitions are not justified or that assumed relations are far more complex. To illustrate the fallacy of our (implicit) intuitions and beliefs, a few of these assumptions are held against the available evidence. Two assumptions related to the learning of students and two assumptions related to the assessment of student achievement are discussed. The illustrations make clear that we do need to use evidence in education, just as we do in any other professional area. Being a professional teacher requires more than being an expert in a content area; it also requires familiarity, use, and perhaps production of educational evidence and theory.

A rating scale for tutor evaluation in a problem-based curriculum: Validity and reliability

Summary: An instrument has been developed to assess tutor performance in problem‐based tutorial groups. This tutor evaluation questionnaire consists of 13 statements reflecting the tutors behaviour. The statements are based on a description of the tasks set for the tutor. This study reports results on the validity and reliability of the instrument. Confirmatory factor analysis showed that a three‐factor model fitted the data reasonably well. The three factors are: (1) guiding students through the learning process, (2) content knowledge input, and (3) commitment to the groups learning. Generalizability studies indicated that the rating scales provide reliable information with student responses of existing tutorial group sizes. It is concluded that the tutor evaluation questionnaire is a fairly valid and reliable instrument that can be used in staff development programmes.

Students’ perceptions of relationships between some educational variables in the out‐patient setting

Background Medical education uses the cognitive apprenticeship model of student learning extensively. Students rotate among different hospitals and out‐ patient clinics where they are exposed to a range of professionally relevant contexts. Here they learn to think and act in different domains under the supervision of experts. Previous research has shown that these learning situations involve little teaching. Students see a narrow range of patient problems and feedback is limited. The aim of this study is to investigate relationships among some educational variables in the out‐patient clinic.

The Relation Between Time Spent on Individual Study and Academic Achievement in a Problem-Based Curriculum

Logically a relation is expected between time spent on individual study and achievement. The purpose of this study is to examine whether a positive relation exists between the amount of time spent on individual study and academic achievement and whether this differs when using a test measuring short-term knowledge or one measuring long-term knowledge. Data were collected in a problem-based medical curriculum. The results suggest that time spent on individual study correlates poorly with scores on the test measuring short-term knowledge but also with those on the test measuring long-term knowledge. This study reaffirms the complexity of the relationship and it demonstrates the importance to search for qualitative factors about the way students learn.

Impact of individual study on tutorial group discussion

Background: This research focuses on the relation between individual study and group discussion. In a problem-based curriculum, it is expected that the way students prepare themselves during individual study (i.e., search and prepare the literature) will influence the quality of the reporting phase. Purpose: To investigate whether searching for different literature resources and preparing the literature (by making summaries to explain the literature) affects the quality of the reporting phase. Method: A 23-item questionnaire was developed, reflecting 2 factors of the search phase and 1 factor of the preparing phase of the individual study. Two factors (breadth and depth of the discussion) reflected the reporting phase. Participants were 1st-year students (N = 195, 90%) at the Medical School of the Maastricht University in the Netherlands during the academic year 1997-1998. Results: Regression analyses show that the search phase and the preparing phase explained 29% of the variance of the breadth and 38% o...

Providing physicians with feedback on how they supervise students during patient contacts

In this descriptive study an instrument is presented that has been developed to provide physicians with feedback about their strengths and weaknesses in facilitating student learning during patient contacts. The instrument is strongly theory based, i.e. it is based on current general theories of context-bound learning environments, and forms of facilitation promoting transfer of knowledge to actual professional practice. In addition, it has been developed in cooperation with physicians supervising students during patient contacts. The authors have shown how physicians can be provided with individualized feedback on their performance in supervising students during patient contacts.

Clinical supervisors' perceived needs for teaching communication skills in clinical practice

Background: Lack of faculty training is often cited as the main obstacle to post-graduate teaching in communication skills. Aims: To explore clinical supervisors’ needs and perceptions regarding their role as communication skills trainers. Methods: Four focus group discussions were conducted with clinical supervisors from two in-patient and one out-patient medical services from the Geneva University Hospitals. Focus groups were audio taped, transcribed verbatim and analyzed in a thematic way using Maxqda© software for qualitative data analysis. Results: Clinical supervisors said that they frequently addressed communication issues with residents but tended to intervene as rescuers, clinicians or coaches rather than as formal instructors. They felt their own training did not prepare them to teach communication skills. Other barriers to teach communication skills include lack of time, competing demands, lack of interest and experience on the part of residents, and lack of institutional priority given to communication issues. Respondents expressed a desire for experiential and reflective training in a work-based setting and emphasised the need for a non-judgmental learning atmosphere. Conclusions: Results suggest that organisational priorities, culture and climate strongly influence the degree to which clinical supervisors may feel comfortable to teach communication skills to residents. Attention must be given to these contextual factors in the development of an effective communication skills teaching program for clinical supervisors.

From quality assurance to total quality management: how can quality assurance result in continuous improvement in health professions education?

CONTEXT Quality assurance is a hot topic in many institutes for higher education. Quality assurance is a cyclic process, consisting of three steps: measuring, judging and improving. Many institutes in higher education collect data about the quality of their educational programme on a regular basis and with proper frequency, but these evaluative data are not always used to improve the quality of the programme. PURPOSE The aim of this article is to demonstrate which conditions need to be fulfilled to ensure that quality assurance is a cyclic process resulting in continuous improvement. CONCLUSIONS Quality assurance can only be successful, i.e. result in continuous improvement, if three conditions are met: the evaluation activities are carried out in (1). a systematic and (2). structural fashion and (3). are integrated in the organizations regular work patterns. Parts of the system for quality assurance at the Maastricht Medical School are presented to demonstrate how the three conditions can be realized in practice.

Can students differentiate between PBL tutors with different tutoring deficiencies

Many medical schools evaluate the performance of their tutors by using questionnaires. One of the aims of these evaluations is to provide tutors with diagnostic feedback on strong and weak aspects of their performance. Although everyone will agree that students are able to distinguish between poor and excellent tutors, one can question whether students are also able to differentiate between tutors with different tutoring deficiencies—tutors who perform badly on a specific key aspect of their performance. The aim of this study was to investigate to what degree students are able to differentiate between tutors with different tutoring deficiencies, how effective tutors are with different deficiencies and what kind of tips students give for improvement of a tutors behaviour. Based on students’ ratings on a tutor evaluation questionnaire, tutors were ranked in groups with different deficiencies and the average overall tutor performance score was computed for each group with a particular deficiency. In addition, students’ tips for improvement given in the open-ended question at the end of the questionnaire were analysed. The results demonstrated that on average one out of five tutors showed a deficiency on only one key aspect. Tutors who did not stimulate students towards active learning were perceived as least effective. Furthermore, students’ tips for improvement could be categorized into four groups: tutors who do not evaluate adequately, tutors who are too directive, tutors who are too passive and tutors who lack content knowledge. The results of this study demonstrate that students are not only able to distinguish between poor and excellent tutors, but are also able to diagnose tutors with different tutoring deficiencies and are able to provide tutors with specific feedback to improve their performance.

Exploring the validity and reliability of a questionnaire for evaluating veterinary clinical teachers' supervisory skills during clinical rotations

Background: Feedback to aid teachers in improving their teaching requires validated evaluation instruments. When implementing an evaluation instrument in a different context, it is important to collect validity evidence from multiple sources. Aim: We examined the validity and reliability of the Maastricht Clinical Teaching Questionnaire (MCTQ) as an instrument to evaluate individual clinical teachers during short clinical rotations in veterinary education. Methods: We examined four sources of validity evidence: (1) Content was examined based on theory of effective learning. (2) Response process was explored in a pilot study. (3) Internal structure was assessed by confirmatory factor analysis using 1086 student evaluations and reliability was examined utilizing generalizability analysis. (4) Relations with other relevant variables were examined by comparing factor scores with other outcomes. Results: Content validity was supported by theory underlying the cognitive apprenticeship model on which the instrument is based. The pilot study resulted in an additional question about supervision time. A five-factor model showed a good fit with the data. Acceptable reliability was achievable with 10–12 questionnaires per teacher. Correlations between the factors and overall teacher judgement were strong. Conclusions: The MCTQ appears to be a valid and reliable instrument to evaluate clinical teachers’ performance during short rotations.