Ara Tekian

Assessment for selection for the health care professions and specialty training : consensus statement and recommendations from the Ottawa 2010 Conference

Assessment for selection in medicine and the health professions should follow the same quality assurance processes as in-course assessment. The literature on selection is limited and is not strongly theoretical or conceptual. For written testing, there is evidence of the predictive validity of Medical College Admission Test (MCAT) for medical school and licensing examination performance. There is also evidence for the predictive validity of grade point average, particularly in combination with MCAT for graduate entry but little evidence about the predictive validity of school leaver scores. Interviews have not been shown to be robust selection measures. Studies of multiple mini-interviews have indicated good predictive validity and reliability. Of other measures used in selection, only the growing interest in personality testing appears to warrant future work. Widening access to medical and health professional programmes is an increasing priority and relates to the social accountability mandate of medical and health professional schools. While traditional selection measures do discriminate against various population groups, there is little evidence on the effect of non-traditional measures in widening access. Preparation and outreach programmes show most promise. In summary, the areas of consensus for assessment for selection are small in number. Recommendations for future action focus on the adoption of principles of good assessment and curriculum alignment, use of multi-method programmatic approaches, development of interdisciplinary frameworks and utilisation of sophisticated measurement models. The social accountability mandate of medical and health professional schools demands that social inclusion, workforce issues and widening of access are embedded in the principles of good assessment for selection.

Assessment in Problem-Based Learning Medical Schools: A Literature Review

Background: Despite widespread use of problem-based learning (PBL) in medical schools, no review currently exists on its assessment. Given the importance of assessment for any curriculum, a critical review of the literature was conducted to explore whether the assessment methods match the philosophical tenets of PBL. Summary: Articles from MEDLINE and other databases on the assessment of PBL were reviewed. The following areas require special attention by PBL medical schools: enhancement of formal continuous formative assessment; use of the context of a working problem to assess knowledge and problem-solving skills; prevention of negative steering effects by a judicious choice of assessment content, instruments, and timing; and implementation of a longitudinal and centralized student profile. Conclusion: Despite the existence of general practical recommendations on assessment in PBL settings, this review reveals a lack of uniformity and consensus on the practical application of general principles. Topics f...

Preparing health professions education leaders worldwide: A description of masters-level programs

Purpose: Until 1996, there were only 7 masters-level programs in health professions education (HPE); currently, there are 76 such programs. The purpose of this article is to provide information and perspectives about the available masters programs in HPE worldwide, with a focus on their mission, objectives, content, instructional strategies, format, duration, and cost, as well as the similarities and differences among them. Method: A literature and web search was conducted to develop a complete list of programs that offer a masters degree in HPE or closely related areas. Results: Forty three percent [43% (33)] of these programs are in Europe, 20% (15) in North America, 17% (13) in Asia, 7% (5) in Latin America, 5% (4) in the Middle East, 5% (4) in Australia, and 3% (2) in Africa. The mission of these masters programs is to prepare leaders in HPE or sometimes narrowly focusing on clinical or dental education. The content is addressed in core courses and electives, generally grouped under the following headings: curriculum development, instructional methods/teaching and learning, assessment strategies, program evaluation, research design, and sometimes leadership and management. Medical schools sponsor approximately two-thirds of these programs, and the average duration for completion is 2 years. The vast majority of these programs are offered in English. Conclusion: The commonalities among these programs include focus, content, and educational requirements. The variations are mostly in organization and structure. There is a need to establish criteria and mechanisms for evaluation of these programs. The geographic maldistribution of these programs is a major concern.

Ensuring a fair and equitable selection of students to serve society's health care needs

This study aimed to evaluate a selection and programmatic intervention designated ‘Conditional Admissions’ (CA), which is intended to expand access to medical education for individuals from under‐represented ethnic, racial and rural groups. Further aims were to establish principles of practice designed to increase access for under‐represented groups based on an empirical comparison of programmatic changes made to CA in 2005, and to quantify the costs associated with its implementation.

The Impact of Mentoring and Advising At-risk Underrepresented Minority Students on Medical School Performance

This pilot study examined the impact of mentoring and advising at-risk underrepresented minority students received prior to matriculation on their performance in medical school.

Can final year medical students significantly contribute to patient care A pilot study about the perception of patients and clinical staff

Background: Active participation of medical students in patient care has been shown to be important for professional development of learners. Not much is known about the impact of active student participation (ASP) to the quality of patient care. Aims: We established a Clinical Education Ward (CEW) for the final year medical students caring for patients under structured clinical supervision. This study investigates the views of both patients and clinical staff on the impact of ASP on patient care. Methods: The Picker Inpatient Questionnaire (PIQ) was used to survey all the patients admitted to the CEW during the pilot phase. Results concerning the general quality of health care and the patient–physician relationship (PPR) were compared to two matched pair control groups: patients of the same department (CG1) and of internal wards in Germany (CG2). In addition, quantitative and qualitative data were collected from patients and clinical staff members to specify the impact of students on patient care. Results: Out of 111 patients, 64 responded. The PIQ results revealed very minor problems in the assessment of the overall general quality of care and in PPR at the CEW, while significant improvements existed when compared to CG2. Furthermore, 79% of the patients and 95% of the staff members recorded a positive impact of ASP. Qualitative data illustrated and complemented these results. Chances and challenges in programs with high participation of students in clinical care are discussed. Conclusion: ASP may not only be useful for learners but also offers chances and benefits for patient care.

Assessment of educational games for health professions: A systematic review of trends and outcomes

Abstract Background: Traditional lecturing used in teaching has the lowest retention rate; the use of games as part of an instruction method may enhance retention and reinforce learning by creating a dynamic educational environment. This study aims to systematically review the literature on educational games for the health professions to identify trends and investigate assessment tools used to measure its learning outcomes. Methods: Seven databases were used in the search: ERIC, Education Research Complete, Medline, Medline Complete, Academic Search Complete, The Cochrane Library and PubMed. Results: The search identified 2865 papers; among them, 1259 were excluded and 22 were evaluated. The selection incorporated five full papers which focused directly on the health professionals. Two studies involved the use of board games and two studies involved card games, crossword puzzles and one study involved a team quiz competition. Overall, studies lacked a strong link between the use of games for both instructional and assessment purposes. Conclusion: Gaming makes a positive impact on the teaching/learning process. However, existing assessment methodologies have been not fully captured the learning that may occur in these games. Robust research is needed to address the use of games that have been assessed objectively.

AM last page: master's degree in health professions education programs.

Disclaimer: The views expressed in this article are those of the authors and do not necessarily reflect the official policy of the U.S. Department of Defense. References and Online Resources 1. Tekian A, Harris I. Preparing health professions education leaders worldwide: A description of masters-level programs. Med Teach. 2012;34:52–58. 2. For a list of master’s degree in HPE programs in the United States, along with their associated Web site addresses, please see Supplemental Digital Table 1, available at http://links.lww.com/ACADMED/A146. 3. For additional information on master’s degree in HPE programs worldwide, please see the Foundation for Advancement of International Medical Education and Research (www.faimer.org) and MedEdWorld (www.mededworld.org). Author contact: tekian@uic.edu Just 15 years ago, the number of master’s degree in health professions education (HPE) programs worldwide was in the single digits.1 Today, there are 121 such programs, with new HPE programs created annually.1–3 The purpose of this AM Last Page is to provide information about the existing master’s degree in HPE programs, with a focus on location, core content, and program characteristics.

Structured training on box trainers for first year surgical residents: does it improve retention of laparoscopic skills? A randomized controlled study.

BACKGROUND AND AIM Structured training on box trainers in laparoscopic skills in the initial years of residency has been used and found to be effective. Although there are studies that confirm immediate improvement after training, there is a lack of well-designed trials addressing the crucial issue of retention of these skills over time. The purpose of this study is to assess improvement in laparoscopic skills of surgical trainees after structured training on box trainers, compared with traditional training (observing and assisting laparoscopic procedures in the operation rooms) immediately and after 5 months. METHODS Forty surgical residents in their first 2 months of residency training were randomized to either structured training on box trainers, in addition to traditional training, or to traditional training alone. Groups were equivalent with regards to demographics, previous operative experience, and baseline skills. Structured training consisted of 4 sessions with 6 tasks on box trainers under supervision and self practice. Task-based objective structured practical examinations (OSPE) were completed before and after each task. At the end of the training, residents were assessed by a blinded faculty member with the global operative assessment of laparoscopic skills (GOALS) rating scale. Residents also completed a satisfaction questionnaire. Focus group discussions were conducted for both groups. The GOALS were repeated for both the groups at the end of 5 months to assess retention of skills. RESULTS The mean GOALS score was significantly higher for the structured training group (mean/SD 20.35 + 0.74) compared with the traditional training group (mean/SD 16.35 + 1.75, p < 0.01) at the end of 5 months. The mean global rating scale (GRS) score was significantly higher (Pre 7.55 + 0.99 vs. Post 16.4 + 0.68, p < 0.01) for the structured training group at the end of course. Residents in the structured training group had significantly improved skills immediately after the training and had better retention of skills at the end of five months. CONCLUSIONS Structured training on box trainers, in addition to traditional training, compared with traditional training alone, leads to better skills and improved confidence of residents. There is significant retention of skills at the end of 5 months. These results provide support for incorporation of structured training with box trainers for laparoscopic skills into surgical training programs.

Assessing competencies using milestones along the way

Abstract This paper presents perspectives and controversies surrounding the use of milestones to assess competency in outcomes-based medical education. Global perspectives (Canada, Europe, and the United States) and developments supporting their rationales are discussed. In Canada, there is a significant movement away from conceptualizing competency based on time, and a move toward demonstration of specific competencies. The success of this movement may require complex (rather than reductionist) milestones that reflect students’ progression through complexity and context and a method to narrate their journey. European countries (United Kingdom, France, and Germany) have stressed the complexity associated with time and milestones for medical students to truly achieve competence. To meet the changing demands of medicine, they view time as actually providing students with knowledge and exposure to achieve various milestones. In the United States, milestones are based on sampling throughout professional development to initiate lifelong learning. However, the use of milestones may not imply overall competence (reductionism). Milestones must be developed alongside outcomes-based curriculum with use of faculty and competency committees. The perspectives outlined in this paper underscore emerging challenges for implementing outcomes-based medical education and call for new conceptualizations of competence.