M. Morris

Tools used to assess medical students competence in procedural skills at the end of a primary medical degree: a systematic review

The objective was to systematically review the literature to identify and grade tools used for the end point assessment of procedural skills (e.g., phlebotomy, IV cannulation, suturing) competence in medical students prior to certification. The authors searched eight bibliographic databases electronically – ERIC, Medline, CINAHL, EMBASE, Psychinfo, PsychLIT, EBM Reviews and the Cochrane databases. Two reviewers independently reviewed the literature to identify procedural assessment tools used specifically for assessing medical students within the PRISMA framework, the inclusion/exclusion criteria and search period. Papers on OSATS and DOPS were excluded as they focused on post-registration assessment and clinical rather than simulated competence. Of 659 abstracted articles 56 identified procedural assessment tools. Only 11 specifically assessed medical students. The final 11 studies consisted of 1 randomised controlled trial, 4 comparative and 6 descriptive studies yielding 12 heterogeneous procedural assessment tools for analysis. Seven tools addressed four discrete pre-certification skills, basic suture (3), airway management (2), nasogastric tube insertion (1) and intravenous cannulation (1). One tool used a generic assessment of procedural skills. Two tools focused on postgraduate laparoscopic skills and one on osteopathic students and thus were not included in this review. The levels of evidence are low with regard to reliability – κ = 0.65–0.71 and minimum validity is achieved – face and content. In conclusion, there are no tools designed specifically to assess competence of procedural skills in a final certification examination. There is a need to develop standardised tools with proven reliability and validity for assessment of procedural skills competence at the end of medical training. Medicine graduates must have comparable levels of procedural skills acquisition entering the clinical workforce irrespective of the country of training.

Communication skills assessment in the final postgraduate years to established practice: a systematic review

Introduction Communication breakdown is a factor in the majority of all instances of medical error. Despite the importance, a relative paucity of time is invested in communication skills in postgraduate curricula. Our objective is to systematically review the literature to identify the current tools used to assess communication skills in postgraduate trainees in the latter 2 years of training and in established practice. Methods Two reviewers independently reviewed the literature identifying communication skill assessment tools, for postgraduate trainees in the latter 2 years of training and in established practice following Preferred Reporting Items for Systematic Reviews and Meta-Analyses framework, and inclusion/exclusion criteria from January 1990 to 15 August 2014. Databases: PubMed/CINAHL/ERIC/EMBASE/PsycInfo/Psyc Articles/Cochrane. Results 222 articles were identified; after review, 34 articles fulfilled criteria for complete evaluation; the majority (26) had a high level of evidence scoring 3 or greater on the Best Evidence Medical Education guide. 22 articles used objective structured clinical examination/standardised patient (SP)-based formats in an assessment or training capacity. Evaluation tools included author-developed questionnaires and validated tools. Nineteen articles demonstrated an educational initiative. Conclusions The reviewed literature is heterogeneous for objectives and measurement techniques for communication. Observed interactions, with patients or SPs, is the current favoured method of evaluation using author-developed questionnaires. The role of self-evaluation of skill level is questioned. The need for a validated assessment tool for communication skills is highlighted.

The Fall and Rise of Medical Students’ Attitudes to Communication Skills Learning in Ireland: A Longitudinal Approach

Many studies have explored attitudes to communication skills learning before and after a teaching intervention but there remains a dearth of published longitudinal studies. An area currently unexplored is medical student’s attitudes to communication skills teaching and learning over the entirety of the undergraduate programme. A longitudinal approach was utilized in this study involving all medical students (n= 128) entering a Medicine Under Graduate Degree Programme in 2007 in a Dublin Medical School. Participants completed a previously validated Communication Skills Attitudinal Scale (CSAS) on Day 1 of the second medical year prior to clinical exposure. Once baseline attitudes were established, the tool was completed sequentially at the end of the 2, 3, 4 and 5 (final) medical years. Results indicated a mean Positive Attitude Score (PAS) of 51.9 (range 13-65) at the beginning of 2nd year declining to 45.5 at the end of this year. This decline in positivity was statistically significant with p < 0.035. Results indicated a mean Negative Attitude Score (NAS) of 29.8 (range 13-65) at the beginning of 2nd year rising to 33.8 at the end of this Research Article British Journal of Education, Society & Behavioural Science, 3(4): 467-477, 2013 468 year. However, the longitudinal approach taken highlighted that attitudes rose again by the end of the 3 year -mean PAS 48.7, and 4th 49.3 and attitudes almost returned to baseline by the end of the 5 Year mean PAS 49.3. The NAS score remained constant at 31.4 at the end of 3rd year, 31.4 at the end of 4th and 29.6 at the end of the 5th year .These changes in NAS were not statistically significant. A qualitative study is indicated to investigate the causation of the fall in positive attitude scores and rise in negative attitudes at the end of the second year. As communication skills are a core requirement for practicing in a professional discipline further research into these findings is warranted.

Pro forma: impact on communication skills?

A doctor performs 160 000–300 000 interviews during a lifetime career, thus making the medical interview the most common procedure in clinical medicine. It is reported that 60–80 per cent of diagnosis is based on history taking, yet there is little published data advising on the best method for medical students to initially attain and further refine these core skills during their medical degree.

An alternative certification examination 'ACE': can post graduate methods be used to assess clinical skills in medical under graduates.

Background: Progressing from undergraduate education to post graduate training has been punctuated by a clinical examination which has not changed significantly in decades. This study investigated the feasibility of using a validated postgraduate assessment methodology in an undergraduate setting; The Toronto Patient Assessment & Management Exam (PAME). Methods: A standardised patient-centred multifaceted healthcare pathway examination consisting of 4 separate consecutive encounters was piloted in the final year of undergraduate training. The entire final year medical class was invited to participate. The final sample of 25 was selected on a consecutive, volunteer basis. Student’s experienced 2 standardised simulated cases; 1 medical, 1 surgical. Candidates were examined by 2 independent examiners (subject experts) and were ranked on a Global Research Article British Journal of Medicine & Medical Research, 3(4): 1806-1817, 2013 1807 Rating Scale. Passing standard was set at 3/5 ‘barely adequate for Intern/PGY1’ but with the addition of second pass criteria of avoidance of an egregious error. Results: 23 students completed the examination. Two arrived late and were excluded. 21/23 demonstrated knowledge and skills at least at minimum expected standard. 18/23 avoided an egregious error. Subgroup analysis identified better performance in the assessment and management of the medical case and the review encounter (encounter 4) was the lowest scoring in both cases. The format was well received by students and examiners. Conclusion: The use of an alternative certification examination ‘ACE’ based on a postgraduate format ‘PAME’ in undergraduate setting appears feasible and discriminatory. Inclusion in the pass criteria of avoidance of egregious error appears to improve the specificity of the examination. The ACE format reveals potential to replace elements of prepractice (PGY1) clinical barrier assessment.

Prepared for Practice? Interns’ Experiences of Undergraduate Clinical Skills Training in Ireland

Background Many previous studies on internship have reported a lack of preparedness for the role. More recently in Ireland, medical schools have introduced formal clinical skills training programmes. This study sought to evaluate the impact, if any, of formal skills training in the medical training on interns preparedness for practice. Methods The study utilized a survey approach followed by focus group discussions. The aim was to identify the skills that were taught and assessed in medical training and the skills that were actually required in their intern year. Results Most interns had received skills training in designated skills laboratories. No intern had received training in all skills advised in the European guidelines. Skills taught to all interns were intravenous cannulation, basic life support, and basic suture. Skills required from all interns were intravenous cannulation, phlebotomy, and arterial blood sampling. Removal of peripherally inserted central line (PICC) lines, central lines, and chest drains were commonly requested but not taught. Senior staff underestimated skill abilities and expected failure. Conclusion These findings identify discordance between the skills taught and the skills required in the job. There is a need for standardization in the clinical skills training to ensure that all interns enter practice with equal competencies. Consideration should be given to experiential learning opportunities such as subintern programmes to consolidate learning and improve preparedness. Improvement in communications with senior clinicians is indicated to ensure that expectations are realistic and reflective of actual training.

An international survey of medical licensing requirements for immigrating physicians, focusing on communication evaluation

Objectives To identify current entry requirements set by international medical licensing bodies for immigrating physicians, focusing on postgraduate level communication skills, clinical and technical skill assessments. Methods A standardised, author developed survey was administered to a selection of national, state and provincial licensing institutions across 6 continents. Representative institutions were selected from the most populated regions of each continent. Surveys were administered by email and telephone. The information was also searched by website review. Website information alone was used if no response was received by the targeted institution after 2 phone/2 email attempts. Statistical analysis of the non-parametric data was conducted using SPSS (v.21). Results Thirty-seven licensing bodies were contacted from 30 countries; verifiable information was available for 29; twenty-six responded to the communication inquiry. Sixty five 65.4% (n=17) surveyed communication skills, 100% involved language proficiency testing; 11.5% tested other forms of communication skills. For clinical and technical skills, 86.2% (n=25) assessed candidates by credential review, 72.4% (n=21) required both credential review and exam and 62.1% (n=18) used country-specific examination. A mentorship period were required by 37.9% (n=11), ranging from 3 months to 1 year. Only 2 countries identified examinations for recertification. No technical/clinical skills nor communication skill evaluation (beyond language proficiency) are routinely assessed at the postgraduate level. Conclusions International assessments of migrating physicians are heterogeneous. Communication skills, beyond language proficiency, are not routinely assessed in foreign trained physicians seeking entry. The majority of clinical and technical skills are assessed by credential review only. This study highlights the lack of standardisation of assessment internationally and the need for steps toward a global agreement on training schemes and summative assessment.

Pilot evaluation of a novel observational tool for collaboration and communication within multidisciplinary team meetings (MDTs)

BACKGROUND Despite widespread use of multidisciplinary team meetings (MDTs) to facilitate patient care, little evidence exists that MDTs improve patient survival or impact care. Research into MDT function and outcome is limited. This pilot study looks to validate a novel tool developed to assess MDT functioning. METHODS A tool was developed with predefined Likert behaviour stems in the areas of structure, communication and collaboration. The tool was evaluated in nine MDTs by three independent observers and included participant evaluation. Inter-rater reliability was calculated with intraclass-correlation coefficients (ICC); Students t-test was used to calculate significance in participant evaluation and matched observations. RESULTS The tool was used to evaluate 9 MDTs, discussing 133 cases. The overall ICC for the three coders was 0.935; for each MDT: 0.776-0.917. The inter-rater reliability for each MDT observation ranged from 0.245-0.923; the majority with an ICC >0.8. No significant difference was noted between participant evaluation and observer response. DISCUSSION MDTs provide a means of coordinating complex care for patients. This tool provides a means of evaluating group interaction within MDTs and is designed for use by different medical personnel. The pilot study has shown promising ICC; further evaluation is needed using a broader group of MDTs and including case complexity and outcome.