Mark D. Hanson

Osce checklists do not capture increasing levels of expertise

PURPOSE To evaluate the effectiveness of binary content checklists in measuring increasing levels of clinical competence. METHOD Fourteen clinical clerks, 14 family practice residents, and 14 family physicians participated in two 15-minute standardized patient interviews. An examiner rated each participants performance using a binary content checklist and a global process rating. The participants provided a diagnosis two minutes into and at the end of the interview. RESULTS On global scales, the experienced clinicians scored significantly better than did the residents and clerks, but on checklists, the experienced clinicians scored significantly worse than did the residents and clerks. Diagnostic accuracy increased for all groups between the two-minute and 15-minute marks without significant differences between the groups. CONCLUSION These findings are consistent with the hypothesis that binary checklists may not be valid measures of increasing clinical competence.

The challenge of creating new OSCE measures to capture the characteristics of expertise.

Purpose  Although expert clinicians approach interviewing in a different manner than novices, OSCE measures have not traditionally been designed to take into account levels of expertise. Creating better OSCE measures requires an understanding of how the interviewing style of experts differs objectively from novices.

Adolescent Standardized Patients: Method of Selection and Assessment of Benefits and Risks

Background: Our psychiatric Objective Structured Clinical Examination (OSCE) group wishes to develop adolescent psychiatry OSCE stations. The literature regarding adolescent standardized patient (SP) selection methods and simulation effects, however, offered limited assurance that such adolescents would not experience adverse simulation effects. Purpose: Evaluation of adolescent SP selection methods and simulation effects for low- and high-stress roles. Method: A two-component (employment-psychological) SP selection method was used. Carefully selected SPs were assigned across three conditions: low-stress medical role, high-stress psychosocial role, and wait list control. Qualitative and quantitative measures were used to assess simulation effects. Results: Our selection method excluded 21% (7% employment and 14% psychological) of SP applicants. For SP participants, beneficial effects included acquisition of job skills and satisfaction in making an important contribution to society. SP reactions of discomfort to roles were reported. Long-term adverse effects were not identified. Conclusions: A two-component adolescent SP selection method is recommended. Adolescent SP benefits outweigh risks.

Teaching Pediatric Residents to Assess Adolescent Suicide Risk With a Standardized Patient Module

OBJECTIVE: We hypothesized that a suicide risk assessment (SRA) training module incorporating standardized patients (SPs) would enhance pediatric resident SRA performance. METHODS: We conducted an educational survey of pediatric residents regarding SRA (N = 80). In addition, we tested the performance of a SRA training module among pediatric interns who received SRA practice with SPs simulating suicidality scenarios, with (n = 6) or without (n = 6) SRA lecture, or SRA lecture only (n = 12) and control interns (n = 10). We examined postintervention confidence in SRA and self-reported and objectively measured knowledge of suicidal risk factors. RESULTS: Resident confidence and knowledge regarding SRA were low, compared with assessment of medical illness. Interns in the SP plus lecture group had significantly greater confidence in screening adolescents for suicide risk factors and assessing suicidal adolescents (screening, 4.2 ± 0.4; assessing, 4.2 ± 0.4), compared with subjects in either the lecture-only (screening, 2.9 ± 0.8; P = .005; assessing, 2.9 ± 1.1; P = .01) or control (screening, 3.1 ± 0.7; P = .025; assessing, 2.6 ± 0.8; P = .003) group. In addition, only the SP plus lecture group demonstrated significantly greater objective knowledge of suicide risk factors (92% vs 25% correct; P = .008) than the control group. Neither the lecture-only group nor the SP-only group was significantly better than the control group in terms of knowledge or confidence relevant to SRA. CONCLUSION: This SRA training module was significantly more effective than lecture alone in enhancing pediatric intern knowledge and confidence in SRA.

Medical school admissions: enhancing the reliability and validity of an autobiographical screening tool.

Background Most medical school applicants are screened out preinterview. Some cognitive scores available preinterview and some noncognitive scores available at interview demonstrate reasonable reliability and predictive validity. A reliable preinterview noncognitive measure would relax dependence upon screening based entirely on cognitive tendencies. Method In 2005, applicants interviewing at McMaster University’s Michael G. DeGroote School of Medicine completed an offsite, noninvigilated, Autobiographical Submission (ABS) preinterview and another onsite, invigilated, ABS at interview. Traditional and new ABS scoring methods were compared, with raters either evaluating all ABS questions for each candidate in turn (vertical scoring–traditional method) or evaluating all candidates for each question in turn (horizontal scoring–new method). Results The new scoring method revealed lower internal consistency and higher interrater reliability relative to the traditional method. More importantly, the new scoring method correlated better with the Multiple Mini-Interview (MMI) relative to the traditional method. Conclusions The new ABS scoring method revealed greater interrater reliability and predictive capacity, thus increasing its potential as a screen for noncognitive characteristics.

Are Clinicians Being Prepared to Care for Abused Women? A Survey of Health Professional Education in Ontario, Canada

BackgroundThe current project undertook a province-wide survey and environmental scan of educational opportunities available to future health care providers on the topic of intimate partner violence (IPV) against women.MethodsA team of experts identified university and college programs in Ontario, Canada as potential providers of IPV education to students in health care professions at the undergraduate and post-graduate levels. A telephone survey with contacts representing these programs was conducted between October 2005 and March 2006. The survey asked whether IPV-specific education was provided to learners, and if so, how and by whom.ResultsIn total, 222 eligible programs in dentistry, medicine, nursing and other allied health professions were surveyed, and 95% (212/222) of programs responded. Of these, 57% reported offering some form of IPV-specific education, with undergraduate nursing (83%) and allied health (82%) programs having the highest rates. Fewer than half of undergraduate medical (43%) and dentistry (46%) programs offered IPV content. Postgraduate programs ranged from no IPV content provision (dentistry) to 41% offering content (nursing).ConclusionSignificant variability exists across program areas regarding the methods for IPV education, its delivery and evaluation. The results of this project highlight that expectations for an active and consistent response by health care professionals to women experiencing the effects of violence may not match the realities of professional preparation.

Delinquent Firesetters: A Comparative Study of Delinquency and Firesetting Histories*

Twenty-five male delinquents (aged nine to 16) who had legal charges for setting fires filed against them were compared to a group of age and sex matched delinquents who had legal charges filed against them that were not related to firesetting. Groups were compared on demographic variables and on their delinquency and fire related histories. The only significant difference to emerge was a higher proportion of past firesetting in the group with fire-related charges. The present results indicate that a subgroup of antisocial recidivist firesetters exists among young offenders and underscore the importance of obtaining a thorough fire related history from youth who are before the court on fire related charges.

What do Psychiatry Residents Think of an Objective Structured Clinical Examination

Although interest is increasing in the use of Objective Structured Clinical Examinations (OSCEs) for the assessment of psychiatry residents, there are very few reports of the experiences and attitudes of residents to this form of evaluation. Eighteen residents in psychiatry participated in an OSCE consisting of eight 12-minute standardized patient interviews. Of the 15 who completed a postexamination survey, 93% rated the scenarios as very realistic and 80% considered the scenarios reflective of clinical situations they had experienced. The residents were very confident that such an examination could discriminate between incompetent and competent medical students, but the residents were less certain of the OSCE’s power to do so between competent and incompetent residents. While residents rejected the idea of such an OSCE for board certification, many felt they would like the experience to be a formative part of their training.

Firesetting during the preschool period: assessment and intervention issues

Objective To describe a methodology of assessing preschoolers involved in firesetting incidents, and outline the psychiatric implications of firesetting incidents in young children. Method To outline The Arson Prevention Program for Children and present case vignettes. Results The heightened risk of burn injury or fatality in fires caused by young children is highlighted and practical suggestions for facilitating the immediate safety of the child and family are presented. Conclusion Despite the popular notion that fire interest and play is relatively benign in young children, the cases show that, as with older children, firesetting in preschoolers can be associated with serious child and/or family psychopathology.

Nonpharmacological interventions for acute wound care distress in pediatric patients with burn injury: a systematic review.

Acute wound care distress among burn-injured pediatric patients is of major clinical concern. This systematic review evaluates the benefits of nonpharmacological interventions to reduce this distress. MEDLINE, EMBASE, CINAHL, PsycINFO, ERIC, and the Cochrane Library databases were searched using appropriate search terms for articles reporting overall psychological effects of pediatric burn injury. Key references were hand-searched. Searches yielded approximately 900 unique citations. Two authors reviewed each abstract, and 198 articles were retrieved, of which 34 were selected for full review. Of these 34 articles, 12 focused on acute wound care distress and nonpharmacological interventions. Critical appraisal of individual studies was conducted using the methods of the US Preventive Services Task Force, with a particular focus on assessing nonrandomized controlled trial designs. Twelve articles were reviewed and categorized according to intervention types child mediated (CM), parent mediated (PM), and health care provider mediated (HCPM). Using the US Preventive Services Task Force criteria, 7 of the 12 articles were rated “fair” or “good” and five were rated as having “poor” internal validity. The HCPM and CM intervention categories reported patient benefit. The two PM studies were both rated “poor.” Studies of nonpharmacological interventions to reduce pediatric burn distress were few, with a significant proportion (5/12) having concerns about internal validity. Patient benefit was reported for HCPM and CM interventions. Research designs incorporating control groups in studies that are adequately powered are needed. Additional research is required in the area of PM interventions in particular.