William F. Bond

Differential Diagnosis Generators: an Evaluation of Currently Available Computer Programs

BackgroundDifferential diagnosis (DDX) generators are computer programs that generate a DDX based on various clinical data.ObjectiveWe identified evaluation criteria through consensus, applied these criteria to describe the features of DDX generators, and tested performance using cases from the New England Journal of Medicine (NEJM©) and the Medical Knowledge Self Assessment Program (MKSAP©).MethodsWe first identified evaluation criteria by consensus. Then we performed Google® and Pubmed searches to identify DDX generators. To be included, DDX generators had to do the following: generate a list of potential diagnoses rather than text or article references; rank or indicate critical diagnoses that need to be considered or eliminated; accept at least two signs, symptoms or disease characteristics; provide the ability to compare the clinical presentations of diagnoses; and provide diagnoses in general medicine. The evaluation criteria were then applied to the included DDX generators. Lastly, the performance of the DDX generators was tested with findings from 20 test cases. Each case performance was scored one through five, with a score of five indicating presence of the exact diagnosis. Mean scores and confidence intervals were calculated.Key ResultsTwenty three programs were initially identified and four met the inclusion criteria. These four programs were evaluated using the consensus criteria, which included the following: input method; mobile access; filtering and refinement; lab values, medications, and geography as diagnostic factors; evidence based medicine (EBM) content; references; and drug information content source. The mean scores (95% Confidence Interval) from performance testing on a five-point scale were Isabel© 3.45 (2.53, 4.37), DxPlain® 3.45 (2.63–4.27), Diagnosis Pro® 2.65 (1.75–3.55) and PEPID™ 1.70 (0.71–2.69). The number of exact matches paralleled the mean score finding.ConclusionsConsensus criteria for DDX generator evaluation were developed. Application of these criteria as well as performance testing supports the use of DxPlain® and Isabel© over the other currently available DDX generators.

Using Innovative Simulation Modalities for Civilian-Based, Chemical, Biological, Radiological, Nuclear, and Explosive Training in the Acute Management of Terrorist Victims: A Pilot Study

OBJECTIVESnChemical, biological, radiological, nuclear, and explosive (CBRNE) incidents are low frequency, high impact events that require specialized training outside of usual clinical practice. Educational modalities must recreate these clinical scenarios in order to provide realistic first responder/receiver training.nnnMETHODSnHigh fidelity, mannequin-based (HFMB) simulation and video clinical vignettes were used to create a simulation-based CBRNE course directed at the recognition, triage, and resuscitation of contaminated victims. The course participants, who consisted of first responders and receivers, were evaluated using a 43-question pre- and post-test that employed 12 video clinical vignettes as scenarios for the test questions. The results of the pre-test were analyzed according to the various medical training backgrounds of the participants to identify differences in baseline performance. A Scheffe post-hoc test and an ANOVA were used to determine differences between the medical training backgrounds of the participants. For those participants who completed both the pre-course and post-course test, the results were compared using a paired Students t-test.nnnRESULTSnA total of 54 first responders/receivers including physicians, nurses, and paramedics completed the course. Pre-course and post-course test results are listed by learner category. For all participants who took the pre-course test (n = 67), the mean value of the test scores was 53.5 +/- 12.7%. For all participants who took the post-course test (n = 54), the mean value of the test scores was 78.3 +/-10.9%. The change in score for those who took both the pre- and post-test (n = 54) achieved statistical significance at all levels of learner.nnnCONCLUSIONSnThe results suggest that video clinical vignettes and HFMB simulation are effective methods of CBRNE training and evaluation. Future studies should be conducted to determine the educational and cost-effectiveness of the use of these modalities.

Competence and retention in performance of the lumbar puncture procedure in a task trainer model

Introduction: Our objective was to establish competency and ensure retention in the steps of the lumbar puncture procedure. Methods: This was a prospective cohort study of first- and second-year emergency medicine residents. Residents completed a survey and then viewed a 5-minute PowerPoint™ slide presentation and a 15-minute video on performing the procedure. They completed a baseline assessment of competency using a lumbar puncture simulator, received feedback on their performance, and practiced the procedure. They self-recorded the number of practice attempts and performed a second procedure for assessment. Within 3 to 6 months, they performed the procedure for a third observation. The assessments were performed with the same simulator and directly observed by two raters. A previously validated critical actions checklist consisting of 23 critical actions was used. Competency was defined as ≥19 critical actions correct (>80%). Inter-rater reliability was examined using the intraclass correlation coefficient [ICC(2,k)]. Results: Seventeen first-year residents and nine second-year residents completed the initial training. Sixteen first-year residents and eight second-year residents completed the retention assessment. An additional four second-year residents were trained several months into their second year. Twelve of 17 first-year residents and 10 of 13 second-year residents demonstrated competence on the baseline evaluation. All residents demonstrated competence after practice (N = 30) and at the retention check (N = 24). The mean (SD) number of practice attempts before the postpractice assessment was 3.6 (1.1) for first years and 2.4 (2.3) for second years. Conclusions: This study demonstrated the achievement and retention of competency in the steps of the lumbar puncture procedure in a task trainer model.

Symptom-based, algorithmic approach for handling the initial encounter with victims of a potential terrorist attack.

OBJECTIVESnThis study intended to create symptom-based triage algorithms for the initial encounter with terror-attack victims. The goals of the triage algorithms include: (1) early recognition; (2) avoiding contamination; (3) early use of antidotes; (4) appropriate handling of unstable, contaminated victims; and (5) provisions of force protection. The algorithms also address industrial accidents and emerging infections, which have similar clinical presentations and risks for contamination as weapons of mass destruction (WMD).nnnMETHODSnThe algorithms were developed using references from military and civilian sources. They were tested and adjusted using a series of theoretical patients from a CD-ROM chemical, biological, radiological/nuclear, and explosive victim simulator. Then, the algorithms were placed into a card format and sent to experts in relevant fields for academic review.nnnRESULTSnSix inter-connected algorithms were created, described, and presented in figure form. The attack algorithm, for example, begins by differentiating between overt and covert attack victims (A covert attack is defined by epidemiological criteria adapted from the Centers for Disease Control and Prevention (CDC) recommendations). The attack algorithm then categorizes patients either as stable or unstable. Unstable patients flow to the Dirty Resuscitation algorithm, whereas, stable patients flow to the Chemical Agent and Biological Agent algorithms. The two remaining algorithms include the Suicide Bomb/Blast/Explosion and the Radiation Dispersal Device algorithms, which are inter-connected through the overt pathway in the Attack algorithm.nnnCONCLUSIONnA civilian, symptom-based, algorithmic approach to the initial encounter with victims of terrorist attacks, industrial accidents, or emerging infections was created. Future studies will address the usability of the algorithms with theoretical cases and utility in prospective, announced and unannounced, field drills. Additionally, future studies will assess the effectiveness of teaching modalities used to reinforce the algorithmic approach.

Is early analgesia associated with delayed treatment of appendicitis

PURPOSEnWe sought to investigate the relationship between delay in treatment of appendicitis and early use of analgesia.nnnBASIC PROCEDURESnWe designed a matched case-control study, with patients having delayed treatment of appendicitis as the cases and patients with no delay in treatment of appendicitis as controls matched for age, sex, Alvarado score, and date of diagnosis. Of 957 patients with appendicitis, there were 103 delayed cases. Matching patients were identified yielding 103 controls.nnnMAIN FINDINGSnIn comparing cases and controls for early opiate use (26/103 cases, 24/103 controls), there was no association with delayed treatment (odds ratio, 1.11; P = .745; 95% confidence interval, 0.59-3.89). When comparing cases and controls for early NSAID use (29/103 cases, 17/103 controls), an association was found with delayed treatment (odds ratio, 1.98; P = .045; 95% confidence interval, 1.01-3.89).nnnCONCLUSIONnFor early analgesia in appendicitis, we did not find an association with delayed treatment for opiate analgesia, but there did appear to be an association with nonsteroidal anti-inflammatory analgesia.

Gender Disparity in Emergency Department Non-ST Elevation Myocardial Infarction Management

BACKGROUNDnMany studies have looked at differences between men and women with acute coronary syndrome. These studies demonstrate that women have worse outcomes, receive fewer invasive interventions, and experience delay in the initiation of established medical therapies.nnnOBJECTIVEnUsing innovative technology, we set out to unveil and resolve any gender disparities in the evaluation and treatment of patients presenting with a positive troponin while in the emergency department. Our goal was to assess the feasibility of using a business management query system to create an automated data report that could identify deficiencies in standards of care and be used to improve the quality of treatment we provide our patients.nnnMETHODSnOver a 12-month period, key markers for patients with non-ST elevation myocardial infarction (NSTEMI) were tracked (e.g., time to electrocardiogram, door to medications). During this time, educational endeavors were initiated utilizing McKessons Horizon Business Insight™ (McKesson Information Solutions, Alpharetta, GA) to illustrate gender differences in standard therapy. Subsequently, indicators were evaluated for improvement.nnnRESULTSnSubstantial improvements in key indicators for management of NSTEMI were obtained and gender differences minimized where education was provided.nnnCONCLUSIONnThe integration of these information systems allowed us to create a successful performance improvement tool and, as an added benefit, nearly eliminated the need for manual retrospective chart reviews.

Simulation for Assessment of Milestones in Emergency Medicine Residents

OBJECTIVESnAll residency programs in the United States are required to report their residents progress on the milestones to the Accreditation Council for Graduate Medical Education (ACGME) biannually. Since the development and institution of this competency-based assessment framework, residency programs have been attempting to ascertain the best ways to assess resident performance on these metrics. Simulation was recommended by the ACGME as one method of assessment for many of the milestone subcompetencies. We developed three simulation scenarios with scenario-specific milestone-based assessment tools. We aimed to gather validity evidence for this tool.nnnMETHODSnWe conducted a prospective observational study to investigate the validity evidence for three mannequin-based simulation scenarios for assessing individual residents on emergency medicine (EM) milestones. The subcompetencies (i.e., patient care [PC]1, PC2, PC3) included were identified via a modified Delphi technique using a group of experienced EM simulationists. The scenario-specific checklist (CL) items were designed based on the individual milestone items within each EM subcompetency chosen for assessment and reviewed by experienced EM simulationists. Two independent live raters who were EM faculty at the respective study sites scored each scenario following brief rater training. The inter-rater reliability (IRR) of the assessment tool was determined by measuring intraclass correlation coefficient (ICC) for the sum of the CL items as well as the global rating scales (GRSs) for each scenario. Comparing GRS and CL scores between various postgraduate year (PGY) levels was performed with analysis of variance.nnnRESULTSnEight subcompetencies were chosen to assess with three simulation cases, using 118 subjects. Evidence of test content, internal structure, response process, and relations with other variables were found. The ICCs for the sum of the CL items and the GRSs were >0.8 for all cases, with one exception (clinical management GRSxa0= 0.74 in sepsis case). The sum of CL items and GRSs (pxa0<xa00.05) discriminated between PGY levels on all cases. However, when the specific CL items were mapped back to milestones in various proficiency levels, the milestones in the higher proficiency levels (level 3 [L3] and 4 [L4]) did not often discriminate between various PGY levels. L3 milestone items discriminated between PGY levels on five of 12 occasions they were assessed, and L4 items discriminated only two of 12 times they were assessed.nnnCONCLUSIONnThree simulation cases with scenario-specific assessment tools allowed evaluation of EM residents on proficiency L1 to L4 within eight of the EM milestone subcompetencies. Evidence of test content, internal structure, response process, and relations with other variables were found. Good to excellent IRR and the ability to discriminate between various PGY levels was found for both the sum of CL items and the GRSs. However, there was a lack of a positive relationship between advancing PGY level and the completion of higher-level milestone items (L3 and L4).

Creating Incentives for Accountability in Patient Care

When looking at incentive and payment systems, what behavior is being incentivized must be considered.

Creation and pilot testing of the advanced bioterrorism triage card

Study objectives: There is no symptom-based civilian triage protocol for bioterrorism victims. To that end, we develop a laminated triage algorithm foldout card with 4 goals: (1) maximize recognition of victims; (2) minimize secondary contamination; (3) teach principles of dirty resuscitation; and (4) expedite treatment of agents requiring early antidotes. The target audience includes any health care provider who may first contact a victim. We describe the card development and report the results of our pilot testing using the card to triage theoretical patients. We test the use of the card in those who have taken our 1-day Advanced Bioterrorism Triage Course (which includes the card) and those who have not taken the course to ascertain whether the card can be used effectively with and without the course. Methods: The initial card algorithms were developed using high-quality references from military and civilian sources. An exercise was then conducted, during which theoretical individual patients from a CD-ROM simulator (Bioterrorism Simulator 2002, Anesoft Corporation, Issaquah, WA) were used to test those patients flow through the algorithms. The algorithms were adjusted during this process until all the patients would flow through satisfactorily. There is an attack card that begins with overt versus covert attacks. The card proceeds to stable patient versus unstable patient and then flows to sections on dirty resuscitation, chemical agents, biological agents, bomb/blast, and radiation dispersal devices. Experts from multiple relevant fields (infectious disease, trauma, toxicology, disaster medicine, emergency medical services, and military chemical weapons response) reviewed and commented on the card to add face validity. To test the usability of the card, a series of 26 paragraph-length paper scenarios were created. The termination points on the attack card and the final cards were labeled. After reading each scenario, the participants were asked to note the attack card termination point and the final card termination point to capture how that individual would have triaged the patient. The scenarios were designed so that there would be 3 scenarios for each attack card termination point. The scenarios were also designed with a proposed difficulty index of easy, medium, and difficult scenarios. The participants were attending physicians and emergency residents. All participants were given a 1-page instruction sheet with 1 example. The answers of those taking the test were compared with the consensus answers developed by the 3 primary investigators. The proportion of correct answers in the training versus no-training groups was compared using the z test for correlations (using Fishers r to z transformation). Results: There were 8 attending physicians and 6 residents in the no-training group (N=14) and 3 attending physicians and 13 residents (N=16) in the trained group. The overall percentage of correct answers on the attack card was 73.1% for the no-training group and 79.8% for the training group ( P =.66). The percentage of those correct for the final card termination point had a variable N because one had to get the attack card correct to have an opportunity to answer the final card point correctly. According to that variable N, the no-training group had 77.8% correct and the training group had 78.5% correct ( P =.92). The attack card test items that showed the greatest percentage increase with training was the decision to go to dirty resuscitation (19.8% improvement) and botulism symptom recognition (17.3% improvement). The overall test percentages achieved are consistent with a moderately difficult testing instrument. We were unable to show good correlation between items the investigators thought would be difficult and those that were proven difficult by the test. Conclusion: The Advanced Bioterrorism Triage Card proved to be successful at assisting theoretical triage decisions with and without formal training in its use. The card includes a symptom-based civilian triage protocol for bioterrorism victims. Future studies will include a revised testing instrument and observational studies of field use during mass casualty drills.