P. Daniel Patterson

Use of ED diagnosis to determine medical necessity of EMS transports

Objective. To examine interrater agreement for classifying emergency medical services transports as medically unnecessary using emergency department diagnosis as the sole determining factor. Methods. Three emergency physicians andtwo family medicine physicians classified 913 International Classification of Diseases, Ninth Revision (ICD-9) codes as medically necessary, unnecessary, or uncertain. Overall agreement, interrater agreement, andagreement within 17 major disease categories were measured using κ statistics in SAS. Results. Physicians rated between 25% and65% of diagnoses codes as medically unnecessary. Overall agreement was fair (κ = 0.31). Agreement within specialties was higher among family medicine–trained physicians than among emergency physicians (κ = 0.52 andκ = 0.22, respectively). Agreement across all raters was highest for diseases classified as symptoms, signs, andill-defined conditions (κ = 0.40) andlowest for diseases of the blood andblood-forming organs (κ = −0.17). Agreement was observably better between physicians with more experience. Conclusions. Considerable doubts about the utility of emergency department diagnosis as a criterion are raised from study findings. Further development of Neely Conference criteria is needed. Priority should be given to testing andvalidation of criteria as well as exploration of differences in judgment between specialists representative of the medical director profession.

Medically Unnecessary Emergency Medical Services (EMS) Transports Among Children Ages 0 to 17 Years

Objectives: Estimate the prevalence of medically unnecessary Emergency Medical Services (EMS) transports among children. Methods: We linked EMS and emergency department (ED) billing records for all EMS-to-hospital transports of children originating in three counties in South Carolina between January 1, 2001 and March 31, 2003. EMS responses resulting in no transport, transports to destinations other than the ED, or multiple trips for the same child in a single day could not be linked to ED data and were excluded. Medically unnecessary transports were identified with an algorithm using pre-hospital impressions, ED diagnoses and ED procedures. After exclusions, 5,693 transports of children between 0 and 17 years were available for study. Results: Sixteen percent (16.4%) of all transports were medically unnecessary. Among children through age 12, upper respiratory and viral problems were the most common diagnoses associated with medically unnecessary transports; among older children, behavioral problems such as conduct disturbance or drug abuse were common. In multivariable analysis, the odds of an unnecessary transport were higher among younger children, non-white children, rural children, and children insured by Medicaid. Conclusions: The proportion of EMS transports which may be medically unnecessary is relatively modest compared to previous studies. However, many questions remain for future research. Further investigation should include examination of primary care availability and occurrence of unnecessary EMS use, existence of race-based disparities, and transports involving conduct disturbance and other behavioral conditions among children.

Evidence-Based Guidelines for Fatigue Risk Management in EMS: Formulating Research Questions and Selecting Outcomes

ABSTRACT Background: Greater than half of Emergency Medical Services (EMS) personnel report work-related fatigue, yet there are no guidelines for the management of fatigue in EMS. A novel process has been established for evidence-based guideline (EBG) development germane to clinical EMS questions. This process has not yet been applied to operational EMS questions like fatigue risk management. The objective of this study was to develop content valid research questions in the Population, Intervention, Comparison, and Outcome (PICO) framework, and select outcomes to guide systematic reviews and development of EBGs for EMS fatigue risk management. Methods: We adopted the National Prehospital EBG Model Process and Grading of Recommendations Assessment, Development, and Evaluation (GRADE) framework for developing, implementing, and evaluating EBGs in the prehospital care setting. In accordance with steps one and two of the Model Process, we searched for existing EBGs, developed a multi-disciplinary expert panel and received external input. Panelists completed an iterative process to formulate research questions. We used the Content Validity Index (CVI) to score relevance and clarity of candidate PICO questions. The panel completed multiple rounds of question editing and used a CVI benchmark of ≥0.78 to indicate acceptable levels of clarity and relevance. Outcomes for each PICO question were rated from 1 = less important to 9 = critical. Results: Panelists formulated 13 candidate PICO questions, of which 6 were eliminated or merged with other questions. Panelists reached consensus on seven PICO questions (n = 1 diagnosis and n = 6 intervention). Final CVI scores of relevance ranged from 0.81 to 1.00. Final CVI scores of clarity ranged from 0.88 to 1.00. The mean number of outcomes rated as critical, important, and less important by PICO question was 0.7 (SD 0.7), 5.4 (SD 1.4), and 3.6 (SD 1.9), respectively. Patient and personnel safety were rated as critical for most PICO questions. PICO questions and outcomes were registered with PROSPERO, an international database of prospectively registered systematic reviews. Conclusions: We describe formulating and refining research questions and selection of outcomes to guide systematic reviews germane to EMS fatigue risk management. We outline a protocol for applying the Model Process and GRADE framework to create evidence-based guidelines

Systematic Review Methodology for the Fatigue in Emergency Medical Services Project

Abstract Background: Guidance for managing fatigue in the Emergency Medical Services (EMS) setting is limited. The Fatigue in EMS Project sought to complete multiple systematic reviews guided by seven explicit research questions, assemble the best available evidence, and rate the quality of that evidence for purposes of producing an Evidence Based Guideline (EBG) for fatigue risk management in EMS operations. Methods: We completed seven systematic reviews that involved searches of six databases for literature relevant to seven research questions. These questions were developed a priori by an expert panel and framed in the Population, Intervention, Comparison, and Outcome (PICO) format and pre-registered with PROSPERO. Our target population was defined as persons 18 years of age and older classified as EMS personnel or similar shift worker groups. A panel of experts selected outcomes for each PICO question as prescribed by the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) methodology. We pooled findings, stratified by study design (experimental vs. observational) and presented results of each systematic review in narrative and quantitative form. We used meta-analyses of select outcomes to generate pooled effects. We used the GRADE methodology and the GRADEpro software to designate a quality of evidence rating for each outcome. Results: We present the results for each systematic review in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA). More than 38,000 records were screened across seven systematic reviews. The median, minimum, and maximum inter-rater agreements (Kappa) between screeners for our seven systematic reviews were 0.66, 0.49, and 0.88, respectively. The median, minimum, and maximum number of records retained for the seven systematic reviews was 13, 1, and 100, respectively. We present key findings in GRADE Evidence Profile Tables in separate publications for each systematic review. Conclusions: We describe a protocol for conducting multiple, simultaneous systematic reviews connected to fatigue with the goal of creating an EBG for fatigue risk management in the EMS setting. Our approach may be informative to others challenged with the creation of EBGs that address multiple, inter-related systematic reviews with overlapping outcomes.

Effect of Fatigue Training on Safety, Fatigue, and Sleep in Emergency Medical Services Personnel and Other Shift Workers: A Systematic Review and Meta-Analysis

Abstract Background: Fatigue training may be an effective way to mitigate fatigue-related risk. We aimed to critically review and synthesize existing literature on the impact of fatigue training on fatigue-related outcomes for Emergency Medical Services (EMS) personnel and similar shift worker groups. Methods: We performed a systematic literature review for studies that tested the impact of fatigue training of EMS personnel or similar shift workers. Outcomes of interest included personnel safety, patient safety, personnel performance, acute fatigue, indicators of sleep duration and quality, indicators of long-term health (e.g., cardiovascular disease), and burnout/stress. A meta-analysis was performed to determine the impact of fatigue training on sleep quality. Results: Of the 3,817 records initially identified for review, 18 studies were relevant and examined fatigue training in shift workers using an experimental or quasi-experimental design. Fatigue training improved patient safety, personal safety, and ratings of acute fatigue and reduced stress and burnout. A meta-analysis of five studies showed improvement in sleep quality (Fixed Effects SMD −0.87; 95% CI −1.05 to −0.69; p < 0.00001; Random Effects SMD −0.80; 95% CI −1.72, 0.12; p < 0.00001). Conclusions: Reviewed literature indicated that fatigue training improved safety and health outcomes in shift workers. Further research is required to identify the optimal components of fatigue training programs to maximize the beneficial outcomes.

Reliability and Validity of Survey Instruments to Measure Work-Related Fatigue in the Emergency Medical Services Setting: A Systematic Review

Abstract Background: This study sought to systematically search the literature to identify reliable and valid survey instruments for fatigue measurement in the Emergency Medical Services (EMS) occupational setting. Methods: A systematic review study design was used and searched six databases, including one website. The research question guiding the search was developed a priori and registered with the PROSPERO database of systematic reviews: “Are there reliable and valid instruments for measuring fatigue among EMS personnel?” (2016:CRD42016040097). The primary outcome of interest was criterion-related validity. Important outcomes of interest included reliability (e.g., internal consistency), and indicators of sensitivity and specificity. Members of the research team independently screened records from the databases. Full-text articles were evaluated by adapting the Bolster and Rourke system for categorizing findings of systematic reviews, and the rated data abstracted from the body of literature as favorable, unfavorable, mixed/inconclusive, or no impact. The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) methodology was used to evaluate the quality of evidence. Results: The search strategy yielded 1,257 unique records. Thirty-four unique experimental and non-experimental studies were determined relevant following full-text review. Nineteen studies reported on the reliability and/or validity of ten different fatigue survey instruments. Eighteen different studies evaluated the reliability and/or validity of four different sleepiness survey instruments. None of the retained studies reported sensitivity or specificity. Evidence quality was rated as very low across all outcomes. Conclusions: In this systematic review, limited evidence of the reliability and validity of 14 different survey instruments to assess the fatigue and/or sleepiness status of EMS personnel and related shift worker groups was identified.

Effects of Napping During Shift Work on Sleepiness and Performance in Emergency Medical Services Personnel and Similar Shift Workers: A Systematic Review and Meta-Analysis

Abstract Background: Scheduled napping during work shifts may be an effective way to mitigate fatigue-related risk. This study aimed to critically review and synthesize existing literature on the impact of scheduled naps on fatigue-related outcomes for EMS personnel and similar shift worker groups. Methods: A systematic literature review was performed of the impact of a scheduled nap during shift work on EMS personnel or similar shift workers. The primary (critical) outcome of interest was EMS personnel safety. Secondary (important) outcomes were patient safety; personnel performance; acute states of fatigue, alertness, and sleepiness; indicators of sleep duration and/or quality; employee retention/turnover; indicators of long-term health; and cost to the system. Meta-analyses were performed to evaluate the impact of napping on a measure of personnel performance (the psychomotor vigilance test [PVT]) and measures of acute fatigue. Results: Of 4,660 unique records identified, 13 experimental studies were determined relevant and summarized. The effect of napping on reaction time measured at the end of shift was small and non-significant (SMD 0.12, 95% CI −0.13 to 0.36; p = 0.34). Napping during work did not change reaction time from the beginning to the end of the shift (SMD −0.01, 95% CI −25.0 to 0.24; p = 0.96). Naps had a moderate, significant effect on sleepiness measured at the end of shift (SMD 0.40, 95% CI 0.09 to 0.72; p = 0.01). The difference in sleepiness from the start to the end of shift was moderate and statistically significant (SMD 0.41, 95% CI 0.09 to 0.72; p = 0.01). Conclusions: Reviewed literature indicated that scheduled naps at work improved performance and decreased fatigue in shift workers. Further research is required to identify the optimal timing and duration of scheduled naps to maximize the beneficial outcomes.

Effect of Task Load Interventions on Fatigue in Emergency Medical Services Personnel and Other Shift Workers: A Systematic Review.

Abstract Background: Modifying the task load of Emergency Medical Services (EMS) personnel may mitigate fatigue, sleep quality and fatigue related risks. A review of the literature addressing task load interventions may benefit EMS administrators as they craft policies related to mitigating fatigue. We conducted a systematic review of the peer-reviewed literature to address the following question: “In EMS personnel, do task load interventions mitigate fatigue, mitigate fatigue-related risks, and/or improve sleep?” (PROSPERO 2016:CRD42016040114). Methods: We performed a systematic review of the literature that described use of randomized controlled trials, quasi-experimental studies, and observational study designs. We retained and reviewed research that involved EMS personnel or similar shift worker groups 18 years of age and older. Studies of ‘healthy volunteers’ and non-shift worker populations were excluded. Studies were included where the methodology of the study implied a theoretical framework of task load (or workload) affecting fatigue, and then fatigue related outcomes. Outcomes of interest included personnel safety, patient safety, personnel performance, acute fatigue, and cost to system. We used the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) methodology to summarize findings and assess quality of evidence from very low to high quality. Results: The search strategy yielded 3,394 unique records resulting in 58 records included as potentially eligible. An additional 69 studies were reviewed in full following searches of bibliographies. We detected wide variation in the description and measurement of task load in the retained and excluded research. Among 127 potentially relevant studies reviewed in full, five were judged eligible. None of the retained studies reported findings germane to personnel safety, patient safety, or cost to system. We judged most studies to have serious or very serious risk of bias. Conclusions: The effect of task load interventions on fatigue, fatigue-related risks, and/or sleep quality was not estimable and the overall quality of evidence was judged low or very low. There was considerable heterogeneity in how task load was defined and measured.

Systematic Review and Meta-analysis of the Effects of Caffeine in Fatigued Shift Workers: Implications for Emergency Medical Services Personnel

Abstract Background: Emergency Medical Services (EMS) workers may experience fatigue as a consequence of shift work. We reviewed the literature to determine the impact of caffeine as a countermeasure to fatigue in EMS personnel and related shift workers. Methods: We employed the GRADE methodology to perform a systematic literature review and search multiple databases for research that examined the impact of caffeine on outcomes of interest, such as patient and EMS personnel safety. For selected outcomes, we performed a meta-analysis of pooled data and reported the pooled effect in the form of a Standardized Mean Difference (SMD) with corresponding 95% confidence intervals. Results: There are no studies that investigate caffeine use and its effects on EMS workers or on patient safety. Four of 8 studies in shift workers showed that caffeine improved psychomotor vigilance, which is important for performance. Caffeine decreased the number of lapses on a standardized test of performance [SMD = 0.75 (95% CI: 0.30 to 1.19), p = 0.001], and lessened the slowing of reaction time at the end of shifts [SMD = 0.52 (95% CI: 0.19 to 0.85); p = 0.002]. Finally, 2 studies reported that caffeine reduced sleep quality and sleep duration. Conclusions: Although the quality of evidence was judged to be low to moderate, when taken together, these studies demonstrate that caffeine can improve psychomotor performance and vigilance. However, caffeine negatively affects sleep quality and sleep duration. More systematic, randomized studies need to be conducted in EMS workers in order to address the critical outcomes of health and safety of EMS personnel and patients. The risk/benefit ratio of chronic caffeine use in shift workers is currently unknown.

Shorter Versus Longer Shift Durations to Mitigate Fatigue and Fatigue-Related Risks in Emergency Medical Services Personnel and Related Shift Workers: A Systematic Review

ABSTRACT Background: This study comprehensively reviewed the literature on the impact of shorter versus longer shifts on critical and important outcomes for Emergency Medical Services (EMS) personnel and related shift worker groups. Methods: Six databases (e.g., PubMed/MEDLINE) were searched, including one website. This search was guided by a research question developed by an expert panel a priori and registered with the PROSPERO database of systematic reviews (2016:CRD42016040099). The critical outcomes of interest were patient safety and personnel safety. The important outcomes of interest were personnel performance, acute fatigue, sleep and sleep quality, retention/turnover, long-term health, burnout/stress, and cost to system. Screeners worked independently and full-text articles were assessed for relevance. Data abstracted from the retained literature were categorized as favorable, unfavorable, mixed/inconclusive, or no impact toward the shorter shift duration. This research characterized the evidence as very low, low, moderate, or high quality according to the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) methodology. Results: The searched yielded n = 21,674 records. Of the 480 full-text articles reviewed, 100 reported comparisons of outcomes of interest by shift duration. We identified 24 different shift duration comparisons, most commonly 8 hours versus 12 hours. No one study reported findings for all 9 outcomes. Two studies reported findings linked to both critical outcomes of patient and personnel safety, 34 reported findings for one of two critical outcomes, and 64 did not report findings for critical outcomes. Fifteen studies were grouped to compare shifts <24 hours versus shifts ≥24 hours. None of the findings for the critical outcomes of patient and personnel safety were categorized as unfavorable toward shorter duration shifts (<24 hours). Nine studies were favorable toward shifts <24 hours for at least one of the 7 important outcomes, while findings from one study were categorized as unfavorable. Evidence quality was low or very low. Conclusions: The quality of existing evidence on the impact of shift duration on fatigue and fatigue-related risks is low or very low. Despite these limitations, this systematic review suggests that for outcomes considered critical or important to EMS personnel, shifts <24 hours in duration are more favorable than shifts ≥24 hours.