J. Stephen Higgins

Evidence-Based Guidelines for Fatigue Risk Management in EMS: Formulating Research Questions and Selecting Outcomes

ABSTRACT Background: Greater than half of Emergency Medical Services (EMS) personnel report work-related fatigue, yet there are no guidelines for the management of fatigue in EMS. A novel process has been established for evidence-based guideline (EBG) development germane to clinical EMS questions. This process has not yet been applied to operational EMS questions like fatigue risk management. The objective of this study was to develop content valid research questions in the Population, Intervention, Comparison, and Outcome (PICO) framework, and select outcomes to guide systematic reviews and development of EBGs for EMS fatigue risk management. Methods: We adopted the National Prehospital EBG Model Process and Grading of Recommendations Assessment, Development, and Evaluation (GRADE) framework for developing, implementing, and evaluating EBGs in the prehospital care setting. In accordance with steps one and two of the Model Process, we searched for existing EBGs, developed a multi-disciplinary expert panel and received external input. Panelists completed an iterative process to formulate research questions. We used the Content Validity Index (CVI) to score relevance and clarity of candidate PICO questions. The panel completed multiple rounds of question editing and used a CVI benchmark of ≥0.78 to indicate acceptable levels of clarity and relevance. Outcomes for each PICO question were rated from 1 = less important to 9 = critical. Results: Panelists formulated 13 candidate PICO questions, of which 6 were eliminated or merged with other questions. Panelists reached consensus on seven PICO questions (n = 1 diagnosis and n = 6 intervention). Final CVI scores of relevance ranged from 0.81 to 1.00. Final CVI scores of clarity ranged from 0.88 to 1.00. The mean number of outcomes rated as critical, important, and less important by PICO question was 0.7 (SD 0.7), 5.4 (SD 1.4), and 3.6 (SD 1.9), respectively. Patient and personnel safety were rated as critical for most PICO questions. PICO questions and outcomes were registered with PROSPERO, an international database of prospectively registered systematic reviews. Conclusions: We describe formulating and refining research questions and selection of outcomes to guide systematic reviews germane to EMS fatigue risk management. We outline a protocol for applying the Model Process and GRADE framework to create evidence-based guidelines

Systematic Review Methodology for the Fatigue in Emergency Medical Services Project

Abstract Background: Guidance for managing fatigue in the Emergency Medical Services (EMS) setting is limited. The Fatigue in EMS Project sought to complete multiple systematic reviews guided by seven explicit research questions, assemble the best available evidence, and rate the quality of that evidence for purposes of producing an Evidence Based Guideline (EBG) for fatigue risk management in EMS operations. Methods: We completed seven systematic reviews that involved searches of six databases for literature relevant to seven research questions. These questions were developed a priori by an expert panel and framed in the Population, Intervention, Comparison, and Outcome (PICO) format and pre-registered with PROSPERO. Our target population was defined as persons 18 years of age and older classified as EMS personnel or similar shift worker groups. A panel of experts selected outcomes for each PICO question as prescribed by the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) methodology. We pooled findings, stratified by study design (experimental vs. observational) and presented results of each systematic review in narrative and quantitative form. We used meta-analyses of select outcomes to generate pooled effects. We used the GRADE methodology and the GRADEpro software to designate a quality of evidence rating for each outcome. Results: We present the results for each systematic review in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA). More than 38,000 records were screened across seven systematic reviews. The median, minimum, and maximum inter-rater agreements (Kappa) between screeners for our seven systematic reviews were 0.66, 0.49, and 0.88, respectively. The median, minimum, and maximum number of records retained for the seven systematic reviews was 13, 1, and 100, respectively. We present key findings in GRADE Evidence Profile Tables in separate publications for each systematic review. Conclusions: We describe a protocol for conducting multiple, simultaneous systematic reviews connected to fatigue with the goal of creating an EBG for fatigue risk management in the EMS setting. Our approach may be informative to others challenged with the creation of EBGs that address multiple, inter-related systematic reviews with overlapping outcomes.

Effect of Fatigue Training on Safety, Fatigue, and Sleep in Emergency Medical Services Personnel and Other Shift Workers: A Systematic Review and Meta-Analysis

Abstract Background: Fatigue training may be an effective way to mitigate fatigue-related risk. We aimed to critically review and synthesize existing literature on the impact of fatigue training on fatigue-related outcomes for Emergency Medical Services (EMS) personnel and similar shift worker groups. Methods: We performed a systematic literature review for studies that tested the impact of fatigue training of EMS personnel or similar shift workers. Outcomes of interest included personnel safety, patient safety, personnel performance, acute fatigue, indicators of sleep duration and quality, indicators of long-term health (e.g., cardiovascular disease), and burnout/stress. A meta-analysis was performed to determine the impact of fatigue training on sleep quality. Results: Of the 3,817 records initially identified for review, 18 studies were relevant and examined fatigue training in shift workers using an experimental or quasi-experimental design. Fatigue training improved patient safety, personal safety, and ratings of acute fatigue and reduced stress and burnout. A meta-analysis of five studies showed improvement in sleep quality (Fixed Effects SMD −0.87; 95% CI −1.05 to −0.69; p < 0.00001; Random Effects SMD −0.80; 95% CI −1.72, 0.12; p < 0.00001). Conclusions: Reviewed literature indicated that fatigue training improved safety and health outcomes in shift workers. Further research is required to identify the optimal components of fatigue training programs to maximize the beneficial outcomes.

Asleep at the Wheel—The Road to Addressing Drowsy Driving

Drowsy driving is a dangerous behavior that leads to thousands of deaths and injuries each year. It is also a controllable factor for drivers. Drivers are capable of modifying this behavior if given sufficient information and motivation. Our goal is to establish a comprehensive and strategic effort to end drowsy driving crashes and deaths. This article highlights some of the conclusions of a unique recent meeting of sleep experts and highway safety professionals and describes the first steps the community has taken and plans to take in the future to address this issue.

Reliability and Validity of Survey Instruments to Measure Work-Related Fatigue in the Emergency Medical Services Setting: A Systematic Review

Abstract Background: This study sought to systematically search the literature to identify reliable and valid survey instruments for fatigue measurement in the Emergency Medical Services (EMS) occupational setting. Methods: A systematic review study design was used and searched six databases, including one website. The research question guiding the search was developed a priori and registered with the PROSPERO database of systematic reviews: “Are there reliable and valid instruments for measuring fatigue among EMS personnel?” (2016:CRD42016040097). The primary outcome of interest was criterion-related validity. Important outcomes of interest included reliability (e.g., internal consistency), and indicators of sensitivity and specificity. Members of the research team independently screened records from the databases. Full-text articles were evaluated by adapting the Bolster and Rourke system for categorizing findings of systematic reviews, and the rated data abstracted from the body of literature as favorable, unfavorable, mixed/inconclusive, or no impact. The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) methodology was used to evaluate the quality of evidence. Results: The search strategy yielded 1,257 unique records. Thirty-four unique experimental and non-experimental studies were determined relevant following full-text review. Nineteen studies reported on the reliability and/or validity of ten different fatigue survey instruments. Eighteen different studies evaluated the reliability and/or validity of four different sleepiness survey instruments. None of the retained studies reported sensitivity or specificity. Evidence quality was rated as very low across all outcomes. Conclusions: In this systematic review, limited evidence of the reliability and validity of 14 different survey instruments to assess the fatigue and/or sleepiness status of EMS personnel and related shift worker groups was identified.

Effects of Napping During Shift Work on Sleepiness and Performance in Emergency Medical Services Personnel and Similar Shift Workers: A Systematic Review and Meta-Analysis

Abstract Background: Scheduled napping during work shifts may be an effective way to mitigate fatigue-related risk. This study aimed to critically review and synthesize existing literature on the impact of scheduled naps on fatigue-related outcomes for EMS personnel and similar shift worker groups. Methods: A systematic literature review was performed of the impact of a scheduled nap during shift work on EMS personnel or similar shift workers. The primary (critical) outcome of interest was EMS personnel safety. Secondary (important) outcomes were patient safety; personnel performance; acute states of fatigue, alertness, and sleepiness; indicators of sleep duration and/or quality; employee retention/turnover; indicators of long-term health; and cost to the system. Meta-analyses were performed to evaluate the impact of napping on a measure of personnel performance (the psychomotor vigilance test [PVT]) and measures of acute fatigue. Results: Of 4,660 unique records identified, 13 experimental studies were determined relevant and summarized. The effect of napping on reaction time measured at the end of shift was small and non-significant (SMD 0.12, 95% CI −0.13 to 0.36; p = 0.34). Napping during work did not change reaction time from the beginning to the end of the shift (SMD −0.01, 95% CI −25.0 to 0.24; p = 0.96). Naps had a moderate, significant effect on sleepiness measured at the end of shift (SMD 0.40, 95% CI 0.09 to 0.72; p = 0.01). The difference in sleepiness from the start to the end of shift was moderate and statistically significant (SMD 0.41, 95% CI 0.09 to 0.72; p = 0.01). Conclusions: Reviewed literature indicated that scheduled naps at work improved performance and decreased fatigue in shift workers. Further research is required to identify the optimal timing and duration of scheduled naps to maximize the beneficial outcomes.

Effect of Task Load Interventions on Fatigue in Emergency Medical Services Personnel and Other Shift Workers: A Systematic Review.

Abstract Background: Modifying the task load of Emergency Medical Services (EMS) personnel may mitigate fatigue, sleep quality and fatigue related risks. A review of the literature addressing task load interventions may benefit EMS administrators as they craft policies related to mitigating fatigue. We conducted a systematic review of the peer-reviewed literature to address the following question: “In EMS personnel, do task load interventions mitigate fatigue, mitigate fatigue-related risks, and/or improve sleep?” (PROSPERO 2016:CRD42016040114). Methods: We performed a systematic review of the literature that described use of randomized controlled trials, quasi-experimental studies, and observational study designs. We retained and reviewed research that involved EMS personnel or similar shift worker groups 18 years of age and older. Studies of ‘healthy volunteers’ and non-shift worker populations were excluded. Studies were included where the methodology of the study implied a theoretical framework of task load (or workload) affecting fatigue, and then fatigue related outcomes. Outcomes of interest included personnel safety, patient safety, personnel performance, acute fatigue, and cost to system. We used the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) methodology to summarize findings and assess quality of evidence from very low to high quality. Results: The search strategy yielded 3,394 unique records resulting in 58 records included as potentially eligible. An additional 69 studies were reviewed in full following searches of bibliographies. We detected wide variation in the description and measurement of task load in the retained and excluded research. Among 127 potentially relevant studies reviewed in full, five were judged eligible. None of the retained studies reported findings germane to personnel safety, patient safety, or cost to system. We judged most studies to have serious or very serious risk of bias. Conclusions: The effect of task load interventions on fatigue, fatigue-related risks, and/or sleep quality was not estimable and the overall quality of evidence was judged low or very low. There was considerable heterogeneity in how task load was defined and measured.

Systematic Review and Meta-analysis of the Effects of Caffeine in Fatigued Shift Workers: Implications for Emergency Medical Services Personnel

Abstract Background: Emergency Medical Services (EMS) workers may experience fatigue as a consequence of shift work. We reviewed the literature to determine the impact of caffeine as a countermeasure to fatigue in EMS personnel and related shift workers. Methods: We employed the GRADE methodology to perform a systematic literature review and search multiple databases for research that examined the impact of caffeine on outcomes of interest, such as patient and EMS personnel safety. For selected outcomes, we performed a meta-analysis of pooled data and reported the pooled effect in the form of a Standardized Mean Difference (SMD) with corresponding 95% confidence intervals. Results: There are no studies that investigate caffeine use and its effects on EMS workers or on patient safety. Four of 8 studies in shift workers showed that caffeine improved psychomotor vigilance, which is important for performance. Caffeine decreased the number of lapses on a standardized test of performance [SMD = 0.75 (95% CI: 0.30 to 1.19), p = 0.001], and lessened the slowing of reaction time at the end of shifts [SMD = 0.52 (95% CI: 0.19 to 0.85); p = 0.002]. Finally, 2 studies reported that caffeine reduced sleep quality and sleep duration. Conclusions: Although the quality of evidence was judged to be low to moderate, when taken together, these studies demonstrate that caffeine can improve psychomotor performance and vigilance. However, caffeine negatively affects sleep quality and sleep duration. More systematic, randomized studies need to be conducted in EMS workers in order to address the critical outcomes of health and safety of EMS personnel and patients. The risk/benefit ratio of chronic caffeine use in shift workers is currently unknown.

Shorter Versus Longer Shift Durations to Mitigate Fatigue and Fatigue-Related Risks in Emergency Medical Services Personnel and Related Shift Workers: A Systematic Review

ABSTRACT Background: This study comprehensively reviewed the literature on the impact of shorter versus longer shifts on critical and important outcomes for Emergency Medical Services (EMS) personnel and related shift worker groups. Methods: Six databases (e.g., PubMed/MEDLINE) were searched, including one website. This search was guided by a research question developed by an expert panel a priori and registered with the PROSPERO database of systematic reviews (2016:CRD42016040099). The critical outcomes of interest were patient safety and personnel safety. The important outcomes of interest were personnel performance, acute fatigue, sleep and sleep quality, retention/turnover, long-term health, burnout/stress, and cost to system. Screeners worked independently and full-text articles were assessed for relevance. Data abstracted from the retained literature were categorized as favorable, unfavorable, mixed/inconclusive, or no impact toward the shorter shift duration. This research characterized the evidence as very low, low, moderate, or high quality according to the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) methodology. Results: The searched yielded n = 21,674 records. Of the 480 full-text articles reviewed, 100 reported comparisons of outcomes of interest by shift duration. We identified 24 different shift duration comparisons, most commonly 8 hours versus 12 hours. No one study reported findings for all 9 outcomes. Two studies reported findings linked to both critical outcomes of patient and personnel safety, 34 reported findings for one of two critical outcomes, and 64 did not report findings for critical outcomes. Fifteen studies were grouped to compare shifts <24 hours versus shifts ≥24 hours. None of the findings for the critical outcomes of patient and personnel safety were categorized as unfavorable toward shorter duration shifts (<24 hours). Nine studies were favorable toward shifts <24 hours for at least one of the 7 important outcomes, while findings from one study were categorized as unfavorable. Evidence quality was low or very low. Conclusions: The quality of existing evidence on the impact of shift duration on fatigue and fatigue-related risks is low or very low. Despite these limitations, this systematic review suggests that for outcomes considered critical or important to EMS personnel, shifts <24 hours in duration are more favorable than shifts ≥24 hours.

Evidence-Based Guidelines for Fatigue Risk Management in Emergency Medical Services

Abstract Background: Administrators of Emergency Medical Services (EMS) operations lack guidance on how to mitigate workplace fatigue, which affects greater than half of all EMS personnel. The primary objective of the Fatigue in EMS Project was to create an evidence-based guideline for fatigue risk management tailored to EMS operations. Methods: Systematic searches were conducted from 1980 to September 2016 and guided by seven research questions framed in the Population, Intervention, Comparison, Outcome (PICO) framework. Teams of investigators applied inclusion criteria, which included limiting the retained literature to EMS personnel or similar shift worker groups. The expert panel reviewed summaries of the evidence based on the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) methodology. The panel evaluated the quality of evidence for each PICO question separately, considered the balance between benefits and harms, considered the values and preferences of the targeted population, and evaluated the resource requirements/needs. The GRADE Evidence-to-Decision (EtD) Framework was used to prepare draft recommendations based on the evidence, and the Content Validity Index (CVI) was used to quantify the panels agreement on the relevance and clarity of each recommendation. CVI scores for relevance and clarity were measured separately on a 1–4 scale to indicate consensus/agreement among panel members and conclusion of recommendation development. Results: The EtD framework was applied to all 7 PICO questions, and the panel created 5 recommendations. PICO1: The panel recommends using fatigue/sleepiness survey instruments to measure and monitor fatigue in EMS personnel. PICO2: The panel recommends that EMS personnel work shifts shorter than 24 hours in duration. PICO3: The panel recommends that EMS personnel have access to caffeine as a fatigue countermeasure. PICO4: The panel recommends that, EMS personnel have the opportunity to nap while on duty to mitigate fatigue. PICO5: The panel recommends that EMS personnel receive education and training to mitigate fatigue and fatigue-related risks. The panel referenced insufficient evidence as the reason for making no recommendation linked to 2 PICO questions. Conclusions: Based on a review of the evidence, the panel developed a guideline with 5 recommendations for fatigue risk management in EMS operations.