Kenneth Bond

Applying the Risk of Bias Tool in a Systematic Review of Combination Long-Acting Beta-Agonists and Inhaled Corticosteroids for Persistent Asthma

Background The Risk of Bias (RoB) tool is used to assess internal validity of randomized controlled trials (RCTs). Our objectives were to: 1) evaluate inter-rater agreement of the RoB tool; 2) determine the time to access supplemental study information; 3) compare the RoB tool with the Jadad scale and Schulz allocation concealment (AC); and 4) examine the relationship between RoB and effect estimates. Methods We conducted a systematic review of long-acting beta agonists (LABA) combined with inhaled corticosteroids (ICS) for adults with persistent asthma. Two reviewers independently assessed 107 trials using RoB, Jadad, and AC. One reviewer searched for study protocols. We assessed inter-rater agreement using weighted Kappa (κ) and the correlation between tools using Kendalls Tau (τ). Mean differences in effect sizes for RCTs with different RoB were calculated using inverse variance method and random effects model. Results Trials had good Jadad scores (median 4, IQR 3-4); however, 85% had unclear AC and 87% high RoB. The factor that most influenced RoB was the potential inappropriate influence of study sponsors (95% industry funded). Agreement on RoB domains was fair (κ = 0.40) to almost perfect (κ = 0.86), and moderate for overall RoB (κ = 0.41). Median time to complete RoB assessments was 21 minutes (IQR 14-27) and 12 minutes (IQR 9-16) to search for protocols. Protocols were identified for 5/42 studies (12%); in 3 cases the assessment of selective outcome reporting changed. There was low correlation between overall RoB vs. Jadad (τ = 0.04, p = 0.3) and AC (τ = −0.02, p = 0.7). Analyses comparing effect estimates and risk showed no important patterns. Conclusions Inter-rater agreement on RoB assessments was better than previously reported suggesting that review-specific guidelines are important. The correlation between RoB and Jadad was low suggesting measurement of different constructs (risk of bias vs. quality of reporting). The extensive involvement of the pharmaceutical industry in this LABA/ICS research should raise concerns about potential overestimates of treatment effects.

Key indicators of overcrowding in Canadian emergency departments: a Delphi study.

OBJECTIVE To identify the level of consensus among a group of Canadian emergency department (ED) experts on the importance of a set of indicators to document ED overcrowding. METHODS A 2-round Delphi survey was conducted from February 2005 to April 2005, with a multidisciplinary group of 38 Canadian experts in various aspects of ED operations who rated the relevance of 36 measures and ranked their relative importance as indicators of ED overcrowding. RESULTS The response rates for the first and second rounds were 84% and 87%, respectively. The most important indicator identified by the experts was the percentage of the ED occupied by inpatients (mean on a 7-point Likert-type scale 6.53, standard deviation [SD] 0.80). The other 9 indicators, in order of the importance attributed, were the total number of ED patients (mean 6.35, SD 0.75), the total time in the ED (mean 6.16, SD 1.04), the percentage of time that the ED was at or above capacity (mean 6.16, SD 1.08), the overall bed occupancy (mean 6.19, SD 0.93), the time from bed request to bed assignment (mean 6.06, SD 1.08), the time from triage to care (mean 5.84, SD 1.08) the physician satisfaction (mean 5.84, SD 1.22), the time from bed availability to ward transfer (mean 5.53, SD 1.72) and the number of staffed acute care beds (mean 5.53, SD 1.57). CONCLUSION Ten clinically important measures were prioritized by the participants as relevant indicators of ED overcrowding. Indicators derived from consensus techniques have face validity, but their metric properties must be tested to ensure their effectiveness for identifying ED overcrowding in different settings.

Data collection on patients in emergency departments in Canada.

OBJECTIVE Relatively little is known about the ability of Canadian emergency departments (EDs) and the federal, provincial and territorial governments to quantify ED activity. The objectives of this study were to determine the use of electronic patient data in Canadian EDs, the accessibility of provincial data on ED visits, and to identify the data elements and current methods of ED information system (EDIS) data collection nationally. METHODS Surveys were conducted of the following 3 groups: 1) all ED directors of Canadian hospitals located in communities of >10,000 people, 2) all electronic EDIS vendors, and 3) representatives from the ministries of health from 13 provincial and territorial jurisdictions who had knowledge of ED data collection. RESULTS Of the 243 ED directors contacted, 158 completed the survey (65% response rate) and 39% of those reported using an electronic EDIS. All 11 EDIS vendor representatives responded. Most of the vendors provide a similar package of basic EDIS options, with add-on features. All 13 provincial or territorial government representatives completed the survey. Nine (69%) provinces and territories collect ED data, however the source of this information varies. Five provinces and territories collect triage data, and 3 have a comprehensive, jurisdiction-wide, population-based ED database. Thirty-nine percent of EDs in larger Canadian communities track patients using electronic methods. A variety of EDIS vendor options are available and used in Canada. CONCLUSION The wide variation in methods and in data collected presents serious barriers to meaningful comparison of ED services across the country. It is little wonder that the majority of information regarding ED overcrowding in Canada is anecdotal, when the collection of this critical health information is so variable. There is an urgent need to place the collection of ED information on the provincial and national agenda and to ensure that the collection of this information consistent, comprehensive and mandatory.

Methods for Developing Evidence Reviews in Short Periods of Time: A Scoping Review.

Introduction Rapid reviews (RR), using abbreviated systematic review (SR) methods, are becoming more popular among decision-makers. This World Health Organization commissioned study sought to summarize RR methods, identify differences, and highlight potential biases between RR and SR. Methods Review of RR methods (Key Question 1 [KQ1]), meta-epidemiologic studies comparing reliability/ validity of RR and SR methods (KQ2), and their potential associated biases (KQ3). We searched Medline, EMBASE, Cochrane Library, grey literature, and checked reference lists, used personal contacts, and crowdsourcing (e.g. email listservs). Selection and data extraction was conducted by one reviewer (KQ1) or two reviewers independently (KQ2-3). Results Across all KQs, we identified 42,743 citations through the literature searches. KQ1: RR methods from 29 organizations were reviewed. There was no consensus on which aspects of the SR process to abbreviate. KQ2: Studies comparing the conclusions of RR and SR (n = 9) found them to be generally similar. Where major differences were identified, it was attributed to the inclusion of evidence from different sources (e.g. searching different databases or including different study designs). KQ3: Potential biases introduced into the review process were well-identified although not necessarily supported by empirical evidence, and focused mainly on selective outcome reporting and publication biases. Conclusion RR approaches are context and organization specific. Existing comparative evidence has found similar conclusions derived from RR and SR, but there is a lack of evidence comparing the potential of bias in both evidence synthesis approaches. Further research and decision aids are needed to help decision makers and reviewers balance the benefits of providing timely evidence with the potential for biased findings.