Tobias Dreischulte

Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting

BackgroundProcess evaluations are recommended to open the ‘black box’ of complex interventions evaluated in trials, but there is limited guidance to help researchers design process evaluations. Much current literature on process evaluations of complex interventions focuses on qualitative methods, with less attention paid to quantitative methods. This discrepancy led us to develop our own framework for designing process evaluations of cluster-randomised controlled trials.MethodsWe reviewed recent theoretical and methodological literature and selected published process evaluations; these publications identified a need for structure to help design process evaluations. We drew upon this literature to develop a framework through iterative exchanges, and tested this against published evaluations.ResultsThe developed framework presents a range of candidate approaches to understanding trial delivery, intervention implementation and the responses of targeted participants. We believe this framework will be useful to others designing process evaluations of complex intervention trials. We also propose key information that process evaluations could report to facilitate their identification and enhance their usefulness.ConclusionThere is no single best way to design and carry out a process evaluation. Researchers will be faced with choices about what questions to focus on and which methods to use. The most appropriate design depends on the purpose of the process evaluation; the framework aims to help researchers make explicit their choices of research questions and methods.Trial registrationClinicaltrials.gov NCT01425502

High risk prescribing in primary care patients particularly vulnerable to adverse drug events: cross sectional population database analysis in Scottish general practice.

Objective To examine the prevalence and patterns of high risk prescribing, defined as potentially inappropriate prescribing of drugs to primary care patients particularly vulnerable to adverse drug events. Design Cross sectional population database analysis. Setting General practices in Scotland. Participants 315 Scottish general practices with 1.76 million registered patients, 139 404 (7.9%) of whom were defined as particularly vulnerable to adverse drug events because of age, comorbidity, or co-prescription. Main outcome measures How reliably each of 15 indicators—four each for non-steroidal anti-inflammatory drugs, co-prescription with warfarin, and prescribing in heart failure, two for dose instructions for methotrexate, and one for antipsychotic prescribing in dementia—and a composite of all 15 could distinguish practices in terms of their rates of high risk prescribing; and characteristics of patients and practices associated with high risk prescribing in a multilevel model. Results 19 308 of 139 404 (13.9%, 95% confidence interval 13.7% to 14.0%) patients had received at least one high risk prescription in the past year. This composite indicator was a reasonably reliable measure of practice rates of high risk prescribing (reliability >0.7 for 95.6% of practices, >0.8 for 88.2%). The patient characteristic most strongly associated with high risk prescribing was the number of drugs prescribed (>11 long term prescribed drugs v 0; odds ratio 7.90, 95% confidence interval 7.19 to 8.68). After adjustment for patient characteristics, rates of high risk prescribing varied by fourfold between practices, which was not explained by structural characteristics of the practices. Conclusions Almost 14% of patients defined as particularly vulnerable to adverse drug events were prescribed one or more high risk drugs. The composite indicator of high risk prescribing used could identify practices as having above average or below average high risk prescribing rates with reasonable confidence. After adjustment, only the number of drugs prescribed long term to patients was strongly associated with high risk prescribing, and considerable unexplained variation existed between practices. High risk prescribing will often be appropriate, but the large variation between practices suggests opportunities for improvement.

Combined use of nonsteroidal anti-inflammatory drugs with diuretics and/or renin–angiotensin system inhibitors in the community increases the risk of acute kidney injury

Nonsteroidal anti-inflammatory drugs (NSAIDs) are associated with an increased risk of acute kidney injury (AKI) when used in triple combination with renin-angiotensin system inhibitors and diuretics, but previous research reported that NSAIDs in dual combinations with either renin-angiotensin system inhibitors or diuretics alone were not. However, earlier studies relied on hospital coding to define AKI, which may underestimate true risk. This nested case-control study characterized the risk of community-acquired AKI associated with NSAID use among 78,379 users of renin-angiotensin system inhibitors and/or diuretics, where AKI was defined as a 50% or greater increase in creatinine from baseline. The AKI incidence was 68/10,000 person-years. The relative increase in AKI risk was similar for NSAID use in both triple (adjusted rate ratio 1.64 (95% CI 1.25-2.14)) and dual combinations with either renin-angiotensin system inhibitors (1.60 (1.18-2.17)) or diuretics (1.64 (1.17-2.29)). However, the absolute increase in AKI risk was higher for NSAIDs used in triple versus dual combinations with renin-angiotensin system inhibitors or diuretics alone (numbers needed to harm for 1 year treatment with NSAID of 158 vs. over 300). AKI risk was highest among users of loop diuretic/aldosterone antagonist combinations, in those over 75 years of age, and in those with renal impairment. Thus, the nephrotoxic potential of both dual and triple combinations of NSAIDs with renin-angiotensin system inhibitors and/or diuretics yields a higher incidence of AKI than previously thought.

Clinical InvestigationCombined use of nonsteroidal anti-inflammatory drugs with diuretics and/or renin–angiotensin system inhibitors in the community increases the risk of acute kidney injury

Nonsteroidal anti-inflammatory drugs (NSAIDs) are associated with an increased risk of acute kidney injury (AKI) when used in triple combination with renin-angiotensin system inhibitors and diuretics, but previous research reported that NSAIDs in dual combinations with either renin-angiotensin system inhibitors or diuretics alone were not. However, earlier studies relied on hospital coding to define AKI, which may underestimate true risk. This nested case-control study characterized the risk of community-acquired AKI associated with NSAID use among 78,379 users of renin-angiotensin system inhibitors and/or diuretics, where AKI was defined as a 50% or greater increase in creatinine from baseline. The AKI incidence was 68/10,000 person-years. The relative increase in AKI risk was similar for NSAID use in both triple (adjusted rate ratio 1.64 (95% CI 1.25-2.14)) and dual combinations with either renin-angiotensin system inhibitors (1.60 (1.18-2.17)) or diuretics (1.64 (1.17-2.29)). However, the absolute increase in AKI risk was higher for NSAIDs used in triple versus dual combinations with renin-angiotensin system inhibitors or diuretics alone (numbers needed to harm for 1 year treatment with NSAID of 158 vs. over 300). AKI risk was highest among users of loop diuretic/aldosterone antagonist combinations, in those over 75 years of age, and in those with renal impairment. Thus, the nephrotoxic potential of both dual and triple combinations of NSAIDs with renin-angiotensin system inhibitors and/or diuretics yields a higher incidence of AKI than previously thought.

Quality in primary care

Individual doctors matter, and pay for performance may reduce hospital admissions

A cluster randomised stepped wedge trial to evaluate the effectiveness of a multifaceted information technology-based intervention in reducing high-risk prescribing of non-steroidal anti-inflammatory drugs and antiplatelets in primary medical care: The DQIP study protocol

BackgroundHigh-risk prescribing of non-steroidal anti-inflammatory drugs (NSAIDs) and antiplatelet agents accounts for a significant proportion of hospital admissions due to preventable adverse drug events. The recently completed PINCER trial has demonstrated that a one-off pharmacist-led information technology (IT)-based intervention can significantly reduce high-risk prescribing in primary care, but there is evidence that effects decrease over time and employing additional pharmacists to facilitate change may not be sustainable.Methods/designWe will conduct a cluster randomised controlled with a stepped wedge design in 40 volunteer general practices in two Scottish health boards. Eligible practices are those that are using the INPS Vision clinical IT system, and have agreed to have relevant medication-related data to be automatically extracted from their electronic medical records. All practices (clusters) that agree to take part will receive the data-driven quality improvement in primary care (DQIP) intervention, but will be randomised to one of 10 start dates. The DQIP intervention has three components: a web-based informatics tool that provides weekly updated feedback of targeted prescribing at practice level, prompts the review of individual patients affected, and summarises each patients relevant risk factors and prescribing; an outreach visit providing education on targeted prescribing and training in the use of the informatics tool; and a fixed payment of 350 GBP (560 USD; 403 EUR) up front and a small payment of 15 GBP (24 USD; 17 EUR) for each patient reviewed in the 12 months of the intervention. We hypothesise that the DQIP intervention will reduce a composite of nine previously validated measures of high-risk prescribing. Due to the nature of the intervention, it is not possible to blind practices, the core research team, or the data analyst. However, outcome assessment is entirely objective and automated. There will additionally be a process and economic evaluation alongside the main trial.DiscussionThe DQIP intervention is an example of a potentially sustainable safety improvement intervention that builds on the existing National Health Service IT-infrastructure to facilitate systematic management of high-risk prescribing by existing practice staff. Although the focus in this trial is on Non-steroidal anti-inflammatory drugs and antiplatelets, we anticipate that the tested intervention would be generalisable to other types of prescribing if shown to be effective.Trial registrationClinicalTrials.gov, dossier number: NCT01425502

Logistic, ethical, and political dimensions of stepped wedge trials: critical review and case studies

BackgroundThree arguments are usually invoked in favour of stepped wedge cluster randomised controlled trials: the logistic convenience of implementing an intervention in phases, the ethical benefit of providing the intervention to all clusters, and the potential to enhance the social acceptability of cluster randomised controlled trials. Are these alleged benefits real? We explored the logistic, ethical, and political dimensions of stepped wedge trials using case studies of six recent evaluations.MethodsWe identified completed or ongoing stepped wedge evaluations using two systematic reviews. We then purposively selected six with a focus on public health in high, middle, and low-income settings. We interviewed their authors about the logistic, ethical, and social issues faced by their teams. Two authors reviewed interview transcripts, identified emerging issues through qualitative thematic analysis, reflected upon them in the context of the literature, and invited all participants to co-author the manuscript.ResultsOur analysis raises three main points. First, the phased implementation of interventions can alleviate problems linked to simultaneous roll-out, but also brings new challenges. Issues to consider include the feasibility of organising intervention activities according to a randomised sequence, estimating time lags in implementation and effects, and accommodating policy changes during the trial period. Second, stepped wedge trials, like parallel cluster trials, require equipoise: without it, randomising participants to a control condition, even for a short time, remains problematic. In stepped wedge trials, equipoise is likely to lie in the degree of effect, effectiveness in a specific operational milieu, and the balance of benefit and harm, including the social value of better evaluation. Third, the strongest arguments for a stepped wedge design are logistic and political rather than ethical. The design is advantageous when simultaneous roll-out is impractical and when it increases the acceptability of using counterfactuals.ConclusionsThe logistic convenience of phased implementation is context-dependent, and may be vitiated by the additional requirements of phasing. The potential for stepped wedge trials to enhance the social acceptability of cluster randomised trials is real, but their ethical legitimacy still rests on demonstrating equipoise and its configuration for each research question and setting.

Pro’s and con’s of the stepped wedge design in cluster randomised trials of quality improvement interventions: two current examples

The stepped wedge design, under which all trial participants receive the intervention but the order in which the intervention is received is randomised, is potentially useful to rigorously evaluate organisational interventions to improve quality and safety. We use two examples of cluster-randomised stepped-wedge trials (DQIP and GP-POLY) to illustrate advantages and disadvantages of the design in evaluations of complex prescribing improvement interventions in primary care. DQIP is nearing completion and GP-POLY will start in 2013. The intervention in both DQIP and GP-Poly involves outreach visits by researchers for education and informatics tool training, making sequential roll out a logistic necessity. The stepped wedge allows for this by design, but trial durations may be prolonged compared to parallel-arm trials and other designs, and arranging initiation visits to fit with randomisation schedules is challenging. Since all participants receive the intervention and there are multiple repeated measurements, practice sample size requirements in DQIP and GP-POLY were reduced compared to a parallel-arm design, but power calculations are more complex. Recruitment may be improved by offering the intervention to all participants, but creates potential problems for retention and avoiding contamination in practices with long lags between recruitment and intervention start. Because of the vulnerability of stepped wedge trials to time varying confounding, avoiding changes in intervention delivery to successive cohorts is important and needs careful planning. The stepped wedge design is attractive for cluster randomised trials of quality improvement interventions, especially when staggering of intervention delivery is inevitable, but presents challenges for implementation that need careful planning. Oral presentation presented at the 2nd Clinical Trials Methodology Conference 2013: Methodology matters, 18-19 November 2013, Edinburgh, UK.

Developing a complex intervention to improve prescribing safety in primary care: mixed methods feasibility and optimisation pilot study

Objectives (A) To measure the extent to which different candidate outcome measures identified high-risk prescribing that is potentially changeable by the data-driven quality improvement in primary care (DQIP) intervention.(B) To explore the value of reviewing identified high-risk prescribing to clinicians.(C) To optimise the components of the DQIP intervention. Design Mixed method study. Setting General practices in two Scottish Health boards. Participants 4 purposively sampled general practices of varying size and socioeconomic deprivation. Outcome measures Prescribing measures targeting (1) high-risk use of the non-steroidal anti-inflammatory drugs (NSAIDs) and antiplatelets; (2) ‘Asthma control’ and (3) ‘Antithrombotics in atrial fibrillation (AF)’. Intervention The prescribing measures were used to identify patients for review by general practices. The ability of the measures to identify potentially changeable high-risk prescribing was measured as the proportion of patients reviewed where practices identified a need for action. Field notes were recorded from meetings between researchers and staff and key staff participated in semistructured interviews exploring their experience of the piloted intervention processes. Results Practices identified a need for action in 68%, 25% and 18% of patients reviewed for prescribing measures (1), (2) and (3), respectively. General practitioners valued being prompted to review patients, and perceived that (1) ‘NSAID and antiplatelet’ and (2) ‘antithrombotics in AF’ were the most important to act on. Barriers to initial and ongoing engagement and to sustaining improvements in prescribing were identified. Conclusions ‘NSAIDs and antiplatelets’ measures were selected as the most suitable outcome measures for the DQIP trial, based on evidence of this prescribing being more easily changeable. In response to the barriers identified, the intervention was designed to include a financial incentive, additional ongoing feedback on progress and reprompting review of patients, whose high-risk prescribing was restarted after a decision to stop. Trial registration number Clinicaltrials.gov NCT01425502.

Respiratory effect of beta-blocker eye drops in asthma: population-based study and meta-analysis of clinical trials.

AIMS To measure the prevalence of beta-blocker eye drop prescribing and respiratory effect of ocular beta-blocker administration in people with asthma. METHODS We measured the prevalence of ocular beta-blocker prescribing in people with asthma and ocular hypertension, and performed a nested case-control study (NCCS) measuring risk of moderate exacerbations (rescue steroids in primary care) and severe exacerbations (asthma hospitalization) using linked data from the UK Clinical Practice Research Datalink. We then performed a systematic review and meta-analysis of clinical trials evaluating changes in lung function following ocular beta-blocker administration in people with asthma. RESULTS From 2000 to 2012, the prevalence of non-selective and selective beta-blocker eye drop prescribing in people with asthma and ocular hypertension fell from 23.0% to 13.4% and from 10.5% to 0.9% respectively. In the NCCS, the relative incidence (IRR) of moderate exacerbations increased significantly with acute non-selective beta-blocker eye drop exposure (IRR 4.83, 95% CI 1.56-14.94) but not with chronic exposure. In the meta-analysis, acute non-selective beta-blocker eye drop exposure caused significant mean falls in FEV1 of -10.9% (95% CI -14.9 to -6.9), and falls in FEV1 of ≥20% affecting one in three. Corresponding values for selective beta-blockers in people sensitive to ocular non-selective beta-blockers was -6.3% (95% CI -11.7 to -0.8), and a non-significant increase in falls in FEV1 of ≥20%. CONCLUSION Non-selective beta-blocker eye drops significantly affect lung function and increase asthma morbidity but are still frequently prescribed to people with asthma and ocular hypertension despite safer agents being available.