Su Golder

Good Practice Guidelines for Decision-Analytic Modelling in Health Technology Assessment A Review and Consolidation of Quality Assessment

The use of decision-analytic modelling for the purpose of health technology assessment (HTA) has increased dramatically in recent years. Several guidelines for best practice have emerged in the literature; however, there is no agreed standard for what constitutes a ‘good model’ or how models should be formally assessed. The objective of this paper is to identify, review and consolidate existing guidelines on the use of decision-analytic modelling for the purpose of HTA and to develop a consistent framework against which the quality of models may be assessed.The review and resultant framework are summarised under the three key themes of Structure, Data and Consistency. ‘Structural’ aspects relate to the scope and mathematical structure of the model including the strategies under evaluation. Issues covered under the general heading of ‘Data’ include data identification methods and how uncertainty should be addressed. ‘Consistency’ relates to the overall quality of the model.The review of existing guidelines showed that although authors may provide a consistent message regarding some aspects of modelling, such as the need for transparency, they are contradictory in other areas. Particular areas of disagreement are how data should be incorporated into models and how uncertainty should be assessed.For the purpose of evaluation, the resultant framework is applied to a decision-analytic model developed as part of an appraisal for the National Institute for Health and Clinical Excellence (NICE) in the UK. As a further assessment, the review based on the framework is compared with an assessment provided by an independent experienced modeller not using the framework.It is hoped that the framework developed here may form part of the appraisals process for assessment bodies such as NICE and decision models submitted to peer review journals. However, given the speed with which decision-modelling methodology advances, there is a need for its continual update.

Meta-analyses of adverse effects data derived from randomised controlled trials as compared to observational studies: methodological overview

Su Golder and colleagues carry out an overview of meta-analyses to assess whether estimates of the risk of harm outcomes differ between randomized trials and observational studies. They find that, on average, there is no difference in the estimates of risk between overviews of observational studies and overviews of randomized trials.

Adverse drug reactions in children--a systematic review.

Background Adverse drug reactions in children are an important public health problem. We have undertaken a systematic review of observational studies in children in three settings: causing admission to hospital, occurring during hospital stay and occurring in the community. We were particularly interested in understanding how ADRs might be better detected, assessed and avoided. Methods and Findings We searched nineteen electronic databases using a comprehensive search strategy. In total, 102 studies were included. The primary outcome was any clinical event described as an adverse drug reaction to one or more drugs. Additional information relating to the ADR was collected: associated drug classification; clinical presentation; associated risk factors; methods used for assessing causality, severity, and avoidability. Seventy one percent (72/102) of studies assessed causality, and thirty four percent (34/102) performed a severity assessment. Only nineteen studies (19%) assessed avoidability. Incidence rates for ADRs causing hospital admission ranged from 0.4% to 10.3% of all children (pooled estimate of 2.9% (2.6%, 3.1%)) and from 0.6% to 16.8% of all children exposed to a drug during hospital stay. Anti-infectives and anti-epileptics were the most frequently reported therapeutic class associated with ADRs in children admitted to hospital (17 studies; 12 studies respectively) and children in hospital (24 studies; 14 studies respectively), while anti-infectives and non-steroidal anti-inflammatory drugs (NSAIDs) were frequently reported as associated with ADRs in outpatient children (13 studies; 6 studies respectively). Fourteen studies reported rates ranging from 7%–98% of ADRs being either definitely/possibly avoidable. Conclusions There is extensive literature which investigates ADRs in children. Although these studies provide estimates of incidence in different settings and some indication of the therapeutic classes most frequently associated with ADRs, further work is needed to address how such ADRs may be prevented.

Systematic review of antimicrobial treatments for diabetic foot ulcers

Background  Foot ulcers in diabetes are associated with increased mortality, illness and reduced quality of life. Ulcer infection impairs healing and antimicrobial interventions may cure infection, aid healing and reduce amputation rates.

Quality of reporting in systematic reviews of adverse events: systematic review

Objectives To examine the quality of reporting of harms in systematic reviews, and to determine the need for a reporting guideline specific for reviews of harms. Design Systematic review. Data sources Cochrane Database of Systematic Reviews (CDSR) and Database of Abstracts of Reviews of Effects (DARE). Review methods Databases were searched for systematic reviews having an adverse event as the main outcome, published from January 2008 to April 2011. Adverse events included an adverse reaction, harms, or complications associated with any healthcare intervention. Articles with a primary aim to investigate the complete safety profile of an intervention were also included. We developed a list of 37 items to measure the quality of reporting on harms in each review; data were collected as dichotomous outcomes (“yes” or “no” for each item). Results Of 4644 reviews identified, 309 were systematic reviews or meta-analyses primarily assessing harms (13 from CDSR; 296 from DARE). Despite a short time interval, the comparison between the years of 2008 and 2010-11 showed no difference on the quality of reporting over time (P=0.079). Titles in fewer than half the reviews (proportion of reviews 0.46 (95% confidence interval 0.40 to 0.52)) did not mention any harm related terms. Almost one third of DARE reviews (0.26 (0.22 to 0.31)) did not clearly define the adverse events reviewed, nor did they specify the study designs selected for inclusion in their methods section. Almost half of reviews (n=170) did not consider patient risk factors or length of follow-up when reviewing harms of an intervention. Of 67 reviews of complications related to surgery or other procedures, only four (0.05 (0.01 to 0.14)) reported professional qualifications of the individuals involved. The overall, unweighted, proportion of reviews with good reporting was 0.56 (0.55 to 0.57); corresponding proportions were 0.55 (0.53 to 0.57) in 2008, 0.55 (0.54 to 0.57) in 2009, and 0.57 (0.55 to 0.58) in 2010-11. Conclusion Systematic reviews compound the poor reporting of harms data in primary studies by failing to report on harms or doing so inadequately. Improving reporting of adverse events in systematic reviews is an important step towards a balanced assessment of an intervention.

The Effectiveness and Cost‐Effectiveness of Respite for Caregivers of Frail Older People

The proportion of frail elderly people in the industrialized world is increasing. Respite care is a potentially important way of maintaining the quality of life for these people and their caregivers. The objective of this systematic review was to determine the effectiveness and cost‐effectiveness of different models of community‐based respite care for frail older people and their caregivers. To identify relevant studies, 37 databases were searched, and reference checking and citation searches were undertaken. Well‐controlled effectiveness studies were eligible for inclusion, with uncontrolled studies admissible only in the absence of higher‐quality evidence. Studies assessed the effect of community‐based respite on caregivers of frail elderly people relative to usual care or to another support intervention. Eligible economic evaluations also addressed costs. Where appropriate, data were synthesized using standard meta‐analytic techniques. Ten randomized, controlled trials, seven quasi‐experimental studies and five uncontrolled studies were included in the review. For all types of respite, the effects upon caregivers were generally small, with better‐controlled studies finding modest benefits only for certain subgroups, although many studies reported high levels of caregiver satisfaction. No reliable evidence was found that respite care delays entry to residential care or adversely affects frail older people. The economic evaluations all assessed day care, which tended to be associated with similar or higher costs than usual care. Given the increasing numbers of frail elderly people and the lack of up‐to‐date, good‐quality evidence for all types of respite care, better‐quality evidence is urgently needed to inform current policy and practice.

PRISMA harms checklist: Improving harms reporting in systematic reviews

Introduction For any health intervention, accurate knowledge of both benefits and harms is needed. Systematic reviews often compound poor reporting of harms in primary studies by failing to report harms or doing so inadequately. While the PRISMA statement (Preferred Reporting Items for Systematic reviews and Meta-Analyses) helps systematic review authors ensure complete and transparent reporting, it is focused mainly on efficacy. Thus, a PRISMA harms checklist has been developed to improve harms reporting in systematic reviews, promoting a more balanced assessment of benefits and harms. Methods A development strategy, endorsed by the EQUATOR Network and existing reporting guidelines (including the PRISMA statement, PRISMA for abstracts, and PRISMA for protocols), was used. After the development of a draft checklist of items, a modified Delphi process was initiated. The Delphi consisted of three rounds of electronic feedback followed by an in-person meeting. Results The PRISMA harms checklist contains four essential reporting elements to be added to the original PRISMA statement to improve harms reporting in reviews. These are reported in the title (“Specifically mention ‘harms’ or other related terms, or the harm of interest in the review”), synthesis of results (“Specify how zero events were handled, if relevant”), study characteristics (“Define each harm addressed, how it was ascertained (eg, patient report, active search), and over what time period”), and synthesis of results (“Describe any assessment of possible causality”). Additional guidance regarding existing PRISMA items was developed to demonstrate relevance when synthesising information about harms. Conclusion The PRISMA harms checklist identifies a minimal set of items to be reported when reviewing adverse events. This guideline extension is intended to improve harms reporting in systematic reviews, whether harms are a primary or secondary outcome.

Scoping Review and Approach to Appraisal of Interventions Intended to Involve Patients in Patient Safety

Objective To review the literature on the involvement of patients in efforts to promote their own or others’ safety while using health care services. Method A total of 1933 reports were identified as potentially relevant and 745 of these were included in the review (437 descriptions of interventions, 299 comment or opinion pieces and 42 discussions or studies of patients’ willingness and ability to adopt safety-promoting behaviours). Results The rate of publication on these topics has increased, especially in the USA and UK. However, there is scant evidence of the impact of patient involvement initiatives on safety outcomes and there has been little exploration of patients’ willingness and ability to adopt particular safety-oriented behaviours. We identified three broad routes by which patients’ actions might contribute to their safety by helping to make sure that: their treatment is appropriate for them (informing the management plan); treatment is given as planned and according to appropriate protocols (monitoring and ensuring safe delivery of treatment); and problems and risks within health care systems are identified and reduced (informing systems improvements). Conclusions An approach for appraising interventions intended to promote patient involvement in patient safety should involve: identification of the routes by which interventions assume patients’ actions might contribute to their safety; identification of the conditions that would need to be met for patients to behave and contribute as the interventions (implicitly) assume; examination of the extent to which the intervention supports fulfilment of those conditions; and consideration of the potential negative effects of the intervention.

Interventions for people bereaved through suicide: systematic review

BACKGROUND Promoting the mental health of people bereaved through suicide is a key aim of the National Suicide Prevention Strategy. AIMS To evaluate the effects of interventions to support people bereaved through suicide. METHOD We conducted a systematic review of data from controlled studies of interventions for people bereaved through suicide. Studies were identified using systematic searches, the methodological quality of included studies was assessed and narrative synthesis conducted. RESULTS Eight studies were identified. None was UK-based and all but one study had substantial methodological limitations. When compared with no intervention, there was evidence of some benefit from single studies of a cognitive-behavioural family intervention of four sessions with a psychiatric nurse; a psychologist-led 10-week bereavement group intervention for children; and 8-week group therapy for adults delivered by a mental health professional and volunteer. The findings from studies comparing two or more active interventions were more equivocal. CONCLUSIONS Although there is evidence of some benefit from interventions for people bereaved by suicide, this is not robust. Further methodologically sound evidence is required to confirm whether interventions are helpful and, if so, for whom.

Poor reporting and inadequate searches were apparent in systematic reviews of adverse effects

OBJECTIVE Systematic reviews incorporating adverse effects are assuming increasing importance as questions raised extend beyond clinical effectiveness to all effects (beneficial and harmful). The aim of this study was to survey the methods used to identify relevant studies for systematic reviews of adverse effects. STUDY DESIGN AND SETTING All records within the Database of Abstracts of Reviews of Effects and the Cochrane Database of Systematic Reviews were scanned for systematic reviews in which the primary outcomes were adverse effects. Two information professionals independently assessed the methods used to identify relevant research as reported in the 277 reviews that met the inclusion criteria. RESULTS A major weakness of the reviews was inadequate reporting of the search strategies used. In addition, of the reviews that did report a search strategy, few used the sensitive search strategies recommended for systematic reviews. The majority of reviews did not search more than one or two databases, and few other methods of identifying information were used. CONCLUSION This investigation shows the variation in the searching element of systematic reviews of adverse effects and demonstrates that the reporting of the methods used to identify research in such reviews could be vastly improved.