Ian Shemilt

GRADE guidelines: 10. Considering resource use and rating the quality of economic evidence

OBJECTIVES In this article, we describe how to include considerations about resource utilization when making recommendations according to the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach. STUDY DESIGN AND SETTINGS We focus on challenges with rating the confidence in effect estimates (quality of evidence) and incorporating resource use into evidence profiles and Summary of Findings (SoF) tables. RESULTS GRADE recommends that important differences in resource use between alternative management strategies should be included along with other important outcomes in the evidence profile and SoF table. Key steps in considering resources in making recommendations with GRADE are the identification of items of resource use that may differ between alternative management strategies and that are potentially important to decision makers, finding evidence for the differences in resource use, making judgments regarding confidence in effect estimates using the same criteria used for health outcomes, and valuing the resource use in terms of costs for the specific setting for which recommendations are being made. CONCLUSIONS With our framework, decision makers will have access to concise summaries of recommendations, including ratings of the quality of economic evidence, and better understand the implications for clinical decision making.

Altering micro-environments to change population health behaviour: towards an evidence base for choice architecture interventions

BackgroundThe idea that behaviour can be influenced at population level by altering the environments within which people make choices (choice architecture) has gained traction in policy circles. However, empirical evidence to support this idea is limited, especially its application to changing health behaviour. We propose an evidence-based definition and typology of choice architecture interventions that have been implemented within small-scale micro-environments and evaluated for their effects on four key sets of health behaviours: diet, physical activity, alcohol and tobacco use.DiscussionWe argue that the limitations of the evidence base are due not simply to an absence of evidence, but also to a prior lack of definitional and conceptual clarity concerning applications of choice architecture to public health intervention. This has hampered the potential for systematic assessment of existing evidence. By seeking to address this issue, we demonstrate how our definition and typology have enabled systematic identification and preliminary mapping of a large body of available evidence for the effects of choice architecture interventions. We discuss key implications for further primary research, evidence synthesis and conceptual development to support the design and evaluation of such interventions.SummaryThis conceptual groundwork provides a foundation for future research to investigate the effectiveness of choice architecture interventions within micro-environments for changing health behaviour. The approach we used may also serve as a template for mapping other under-explored fields of enquiry.

A web-based tool for adjusting costs to a specific target currency and price year

Objective: To develop a web-based tool to automate the adjustment of estimates of costs drawn from previously published or unpublished studies to a specified target currency and price year. Methods: A web-based tool was programmed using C#, utilising GDP deflator index values and Purchasing Power Parities conversion rates produced by the International Monetary Fund and the Organisation for Economic Co-operation and Development. Results: Version 1.0 is available at http://eppi.ioe.ac.uk/costconversion/default.aspx Conclusions: The tool can be used as a first-line approach to cost adjustment in non-healthcare applications and as an optional approach in healthcare applications when use of more sophisticated methods is not feasible.

Pinpointing needles in giant haystacks: use of text mining to reduce impractical screening workload in extremely large scoping reviews

In scoping reviews, boundaries of relevant evidence may be initially fuzzy, with refined conceptual understanding of interventions and their proposed mechanisms of action an intended output of the scoping process rather than its starting point. Electronic searches are therefore sensitive, often retrieving very large record sets that are impractical to screen in their entirety. This paper describes methods for applying and evaluating the use of text mining (TM) technologies to reduce impractical screening workload in reviews, using examples of two extremely large-scale scoping reviews of public health evidence (choice architecture (CA) and economic environment (EE)). Electronic searches retrieved >800,000 (CA) and >1 million (EE) records. TM technologies were used to prioritise records for manual screening. TM performance was measured prospectively. TM reduced manual screening workload by 90% (CA) and 88% (EE) compared with conventional screening (absolute reductions of ≈430 000 (CA) and ≈378 000 (EE) records). This study expands an emerging corpus of empirical evidence for the use of TM to expedite study selection in reviews. By reducing screening workload to manageable levels, TM made it possible to assemble and configure large, complex evidence bases that crossed research discipline boundaries. These methods are transferable to other scoping and systematic reviews incorporating conceptual development or explanatory dimensions.

Investigating complexity in systematic reviews of interventions by using a spectrum of methods

Systematic reviews framed by PICOS (Populations, Interventions, Comparisons, Outcomes, and Study designs) have been valuable for synthesizing evidence about the effects of interventions. However, this framework is limited in its utility for exploring the influence of variations within populations or interventions, or about the mechanisms of action or causal pathways thought to mediate outcomes, other contextual factors that might similarly moderate outcomes, or how and when these mechanisms and elements interact. Valuable insights into these issues come from configurative as well as aggregative methods of synthesis. This article considers the range of evidence that can be used in systematic reviews of interventions to investigate complexity in terms of potential sources of variation in interventions and their effects, and presents a continuum of purposes for, and approaches to, evidence synthesis. Choosing an appropriate synthesis method takes into account whether the purpose of the synthesis is to generate, explore, or test theories. Taking complexity into account in a synthesis of economic evidence similarly shifts emphasis from evidence synthesis strategies focused on aggregation toward configurative strategies that aim to develop, explore, and refine (in advance of testing) theories or explanations of how and why interventions are more or less resource intensive, costly or cost-effective in different settings, or when implemented in different ways.

Socioeconomic differences in purchases of more vs. less healthy foods and beverages: analysis of over 25,000 British households in 2010.

Socioeconomic inequalities in diet-related health outcomes are well-recognised, but are not fully explained by observational studies of consumption. We provide a novel analysis to identify purchasing patterns more precisely, based on data for take-home food and beverage purchases from 25,674 British households in 2010. To examine socioeconomic differences (measured by occupation), we conducted regression analyses on the proportion of energy purchased from (a) each of 43 food or beverage categories and (b) major nutrients. Results showed numerous small category-level socioeconomic differences. Aggregation of the categories showed lower SES groups generally purchased a greater proportion of energy from less healthy foods and beverages than those in higher SES groups (65% and 60%, respectively), while higher SES groups purchased a greater proportion of energy from healthier food and beverages (28% vs. 24%). At the nutrient-level, socioeconomic differences were less marked, although higher SES was associated with purchasing greater proportions of fibre, protein and total sugars, and smaller proportions of sodium. The observed pattern of purchasing across SES groups contributes to the explanation of observed health differences between groups and highlights targets for interventions to reduce health inequalities.

Evidence-Based Decisions and Economics:Health Care, Social Welfare, Education and Criminal Justice

Preface 1. From effectiveness to efficiency? An introduction toevidence-based decisions and economics for health care, socialwelfare, education and criminal justice (Miranda Mugford, IanShemilt, Luke Vale, Kevin Marsh, Cam Donaldson, JacquelineMallender). 2. The role of review and synthesis methods in decision models(Kevin Marsh). 3. The role of economic perspectives and evidence in systematicreview (Rob Anderson, Ian Shemilt). 4. The role of economic evidence in formulation of public policyand practice (Sarah Byford, Barbara Barrett, Richard Dubourg,Jennifer Francis, Jane Sisk). 5. Generalisability, transferability, complexity and relevance(Damian G Walker, Yot Teerawattananon, Rob Anderson, GerryRichardson). 6. Equity, efficiency and research synthesis (David McDaid,Franco Sassi). 7.Searching for evidence for cost-effectiveness decisions(Julie Glanville, Suzy Paisley). 8. Identifying and reviewing health state utility values forpopulating decision models (John Brazier, Diana Papaioannou,Anna Cantrell, Suzy Paisley, Kirsten Herrmann). 9. Use of evidence in decision models (Doug Coyle, Karen MLee, Nicola J Cooper). 10. Grading economic evidence (Massimo Brunetti, FrancisRuiz, Joanne Lord, Silvia Pregno, Andrew D Oxman). 11. Meta-regression models of economics and medical research(TD Stanley). 12. From evidence-based economics to economics-based evidence:using systematic review to inform the design of future research(Ed Wilson, Keith Abrams). 13. Complex problems or simple solutions? Enhancingevidence-based economics to reflect reality (Chantale Lessard,Stephen Birch). 14. Evidence-based decisions and economics: lessons for practice(Luke Vale). 15. Evidence-based decisions and economics: an agenda forresearch (Michael Drummond). 16. Glossary (Asmaa Abdelhamid, Ian Shemilt). Index.

Inclusion of quasi-experimental studies in systematic reviews of health systems research

Systematic reviews of health systems research commonly limit studies for evidence synthesis to randomized controlled trials. However, well-conducted quasi-experimental studies can provide strong evidence for causal inference. With this article, we aim to stimulate and inform discussions on including quasi-experiments in systematic reviews of health systems research. We define quasi-experimental studies as those that estimate causal effect sizes using exogenous variation in the exposure of interest that is not directly controlled by the researcher. We incorporate this definition into a non-hierarchical three-class taxonomy of study designs - experiments, quasi-experiments, and non-experiments. Based on a review of practice in three disciplines related to health systems research (epidemiology, economics, and political science), we discuss five commonly used study designs that fit our definition of quasi-experiments: natural experiments, instrumental variable analyses, regression discontinuity analyses, interrupted times series studies, and difference studies including controlled before-and-after designs, difference-in-difference designs and fixed effects analyses of panel data. We further review current practices regarding quasi-experimental studies in three non-health fields that utilize systematic reviews (education, development, and environment studies) to inform the design of approaches for synthesizing quasi-experimental evidence in health systems research. Ultimately, the aim of any review is practical: to provide useful information for policymakers, practitioners, and researchers. Future work should focus on building a consensus among users and producers of systematic reviews regarding the inclusion of quasi-experiments.

Downsizing: policy options to reduce portion sizes to help tackle obesity.

Larger portions of food increase consumption. Theresa Marteau and colleagues suggest ways to reduce their size, availability, and appeal

Economic Instruments for Population Diet and Physical Activity Behaviour Change: A Systematic Scoping Review

Background Unhealthy diet and low levels of physical activity are common behavioural factors in the aetiology of many non-communicable diseases. Recent years have witnessed an upsurge of policy and research interest in the use of taxes and other economic instruments to improve population health. Objective To assemble, configure and analyse empirical research studies available to inform the public health case for using economic instruments to promote dietary and physical activity behaviour change. Methods We conducted a systematic scoping review of evidence for the effects of specific interventions to change, or general exposure to variations in, prices or income on dietary and physical activity behaviours and corollary outcomes. Systematic electronic searches and parallel snowball searches retrieved >1 million study records. Text mining technologies were used to prioritise title-abstract records for screening. Eligible studies were selected, classified and analysed in terms of key characteristics and principal findings, using a narrative, configuring synthesis focused on implications for policy and further research. Results We identified 880 eligible studies, including 192 intervention studies and 768 studies that incorporated evidence for prices or income as correlates or determinants of target outcomes. Current evidence for the effects of economic instruments and exposures on diet and physical activity is limited in quality and equivocal in terms of its policy implications. Direct evidence for the effects of economic instruments is heavily skewed towards impacts on diet, with a relative lack of evidence for impacts on physical activity. Conclusions The evidence-based case for using economic instruments to promote dietary and physical activity behaviour change may be less compelling than some proponents have claimed. Future research should include measurement of people’s actual behavioural responses using study designs capable of generating reliable causal inferences regarding intervention effects. Policy implementation needs to be carefully aligned with evaluation planning and design.