Mhairi Campbell

Considering methodological options for reviews of theory: illustrated by a review of theories linking income and health

BackgroundReview of theory is an area of growing methodological advancement. Theoretical reviews are particularly useful where the literature is complex, multi-discipline, or contested. It has been suggested that adopting methods from systematic reviews may help address these challenges. However, the methodological approaches to reviews of theory, including the degree to which systematic review methods can be incorporated, have received little discussion in the literature. We recently employed systematic review methods in a review of theories about the causal relationship between income and health.MethodsThis article discusses some of the methodological issues we considered in developing the review and offers lessons learnt from our experiences. It examines the stages of a systematic review in relation to how they could be adapted for a review of theory. The issues arising and the approaches taken in the review of theories in income and health are considered, drawing on the approaches of other reviews of theory.ResultsDifferent approaches to searching were required, including electronic and manual searches, and electronic citation tracking to follow the development of theories. Determining inclusion criteria was an iterative process to ensure that inclusion criteria were specific enough to make the review practical and focused, but not so narrow that key literature was excluded. Involving subject specialists was valuable in the literature searches to ensure principal papers were identified and during the inductive approaches used in synthesis of theories to provide detailed understanding of how theories related to another. Reviews of theory are likely to involve iterations and inductive processes throughout, and some of the concepts and techniques that have been developed for qualitative evidence synthesis can be usefully translated to theoretical reviews of this kind.ConclusionsIt may be useful at the outset of a review of theory to consider whether the key aim of the review is to scope out theories relating to a particular issue; to conduct in-depth analysis of key theoretical works with the aim of developing new, overarching theories and interpretations; or to combine both these processes in the review. This can help decide the most appropriate methodological approach to take at particular stages of the review.

Lone parents, health, wellbeing and welfare to work: a systematic review of qualitative studies

BackgroundLone parents and their children experience higher than average levels of adverse health and social outcomes, much of which are explained by high rates of poverty. Many high income countries have attempted to address high poverty rates by introducing employment requirements for lone parents in receipt of welfare benefits. However, there is evidence that employment may not reduce poverty or improve the health of lone parents and their children.MethodsWe conducted a systematic review of qualitative studies reporting lone parents’ accounts of participation in welfare to work (WtW), to identify explanations and possible mechanisms for the impacts of WtW on health and wellbeing. Twenty one bibliographic databases were searched. Two reviewers independently screened references and assessed study quality. Studies from any high income country that met the criteria of focussing on lone parents, mandatory WtW interventions, and health or wellbeing were included. Thematic synthesis was used to investigate analytic themes between studies.ResultsScreening of the 4703 identified papers and quality assessment resulted in the inclusion of 16 qualitative studies of WtW in five high income countries, USA, Canada, UK, Australia, and New Zealand, covering a variety of welfare regimes. Our synthesis found that WtW requirements often conflicted with child care responsibilities. Available employment was often poorly paid and precarious. Adverse health impacts, such as increased stress, fatigue, and depression were commonly reported, though employment and appropriate training was linked to increased self-worth for some. WtW appeared to influence health through the pathways of conflict and control, analytical themes which emerged during synthesis. WtW reduced control over the nature of employment and care of children. Access to social support allowed some lone parents to manage the conflict associated with employment, and to increase control over their circumstances, with potentially beneficial health impacts.ConclusionWtW can result in increased conflict and reduced control, which may lead to negative impacts on mental health. Availability of social support may mediate the negative health impacts of WtW.

Autism in Glasgow: cumulative incidence and the effects of referral age, deprivation and geographical location

BACKGROUND Referrals to the Greater Glasgow Community Autism Team (CAT) made before the childs sixth birthday were analysed to obtain an estimation of the proportion of children in Greater Glasgow with childhood autism and investigate whether there were any variations in diagnosis rates, or in age at referral and diagnosis, depending on deprivation or geographical location. METHODS An analysis was made of the database recording referrals to Greater Glasgow CAT, between 2004 and 2007 inclusive, of children referred by age 6 years, comprising 584 cases. Cumulative incidence was calculated for childhood autism. Ages at referral and diagnosis were also analysed. RESULTS For this subset of children, there were 246 diagnosed cases of childhood autism, a cumulative incidence from 2004 until 2007 of 11.1 per year per 10,000 children aged 0-6 years. Of children with an eventual diagnosis of autism by age 6, 72% were referred by the age of 4 years. Deprivation was found to have an association with referral and diagnostic rates, with higher rates seen in the most deprived. There was geographical variation in the cumulative incidence of autism. CONCLUSION Given that the populations were not known to differ in any manner that would lead to a true variation, the geographical variation in the cumulative incidence of autism in children up to 6 years in Greater Glasgow observed in this study is likely to represent differences in the care pathway between areas. Such differences may also explain the observed association with deprivation. Reasons for the variation are being explored.

Reporting guidelines for population health and policy interventions: TIDieR-PHP

Abstract Background Inadequate descriptions of interventions in reports of research studies impede evidence synthesis, and result in unnecessary research duplication and suboptimum real-world implementation. The Template for Intervention Description and Replication (TIDieR) provides guidance for reporting interventions, and is primarily focussed on interventions delivered to individuals or small groups. We aimed to assess the need for additional reporting guidance for population health and policy (PHP) interventions and, if appropriate, develop an extension to TIDieR. Methods This study was the preliminary stage of an expert consensus consultation. We drew on our collective experience of evaluating and synthesising evidence of PHP interventions that fall within the scope of Cochrane Public Health to purposively select a diverse range of interventions and compare these with the original TIDieR checklist and to assess whether additional reporting items were needed. Interventions studied included tobacco and alcohol regulation, health service reorganisation, welfare system changes, conditional incentives for behaviour change, and environmental or infrastructural improvement programmes. Intervention descriptions from published evaluations were abstracted independently by four authors. Items that could be fitted into TIDieR were identified, and new categories were developed for those items that did not fit the existing template. A revised TIDieR-PHP reporting template was created through an iterative process of incorporating items into the draft template and testing them against additional interventions. Findings Six example intervention studies were selected for analysis, covering categories including incentive-based, regulatory, infrastructural, material benefits, systems and policy, and taxation and pricing interventions. Many population-level interventions have characteristics that do not fit well within the existing TIDieR framework, but affect intervention implementation and success. Attributes of PHP interventions not adequately captured include the policy and organisational setting of the intervention; political acceptability and relevant co-interventions; and key features of the dose involved (eg, for interventions offering transfer payments or imposing sanctions). Interpretation This preliminary stage of the development of TIDieR-PHP found that important features of population-level interventions might not be easily captured by existing guidelines. TIDieR-PHP aims to provide guidance for reporting population health and policy interventions so that learning from population health research is maximised. It will be further developed via a modified Delphi survey. Funding None.

Effectiveness of Mass Media Campaigns to Reduce Alcohol Consumption and Harm: A Systematic Review (Forthcoming/Available Online)

There is little evidence that mass media campaigns have reduced alcohol consumption although most did not state that they aimed to do so. Studies show recall of campaigns is high and that they can have an impact on knowledge, attitudes and beliefs about alcohol consumption.

Effectiveness of mass media campaigns to reduce alcohol consumption and harm: a systematic review

Aims: To assess the effectiveness of mass media messages to reduce alcohol consumption and related harms using a systematic literature review. Methods: Eight databases were searched along with reference lists of eligible studies. Studies of any design in any country were included, provided that they evaluated a mass media intervention targeting alcohol consumption or related behavioural, social cognitive or clinical outcomes. Drink driving interventions and college campus campaigns were ineligible. Studies quality were assessed, data were extracted and a narrative synthesis conducted. Results: Searches produced 10,212 results and 24 studies were included in the review. Most campaigns used TV or radio in combination with other media channels were conducted in developed countries and were of weak quality. There was little evidence of reductions in alcohol consumption associated with exposure to campaigns based on 13 studies which measured consumption, although most did not state this as a specific aim of the campaign. There were some increases in treatment seeking and information seeking and mixed evidence of changes in intentions, motivation, beliefs and attitudes about alcohol. Campaigns were associated with increases in knowledge about alcohol consumption, especially where levels had initially been low. Recall of campaigns was high. Conclusion: Mass media health campaigns about alcohol are often recalled by individuals, have achieved changes in knowledge, attitudes and beliefs about alcohol but there is little evidence of reductions in alcohol consumption. Short summary: There is little evidence that mass media campaigns have reduced alcohol consumption although most did not state that they aimed to do so. Studies show recall of campaigns is high and that they can have an impact on knowledge, attitudes and beliefs about alcohol consumption.

Reporting of narrative synthesis in systematic reviews of public health interventions: a methodological assessment

Abstract Background The value of systematic reviews for decision making is their transparency, reproducibility, and robust approach to assessing the evidence base. For reviews of public health interventions, quantitative data are often narratively synthesised. Narrative synthesis has been criticised for its lack of transparency, raising concerns about potential bias in many systematic reviews used in public health, and threatening their value for decision making. We systematically assessed the reporting and conduct of narrative synthesis in reviews assessing public health interventions. Methods We screened a 20% (474/2372) random sample of systematic reviews published between Jan 1, 2010, and Oct 31, 2015, from the McMaster Health Evidence database (a comprehensive database of systematic reviews of public health interventions), to identify reviews that had synthesised quantitative data narratively. From these reviews we investigated a 30% (n=75) random sample in greater detail. Informed by key guidance on narrative synthesis methods, we extracted data about the description of narrative synthesis methods, presentation of data, and management of heterogeneity by review authors. These data were used to assess the adequacy of reporting of narrative synthesis. Findings Narrative synthesis was used in 251 (53%) of the 474 reviews sampled. Reporting of narrative synthesis methods and data were limited in most reviews: 58 (77%) of the 75 reviews assessed in more detail provided no rationale for or description of the narrative synthesis process, with 36 (48%) simply stating that meta-analysis was not possible. Presentation of data was often incomplete (40%, n=30), with investigation and management of heterogeneity unclear. Of the 51 reviews in which data were grouped, 22 (43%) tabulated data differently from how the data were grouped in the text. Interpretation Narrative synthesis of quantitative data in public health reviews is often inadequate. Reporting of methods is limited, and available guidance is rarely referred to. Links between the data and the narrative summary are often unclear. This lack of transparency prevents assessment of the reliability of review findings, and threatens the credibility of systematic reviews that use narrative synthesis. For many public health questions, narrative synthesis will continue to be a necessary synthesis method. Updated guidance is therefore required, and review authors need to be transparent in their reporting of narrative synthesis, thereby improving reliability. Funding MC and HT were funded by the UK Medical Research Council Chief Scientist Office Informing Healthy Public Policy programme (MC_UU_12017-15). SVK is supported by a NHS Research Scotland Scottish senior clinical fellowship (SCAF/15/02).

Making the most of natural experiments: What can studies of the withdrawal of public health interventions offer?

Many interventions that may have large impacts on health and health inequalities, such as social and public health policies and health system reforms, are not amenable to evaluation using randomised controlled trials. The United Kingdom Medical Research Councils guidance on the evaluation of natural experiments draws attention to the need for ingenuity to identify interventions which can be robustly studied as they occur, and without experimental manipulation. Studies of intervention withdrawal may usefully widen the range of interventions that can be evaluated, allowing some interventions and policies, such as those that have developed piecemeal over a long period, to be evaluated for the first time. In particular, sudden removal may allow a more robust assessment of an interventions long-term impact by minimising ‘learning effects’. Interpreting changes that follow withdrawal as evidence of the impact of an intervention assumes that the effect is reversible and this assumption must be carefully justified. Otherwise, withdrawal-based studies suffer similar threats to validity as intervention studies. These threats should be addressed using recognised approaches, including appropriate choice of comparators, detailed understanding of the change processes at work, careful specification of research questions, and the use of falsification tests and other methods for strengthening causal attribution. Evaluating intervention withdrawal provides opportunities to answer important questions about effectiveness of population health interventions, and to study the social determinants of health. Researchers, policymakers and practitioners should be alert to the opportunities provided by the withdrawal of interventions, but also aware of the pitfalls.

Down but not out: incidence and estimated costs to society of road casualties in Strathclyde, Scotland

OBJECTIVES To investigate the recent epidemiological patterns and costs of road traffic casualties (RTCs) in Strathclyde, Scotland. STUDY DESIGN Retrospective record-linkage epidemiological study using routine data sources. METHODS A linked police-hospital database was analysed to describe the epidemiology of RTCs from 2004 to 2009. Using UK government methodology, the costs of road casualties to the National Health Service (NHS) and society were assessed. RESULTS RTC rates declined over the study period. Males were at higher risk than females as were those residing in more socially deprived addresses. The estimated costs of RTCs in Strathclyde amounted to £400 million annually. Of this, around one twentieth (£20 million per year), was attributable to direct NHS costs. CONCLUSIONS Road casualties remain a major public health threat in Strathclyde, and contribute to health inequalities. RTC costs to society amount to almost a tenth of NHS revenue expenditure. Cost-effective road safety measures should be deployed more widely.

Lack of transparency in reporting narrative synthesis of quantitative data: a methodological assessment of systematic reviews

Objective To assess the adequacy of reporting and conduct of narrative synthesis of quantitative data (NS) in reviews evaluating the effectiveness of public health interventions. Study Design and Setting A retrospective comparison of a 20% (n = 474/2,372) random sample of public health systematic reviews from the McMaster Health Evidence database (January 2010–October 2015) to establish the proportion of reviews using NS. From those reviews using NS, 30% (n = 75/251) were randomly selected and data were extracted for detailed assessment of: reporting NS methods, management and investigation of heterogeneity, transparency of data presentation, and assessment of robustness of the synthesis. Results Most reviews used NS (56%, n = 251/446); meta-analysis was the primary method of synthesis for 44%. In the detailed assessment of NS, 95% (n = 71/75) did not describe NS methods; 43% (n = 32) did not provide transparent links between the synthesis data and the synthesis reported in the text; of 14 reviews that identified heterogeneity in direction of effect, only one investigated the heterogeneity; and 36% (n = 27) did not reflect on limitations of the synthesis. Conclusion NS methods are rarely reported in systematic reviews of public health interventions and many NS reviews lack transparency in how the data are presented and the conclusions are reached. This threatens the validity of much of the evidence synthesis used to support public health. Improved guidance on reporting and conduct of NS will contribute to improved utility of NS systematic reviews.