Dylan Kneale

Developing and Optimising the Use of Logic Models in Systematic Reviews: Exploring Practice and Good Practice in the Use of Programme Theory in Reviews

Background Logic models are becoming an increasingly common feature of systematic reviews, as is the use of programme theory more generally in systematic reviewing. Logic models offer a framework to help reviewers to ‘think’ conceptually at various points during the review, and can be a useful tool in defining study inclusion and exclusion criteria, guiding the search strategy, identifying relevant outcomes, identifying mediating and moderating factors, and communicating review findings. Methods and Findings In this paper we critique the use of logic models in systematic reviews and protocols drawn from two databases representing reviews of health interventions and international development interventions. Programme theory featured only in a minority of the reviews and protocols included. Despite drawing from different disciplinary traditions, reviews and protocols from both sources shared several limitations in their use of logic models and theories of change, and these were used almost unanimously to solely depict pictorially the way in which the intervention worked. Logic models and theories of change were consequently rarely used to communicate the findings of the review. Conclusions Logic models have the potential to be an aid integral throughout the systematic reviewing process. The absence of good practice around their use and development may be one reason for the apparent limited utility of logic models in many existing systematic reviews. These concerns are addressed in the second half of this paper, where we offer a set of principles in the use of logic models and an example of how we constructed a logic model for a review of school-based asthma interventions.

Distribution and determinants of risk of teenage motherhood in three British longitudinal studies: implications for targeted prevention interventions

Purpose In order to consider the potential contribution of universal versus targeted prevention interventions, the authors examined what is the distribution of established risk variables for teenage motherhood? from where in these distributions do births arise? and how does this distribution/determination of risk vary between studies? Methods Secondary data analysis of three British longitudinal studies. Results For all cohorts and variables, the ‘risk’ category was the least frequent. Continuous risk factors were normally distributed. A high rate of teenage motherhood within a risk category often translated into low ‘contribution’ to the overall rate (eg, expectation to leave school at the minimum age among the 1989/1990-born cohort) and vice versa. Most young women had a low probability of teenage motherhood. For any targeting strategy, combining risk factors and a low threshold of predicted probability would be necessary to achieve adequate sensitivity. Assessing between-cohort applicability of findings, the authors find that the numbers of teenage parents is poorly estimated and estimates of the variability and direction of risk may also be inadequate. Conclusions With reference to a number of established risk factors, there is not a core of easily identifiable multiply disadvantaged girls who go on to constitute the majority of teenage mothers in these studies. While individual risk factors are unlikely to enable targeting, a composite may have some limited potential, albeit with a low threshold for ‘risk’ and with the caveat that evidence from one population may not inform good targeting in another. It is likely that universal approaches will have more impact.

Theorising and Measuring Place in Neighbourhood Effects Research: The Example of Teenage Parenthood in England

Government policies to reduce teenage parenthood are, in part, informed by a belief in neighbourhood effects, although the current evidence for neighbourhood effects on teenage parenthood is remarkably weak. This chapter highlights the conceptual problems in the existing research around the importance of place and geography. It critiques the fact that many studies search for general evidence for neighbourhood effects without formulating specific hypotheses about causal mechanisms, and often without detailed knowledge of the outcome in question. The chapter also critiques the lack of attention paid to the most appropriate spatial scale to study specific effects. Using data from the British Cohort Study (BCS70), a longitudinal study of people born in 1970, with postcode geo-coding of neighbourhood characteristics, the effects of several geographies on teenage parenthood are tested. The results suggested that place effects on values around fertility operate at a fine spatial scale. In conclusion, it may be impossible to separate the social processes leading to early parenthood from one another using quantitative methods and that neighbourhood effects research should move towards more explicit and transparent considerations of geography in order to make a stronger contribution to knowledge of place effects.

Fertility, Living Arrangements, Care and Mobility

Foreword: Ian Diamond.- Preface.- 1 Fertility, living arrangements, care and mobility: Dylan Kneale, Ernestina Coast and John Stillwell.- 2 Delayed childbearing and childlessness: Roona Simpson.- 3 Womens education and childbearing: a growing divide: Sarah Smith and Anita Ratcliffe.- 4 The timing of motherhood, mothers employment and child outcomes: Kirstine Hansen, Heather Joshi and Denise Hawkes.- 5 Early parenthood: definition and prediction in two British cohorts: Dylan Kneale.- 6. Currently cohabiting: relationship attitudes, expectations and outcomes: Ernestina Coast.- 7. Living arrangements, health and well-being: Harriet Young and Emily Grundy.- 8 Stepparenting and mental health: Paul Boyle, Peteke Feijten, Zhiqiang Feng, Vernon Gayle and Elspeth Graham.- 9 Grandparents and the care of their grandchildren: Alison Smith Koslowski.- 10. Internal Migration and inter-households relationships: Oliver Duke-Williams.- 11. Exploring dimensions of school change during primary education: Joan Wilson.- Subject Index

The use of evidence in English local public health decision-making: a systematic scoping review

BackgroundPublic health decision-making structures in England have transformed since the implementation of reforms in 2013, with responsibility for public health services and planning having shifted from the “health” boundary to local authority (LA; local government) control. This transformation may have interrupted flows of research evidence use in decision-making and introduced a new political element to public health decision-making. For generators of research evidence, understanding and responding to this new landscape and decision-makers’ evidence needs is essential.MethodsWe conducted a systematic scoping review of the literature, drawing upon four databases and undertaking manual searching and citation tracking. Included studies were English-based, published in 2010 onwards, and were focused on public health decision-making, including the utilisation or underutilisation of research evidence use, in local (regional or sub-regional) areas. All studies presented empirical findings collected through primary research methods or through the reanalysis of existing primary data.ResultsFrom a total of 903 records, 23 papers from 21 studies were deemed to be eligible and were included for further data extraction. Three clear trends in evidence use were identified: (i) the primacy of local evidence, (ii) the important role of local experts in providing evidence and knowledge, and (iii) the high value placed on local evaluation evidence despite the varying methodological rigour. Barriers to the use of research evidence included issues around access and availability of applicable research evidence, and indications that the use of evidence could be perceived as a bureaucratic process. Two new factors resulting from reforms to public health structures were identified that potentially changed existing patterns of research evidence use and decision-making requirements: (i) greater emphasis among public health practitioners on the perceived uniqueness of LA areas and structures following devolution of public health into LAs and (ii) challenges introduced in responding to higher levels of local political accountability.ConclusionsThere is a need to better understand and respond to the evidence needs of decision-makers working in public health and to work more collaboratively in developing solutions to the underutilisation of research evidence in decision-making.

School‐based self management interventions for asthma in children and adolescents: a mixed methods systematic review

BACKGROUND: Asthma is a common respiratory condition in children that is characterised by symptoms including wheeze, shortness of breath, chest tightness, and cough. Children with asthma may be able to manage their condition more effectively by improving inhaler technique, and by recognising and responding to symptoms. Schools offer a potentially supportive environment for delivering interventions aimed at improving self-management skills among children. The educational ethos aligns with skill and knowledge acquisition and makes it easier to reach children with asthma who do not regularly engage with primary care. Given the multi-faceted nature of self-management interventions, there is a need to understand the combination of intervention features that are associated with successful delivery of asthma self-management programmes. OBJECTIVES: This review has two primary objectives.• To identify the intervention features that are aligned with successful intervention implementation.• To assess effectiveness of school-based interventions provided to improve asthma self-management among children.We addressed the first objective by performing qualitative comparative analysis (QCA), a synthesis method described in depth later, of process evaluation studies to identify the combination of intervention components and processes that are aligned with successful intervention implementation.We pursued the second objective by undertaking meta-analyses of outcomes reported by outcome evaluation studies. We explored the link between how well an intervention is implemented and its effectiveness by using separate models, as well as by undertaking additional subgroup analyses. SEARCH METHODS: We searched the Cochrane Airways Trials Register for randomised studies. To identify eligible process evaluation studies, we searched MEDLINE, Embase, the Cumulative Index to Nursing and Allied Health Literature (CINAHL), PsycINFO, the Cochrane Database of Systematic Reviews (CDSR), Web of Knowledge, the Database of Promoting Health Effectiveness Reviews (DoPHER), the Database of Abstracts of Reviews of Effects (DARE), the International Biography of Social Science (IBSS), Bibliomap, Health Technology Assessment (HTA), Applied Social Sciences Index and Abstracts (ASSIA), and Sociological Abstracts (SocAbs). We conducted the latest search on 28 August 2017. SELECTION CRITERIA: Participants were school-aged children with asthma who received the intervention in school. Interventions were eligible if their purpose was to help children improve management of their asthma by increasing knowledge, enhancing skills, or changing behaviour. Studies relevant to our first objective could be based on an experimental or quasi-experimental design and could use qualitative or quantitative methods of data collection. For the second objective we included randomised controlled trials (RCTs) where children were allocated individually or in clusters (e.g. classrooms or schools) to self-management interventions or no intervention control. DATA COLLECTION AND ANALYSIS: We used qualitative comparative analysis (QCA) to identify intervention features that lead to successful implementation of asthma self-management interventions. We measured implementation success by reviewing reports of attrition, intervention dosage, and treatment adherence, irrespective of effects of the interventions.To measure the effects of interventions, we combined data from eligible studies for our primary outcomes: admission to hospital, emergency department (ED) visits, absence from school, and days of restricted activity due to asthma symptoms. Secondary outcomes included unplanned visits to healthcare providers, daytime and night-time symptoms, use of reliever therapies, and health-related quality of life as measured by the Asthma Quality of Life Questionnaire (AQLQ). MAIN RESULTS: We included 55 studies in the review. Thirty-three studies in 14,174 children provided information for the QCA, and 33 RCTs in 12,623 children measured the effects of interventions. Eleven studies contributed to both the QCA and the analysis of effectiveness. Most studies were conducted in North America in socially disadvantaged populations. High school students were better represented among studies contributing to the QCA than in studies contributing to effectiveness evaluations, which more commonly included younger elementary and junior high school students. The interventions all attempted to improve knowledge of asthma, its triggers, and stressed the importance of regular practitioner review, although there was variation in how they were delivered.QCA results highlighted the importance of an intervention being theory driven, along with the importance of factors such as parent involvement, child satisfaction, and running the intervention outside the childs own time as drivers of successful implementation.Compared with no intervention, school-based self-management interventions probably reduce mean hospitalisations by an average of about 0.16 admissions per child over 12 months (SMD -0.19, 95% CI -0.35 to -0.04; 1873 participants; 6 studies, moderate certainty evidence). They may reduce the number of children who visit EDs from 7.5% to 5.4% over 12 months (OR 0.70, 95% CI 0.53 to 0.92; 3883 participants; 13 studies, low certainty evidence), and probably reduce unplanned visits to hospitals or primary care from 26% to 21% at 6 to 9 months (OR 0.74, 95% CI 0.60 to 0.90; 3490 participants; 5 studies, moderate certainty evidence). Self-management interventions probably reduce the number of days of restricted activity by just under half a day over a two-week period (MD 0.38 days 95% CI -0.41 to -0.18; 1852 participants; 3 studies, moderate certainty evidence). Effects of interventions on school absence are uncertain due to the variation between the results of the studies (MD 0.4 fewer school days missed per year with self-management (-1.25 to 0.45; 4609 participants; 10 studies, low certainty evidence). Evidence is insufficient to show whether the requirement for reliever medications is affected by these interventions (OR 0.52, 95% CI 0.15 to 1.81; 437 participants; 2 studies; very low-certainty evidence). Self-management interventions probably improve childrens asthma-related quality of life by a small amount (MD 0.36 units higher on the Paediatric AQLQ(95% CI 0.06 to 0.64; 2587 participants; 7 studies, moderate certainty evidence). AUTHORS CONCLUSIONS: School-based asthma self-management interventions probably reduce hospital admission and may slightly reduce ED attendance, although their impact on school attendance could not be measured reliably. They may also reduce the number of days where children experience asthma symptoms, and probably lead to small improvements in asthma-related quality of life. Many of the studies tested the intervention in younger children from socially disadvantaged populations. Interventions that had a theoretical framework, engaged parents and were run outside of childrens free time were associated with successful implementation.

How can additional secondary data analysis of observational data enhance the generalisability of meta-analytic evidence for local public health decision making?

This paper critically explores how survey and routinely collected data could aid in assessing the generalisability of public health evidence. We propose developing approaches that could be employed in understanding the relevance of public health evidence, and investigate ways of producing meta-analytic estimates tailored to reflect local circumstances, based on analyses of secondary data. Currently, public health decision makers face challenges in interpreting global review evidence to assess its meaning in local contexts. A lack of clarity on the definition and scope of generalisability, and the absence of consensus on its measurement, has stunted methodological progress. The consequence of failing to tackle generalisability means that systematic review evidence often fails to fulfil its potential contribution in public health decision making. Three approaches to address these problems are considered and emerging challenges discussed: (1) purposeful exploration after a review has been conducted, and we present a framework of potential avenues of enquiry and a worked example; (2) recalibration of the results to weight studies differentially based on their similarity to conditions in an inference population, and we provide a worked example using UK Census data to understand potential differences in the effectiveness of community engagement interventions among sites in England and Wales; (3) purposeful exploration before starting a review to ensure that the findings are relevant to an inference population. The paper aims to demonstrate how a more nuanced treatment of context in reviews of public health interventions could be achieved through greater engagement with existing large sources of secondary data.

Assessing the applicability of public health intervention evaluations from one setting to another: a methodological study of the usability and usefulness of assessment tools and frameworks

BackgroundPublic health interventions can be complicated, complex and context dependent, making the assessment of applicability challenging. Nevertheless, for them to be of use beyond the original study setting, they need to be generalisable to other settings and, crucially, research users need to be able to identify to which contexts it may be applicable. There are many tools with set criteria for assessing generalisability/applicability, yet few seem to be widely used and there is no consensus on which should be used, or when. This methodological study aimed to test these tools to assess how easy they were to use and how useful they appeared to be.MethodsWe identified tools from an existing review and an update of its search. References were screened on pre-specified criteria. Included tools were tested by using them to assess the applicability of a Swedish weight management intervention to the English context. Researcher assessments and reflections on the usability and utility of the tools were gathered using a standard pro-forma.ResultsEleven tools were included. Their length, content, style and time required to complete varied. No tool was considered ideal for assessing applicability. Their limitations included unrealistic criteria (requiring unavailable information), a focus on implementation to the neglect of transferability (i.e. little focus on potential effectiveness in the new setting), overly broad criteria (associated with low reliability), and a lack of an explicit focus on how interventions worked (i.e. their mechanisms of action).ConclusionTools presenting criteria ready to be used may not be the best method for applicability assessments. They are likely to be either too long or incomplete, too focused on differences and fail to address elements that matter for the specific topic of interest. It is time to progress from developing lists of set criteria that are not widely used in the literature, to creating a new approach to applicability assessment. Focusing on mechanisms of action, rather than solely on characteristics, could be a useful approach, and one that remains underutilised in current tools. New approaches to assessing generalisability that evolve away from checklist style assessments need to be developed, tested, reported and discussed.

Examining life course trajectories of lesbian, gay and bisexual people in England - exploring convergence and divergence among a heterogeneous population of older people

Because of limitations in collecting sexuality data, there are very few studies that quantitatively explore the life courses of lesbian, gay, bisexual (LGB) individuals. Likewise it is rare that normative patterns of life course trajectories are assessed in terms of their applicability to LGB individuals. We review the current literature on LGB life course trajectories and discuss potential reasons for gaps in the literature. We explore approaches for defining LGB status. We use data from a cohort of people aged 50 and over (English Longitudinal Study of Ageing) to explore the tempo and occurrences of transitions to adulthood and to older age, and establish some of the differences based on sexual orientation. We examine the connecting health behaviours and life course turning points that may explain some of the differences described above. We show that while the first quartile of transitions to adulthood are experienced fairly uniformly by sexual orientation, differences open up thereafter. LGB peoples life course trajectories are marked by different patterns of care, with LGB people less likely to provide care in the form of parenthood, but potentially more likely to provide care earlier to close friends or relatives. Analyses of connecting events suggest that LGB life course trajectories may be marred by higher levels of volatility, including higher risk financial hardship. Caveats to these results are outlined in full in the paper.

Exploring the importance of evidence in local health and wellbeing strategies

As evidence generators, we need to respond to the changes in the health delivery landscape if we are to continue to support public health decision-makers to make informed and judicious evidence-based choices. This study employs documentary analysis to (i) explore the extent of research evidence use in public health decision-making; (ii) to analyse occurrences of research evidence use in decision-making and (iii) to ascertain whether patterns of evidence use overlap with other area characteristics. Health and Wellbeing Strategies constitute the main source of documentary evidence. Initial results highlight that local areas are undertaking their own programmes of research that are used to inform specific questions, although the methodological robustness of these studies is unknown. There are also commonalities with previous findings, particularly with regard to the underutilization of qualitative research evidence and evidence on the effectiveness of interventions. Using Qualitative Comparative Analysis, this paper also highlights that underutilization of more academic research evidence appears disproportionally weighted towards areas with some of the most complex needs but that are not receiving the highest level of spending to meet these challenges. These areas in particular may be those where knowledge brokerage activities may have the greatest impacts.