Farah Jamal

Realist complex intervention science: Applying realist principles across all phases of the Medical Research Council framework for developing and evaluating complex interventions

The integration of realist evaluation principles within randomised controlled trials (‘realist RCTs’) enables evaluations of complex interventions to answer questions about what works, for whom and under what circumstances. This allows evaluators to better develop and refine mid-level programme theories. However, this is only one phase in the process of developing and evaluating complex interventions. We describe and exemplify how social scientists can integrate realist principles across all phases of the Medical Research Council framework. Intervention development, modelling, and feasibility and pilot studies need to theorise the contextual conditions necessary for intervention mechanisms to be activated. Where interventions are scaled up and translated into routine practice, realist principles also have much to offer in facilitating knowledge about longer-term sustainability, benefits and harms. Integrating a realist approach across all phases of complex intervention science is vital for considering the feasibility and likely effects of interventions for different localities and population subgroups.

‘Dark logic’: theorising the harmful consequences of public health interventions

Although it might be assumed that most public health programmes involving social or behavioural rather than clinical interventions are unlikely to be iatrogenic, it is well established that they can sometimes cause serious harms. However, the assessment of adverse effects remains a neglected topic in evaluations of public health interventions. In this paper, we first argue for the importance of evaluations of public health interventions not only aiming to examine potential harms but also the mechanisms that might underlie these harms so that they might be avoided in the future. Second, we examine empirically whether protocols for the evaluation of public health interventions do examine harmful outcomes and underlying mechanisms and, if so, how. Third, we suggest a new process by which evaluators might develop ‘dark logic models’ to guide the evaluation of potential harms and underlying mechanisms, which includes: theorisation of agency-structure interactions; building comparative understanding across similar interventions via reciprocal and refutational translation; and consultation with local actors to identify how mechanisms might be derailed, leading to harmful consequences. We refer to the evaluation of a youth work intervention which unexpectedly appeared to increase the rate of teenage pregnancy it was aiming to reduce, and apply our proposed process retrospectively to see how this might have strengthened the evaluation. We conclude that the theorisation of dark logic models is critical to prevent replication of harms. It is not intended to replace but rather to inform empirical evaluation.

School-Based Interventions Going Beyond Health Education to Promote Adolescent Health: Systematic Review of Reviews

PURPOSEnHealth education in school classrooms can be effective in promoting sexual health and preventing violence and substance use but effects are patchy and often short term. Classroom education is also challenging because of schools increasing focus on academic-performance metrics. Other school-based approaches are possible, such as healthy school policies, improving how schools respond to bullying, and parent outreach, which go beyond health education to address broader health determinants. Existing systematic reviews include such interventions but often alongside traditional health education. There is scope for a systematic review of reviews to assess and synthesize evidence across existing reviews to develop an overview of the potential of alternative school-based approaches.nnnMETHODSnWe searched 12 databases to identify reviews published after 1980. Data were reviewed by two researchers. Quality was assessed using a modified Assessing the Methodological Quality of Systematic Reviews checklist and results were synthesized narratively.nnnRESULTSnWe screened 7,544 unique references and included 22 reviews. Our syntheses suggest that multicomponent school-based interventions, for example, including school policy changes, parent involvement, and work with local communities, are effective for promoting sexual health and preventing bullying and smoking. There is less evidence that such intervention can reduce alcohol and drug use. Economic incentives to keep girls in school can reduce teenage pregnancies. School clinics can promote smoking cessation. There is little evidence that, on their own, sexual-health clinics, antismoking policies, and various approaches targeting at-risk students are effective.nnnCONCLUSIONSnThere is good evidence that various whole-school health interventions are effective in preventing teenage pregnancy, smoking, and bullying.

The three stages of building and testing mid-level theories in a realist RCT: a theoretical and methodological case-example

BackgroundRandomised controlled trials (RCTs) of social interventions are often criticised as failing to open the ‘black box’ whereby they only address questions about ‘what works’ without explaining the underlying processes of implementation and mechanisms of action, and how these vary by contextual characteristics of person and place. Realist RCTs are proposed as an approach to evaluation science that addresses these gaps while preserving the strengths of RCTs in providing evidence with strong internal validity in estimating effects.MethodsIn the context of growing interest in designing and conducting realist trials, there is an urgent need to offer a worked example to provide guidance on how such an approach might be practically taken forward. The aim of this paper is to outline a three-staged theoretical and methodological process of undertaking a realist RCT using the example of the evaluation of a whole-school restorative intervention aiming to reduce aggression and bullying in English secondary schools.DiscussionFirst, informed by the findings of our initial pilot trial and sociological theory, we elaborate our theory of change and specific a priori hypotheses about how intervention mechanisms interact with context to produce outcomes. Second, we describe how we will use emerging findings from the integral process evaluation within the RCT to refine, and add to, these a priori hypotheses before the collection of quantitative, follow-up data. Third, we will test our hypotheses using a combination of process and outcome data via quantitative analyses of effect mediation (examining mechanisms) and moderation (examining contextual contingencies). The results are then used to refine and further develop the theory of change.ConclusionThe aim of the realist RCT approach is thus not merely to assess whether the intervention is effective or not, but to develop empirically informed mid-range theory through a three-stage process. There are important implications for those involved with reporting and reviewing RCTs, including the use of new, iterative protocols.Trial registrationCurrent Controlled Trials ISRCTN10751359 (Registered 11 March 2014)

Timing of the initiation of antenatal care: An exploratory qualitative study of women and service providers in East London

Objective to explore the factors which influence the timing of the initiation of a package of publically-funded antenatal care for pregnant women living in a diverse urban setting Design a qualitative study involving thematic analysis of 21 individual interviews and six focus group discussions. Setting Newham, a culturally diverse borough in East London, UK Participants individual interviews were conducted with 21 pregnant and postnatal women and focus group discussions were conducted with a total of 26 health service staff members(midwives and bilingual health advocates) and 32 women from four community groups (Bangladeshi, Somali, Lithuanian and Polish). Findings initial care-seeking by pregnant women is influenced by the perception that the package of antenatal care offered by the National Health Service is for viable and continuing pregnancies, as well as little perceived urgency in initiating antenatal care. This is particularly true when set against competing responsibilities and commitments in women’s lives and for pregnancies with no apparent complications or disconcerting symptoms. Barriers to access to this package of antenatal care include difficulties in navigating the health service and referral system, which are compounded for women unable to speak English, and service provider delays in the processing of referrals. Accessing antenatal care was sometimes equated with relinquishing control, particularly for young women and women for whom language barriers prohibit active engagement with care. Conclusions and implications for practice if women are to be encouraged to seek antenatal care from maternity services early in pregnancy, the purpose and value to all women of doing so need to be made clear across the communities in which they live. As a woman may need time to accept her pregnancy and address other priorities in her life before seeking antenatal care, it is crucial that once she does decide to seek such care, access is quick and easy. Difficulties found in navigating the system of referral for antenatal care point to a need for improved access to primary care and a simple and efficient process of direct referral to antenatal care, alongside the delivery of antenatal care which is woman-centred and experienced as empowering.

Student- and school-level belonging and commitment and student smoking, drinking and misbehaviour

Objectives: It has been suggested that students are healthier in schools where more students are committed to school. Previous research has examined this only using a proxy measure of value-added education (a measure of whether school-level attendance and attainment are higher than predicted by students’ social profile), finding associations with smoking tobacco, use of alcohol and illicit drugs, and violence. These findings do not provide direct insights into the associations between school-level aggregate student commitment and health behaviours, and may simply reflect the proxy measure being residually confounded by unmeasured student characteristics. We examined the previously used proxy measure of value-added education, as well as direct measures at the level of the school and the student of lack of student commitment to school to see whether these were associated with students’ self-reported smoking tobacco, alcohol use and school misbehaviour. Design: Cross-sectional survey. Setting: A total of 40 schools in south-east England. Methods: Multi-level analyses. Results: There were associations between school- and student-level measures of lack of commitment to school and tobacco smoking, alcohol use and school misbehaviour outcomes, but the proxy measure of school-level commitment, value-added education, was not associated with these outcomes. A sensitivity analysis focused only on violent aspects of school misbehaviour found a pattern of associations identical to that found for the measure of misbehaviour. Conclusion: Our study provides the first direct evidence in support of the Theory of Human Functioning and School Organisation.

Narratives of community engagement: a systematic review-derived conceptual framework for public health interventions

BackgroundGovernment policy increasingly supports engaging communities to promote health. It is critical to consider whether such strategies are effective, for whom, and under what circumstances. However, ‘community engagement’ is defined in diverse ways and employed for different reasons. Considering the theory and context we developed a conceptual framework which informs understanding about what makes an effective (or ineffective) community engagement intervention.MethodsWe conducted a systematic review of community engagement in public health interventions using: stakeholder involvement; searching, screening, appraisal and coding of research literature; and iterative thematic syntheses and meta-analysis. A conceptual framework of community engagement was refined, following interactions between the framework and each review stage.ResultsFrom 335 included reports, three products emerged: (1) two strong theoretical ‘meta-narratives’: one, concerning the theory and practice of empowerment/engagement as an independent objective; and a more utilitarian perspective optimally configuring health services to achieve defined outcomes. These informed (2) models that were operationalized in subsequent meta-analysis. Both refined (3) the final conceptual framework. This identified multiple dimensions by which community engagement interventions may differ. Diverse combinations of intervention purpose, theory and implementation were noted, including: ways of defining communities and health needs; initial motivations for community engagement; types of participation; conditions and actions necessary for engagement; and potential issues influencing impact. Some dimensions consistently co-occurred, leading to three overarching models of effective engagement which either: utilised peer-led delivery; employed varying degrees of collaboration between communities and health services; or built on empowerment philosophies.ConclusionsOur conceptual framework and models are useful tools for considering appropriate and effective approaches to community engagement. These should be tested and adapted to facilitate intervention design and evaluation. Using this framework may disentangle the relative effectiveness of different models of community engagement, promoting effective, sustainable and appropriate initiatives.

A new measure of unhealthy school environments and its implications for critical assessments of health promotion in schools

Abstract The theory of human functioning and school organisation informed by Basil Bernstein’s sociology of education suggests that to gain the commitment and promote the health of students, particularly those from disadvantaged backgrounds, schools require radical transformations eroding various ‘boundaries’: between and among staff and students; between students’ academic learning and broader social development and welfare; and between schools and their local communities. Existing research examining this theory has reported associations between school-level proxy measures of student commitment and lower rates of student smoking, drinking alcohol, use of drugs and violence. But this research has not directly assessed whether reduced school boundaries explain this. We piloted a new scale derived from teacher reports to measure unhealthy school boundaries and examined its inter-item reliability and its criterion validity in terms of associations with various measures of school commitment and smoking. Data on boundaries came from 101 teachers across 40 schools. Data on student commitment and smoking came from 6667 students. We assessed reliability by examining correlations between scale-items and criterion validity in terms of associations with student-reported commitment and smoking. Inter-item reliability was sub-optimal but better within the subscales about boundaries between academic/broader learning and schools/local communities. The scale had good criterion validity, strongly associated with reduced student-reported school commitment and increased student-reported smoking. We reflect on the implications of these findings in terms of critical perspectives on health promotion in schools and the strengths and limitations of quantitative research in examining health behaviours as opposed to practices.

Where next with theory and research on how the school environment influences young people's substance use?

Substance use (smoking, drinking and illicit drug use) remains, a serious problem for young people living in industrialised countries. There is increasing interest in interventions to modify the school, environment, addressing the multiple upstream determinants of young, peoples health. This article provides an overview of current theory, about how secondary school environments influence young peoples, substance use before focusing on the Theory of Human Functioning and, School Organisation. It critically examines the extent to which this, theory is substantiated by quantitative and qualitative evidence and, considers how the theory might be elaborated to better inform future, empirical research.

Systematic review of reviews of observational studies of school-level effects on sexual health, violence and substance use.

For three decades there have been reports that the quality of schools affects student health. The literature is diverse and reviews have addressed different aspects of how the school environment may affect health. This paper is the first to synthesise this evidence using a review of reviews focusing on substance-use, violence and sexual-health. Twelve databases were searched. Eleven included reviews were quality-assessed and synthesised narratively. There is strong evidence that schools success in engaging students is associated with reduced substance use. There is little evidence that tobacco-control policies and school sexual-health clinics on their own are associated with better outcomes.