Lyndal Bond

Process evaluation of complex interventions: Medical Research Council guidance

Process evaluation is an essential part of designing and testing complex interventions. New MRC guidance provides a framework for conducting and reporting process evaluation studies

Does bullying cause emotional problems? A prospective study of young teenagers

Abstract Objectives: To establish the relation between recurrent peer victimisation and onset of self reported symptoms of anxiety or depression in the early teen years. Design: Cohort study over two years. Setting: Secondary schools in Victoria, Australia. Participants: 2680 students surveyed twice in year 8 (aged 13 years) and once in year 9. Main outcome measures: Self reported symptoms of anxiety or depression were assessed by using the computerised version of the revised clinical interview schedule. Incident cases were students scoring ≥12 in year 9 but not previously. Prior victimisation was defined as having been bullied at either or both survey times in year 8. Results: Prevalence of victimisation at the second survey point in year 8 was 51% (95% confidence interval 49% to 54%), and prevalence of self reported symptoms of anxiety or depression was 18% (16% to 20%). The incidence of self reported symptoms of anxiety or depression in year 9 (7%) was significantly associated with victimisation reported either once (odds ratio 1.94, 1.1 to 3.3) or twice (2.30, 1.2 to 4.3) in year 8. After adjustment for availability of social relations and for sociodemographic factors, recurrent victimisation remained predictive of self reported symptoms of anxiety or depression for girls (2.60, 1.2 to 5.5) but not for boys (1.36, 0.6 to 3.0). Newly reported victimisation in year 9 was not significantly associated with prior self report of symptoms of anxiety or depression (1.48, 0.4 to 6.0). Conclusion: A history of victimisation and poor social relationships predicts the onset of emotional problems in adolescents. Previous recurrent emotional problems are not significantly related to future victimisation. These findings have implications for how seriously the occurrence of victimisation is treated and for the focus of interventions aimed at addressing mental health issues in adolescents. What is already known on this topic Being bullied is a common experience for many young people Victimisation is related to depression and, to a lesser extent, anxiety, loneliness, and general self esteem Debate remains as to whether victimisation precedes the onset of emotional problems or whether young people with emotional problems “invite” victimisation What this study adds A history of victimisation predicts the onset of anxiety or depression, especially in adolescent girls Previous recurrent emotional problems are not significantly related to future victimisation Reduction in bullying in schools could have a substantial impact on the emotional wellbeing of young people

Adolescent resilience : a concept analysis

There is need for greater clarity around the concept of resilience as it relates to the period of adolescence. Literature on resilience published between 1990 and 2000 and relevant to adolescents aged between 12- and 18-years of age was reviewed with the aim of examining the various uses of the term, and commenting on how specific ways of conceptualizing of resilience may help develop new research agendas in the field. By bringing together ideas on resilience from a variety of research and clinical perspectives, the purpose of the review is to explicate core elements of resilience in more precise ways, in the hope that greater conceptual clarity will lead to a range of tailored interventions that benefit young people.

The Gatehouse Project: can a multilevel school intervention affect emotional wellbeing and health risk behaviours?

Study objective: The aim of this study was to determine the effect of a multilevel school based intervention on adolescents’ emotional wellbeing and health risk behaviours. Design: School based cluster randomised controlled trial. Students were surveyed using laptop computers, twice in the first year of intervention and annually thereafter for a further two years. Setting: Secondary schools. Participants: 2678 year 8 students (74%) participated in the first wave of data collection. Attrition across the waves was less than 3%, 8%, and 10% respectively with no differential response rate between intervention and control groups at the subsequent waves (98% v 96%; 92% v 92%, and 90% v 89% respectively). Main results: A comparatively consistent 3% to 5% risk difference was found between intervention and control students for any drinking, any and regular smoking, and friends’ alcohol and tobacco use across the three waves of follow up. The largest effect was a reduction in the reporting of regular smoking by those in the intervention group (OR 0.57, 0.62, and 0.72 at waves 2, 3, and 4 respectively). There was no significant effect of the intervention on depressive symptoms, and social and school relationships. Conclusions: While further research is required to determine fully the processes of change, this study shows that a focus on general cognitive skills and positive changes to the social environment of the school can have a substantial impact on important health risk behaviours.

Promoting Social Inclusion in Schools: A Group-Randomized Trial of Effects on Student Health Risk Behavior and Well-Being

OBJECTIVES We sought to test the efficacy of an intervention that was designed to promote social inclusion and commitment to education, in reducing among students health risk behaviors and improving emotional well-being. METHODS The design was a cluster-randomized trial in 25 secondary schools in Victoria, Australia. The subjects were 8th-grade students (aged 13 to 14 y) in 1997 (n=2545) and subsequent 8th-grade students in 1999 (n=2586) and 2001 (n=2463). The main outcomes were recent substance use, antisocial behavior, initiation of sexual intercourse, and depressive symptoms. RESULTS At 4-year follow-up, the prevalence of marked health risk behaviors was approximately 20% in schools in the comparison group and 15% in schools in the intervention group, an overall reduction of 25%. In ordinal logistic regression models a protective effect of intervention was found for a composite measure of health risk behaviors in unadjusted models (odds ratio [OR]= 0.69; 95% confidence interval [CI]= 0.50, 0.95) and adjusted models (OR= 0.71; CI =0.52, 0.97) for potential confounders. There was no evidence of a reduction in depressive symptoms. CONCLUSION The study provides support for prevention strategies in schools that move beyond health education to promoting positive social environments.

The Gatehouse Project: a systematic approach to mental health promotion in secondary schools

Objective: The objective of this paper is to outline the conceptual background and strategy of intervention for a systematic and sustainable approach to mental health promotion in secondary schools. Method: The conceptual origins of the Gatehouse Project are described in terms of the epidemiology of adolescent mental health problems, attachment theory, education reform research and health promotional theory and practice. The elements of health promotional work are described in terms of structural change and priority setting; implementation at multiple levels within the participating schools is described. Results: The conceptual framework of the Gatehouse Project emphasises healthy attachments with peers and teachers through the promotion of a sense of security and trust, effective communication and a sense of positive self-regard based on participation in varied aspects of school and community life. A school social climate profile is derived from a questionnaire survey of students. An adolescent health team uses this information to set priorities for change within the school. Interventions may focus on the promotion of a positive social climate of the whole school or in the classroom. Curriculum-based health education is also used and based on materials that are relevant to the normal developmental experiences of teenagers. These are integrated into the mainstream curriculum and incorporate a strong component of teacher professional development. Lastly, the intervention promotes linkage between the school and broader community with a particular emphasis on the needs of young people at high risk of school drop-out. Conclusions: Educational environments are complex systems undergoing continuous and simultaneous changes. The Gatehouse Project will provide unique information on the relationship between the social environment and the emotional wellbeing of young people. More importantly it outlines a sustainable process for building the capacity of schools to promote the social and emotional development of young people.

The health and socioeconomic impacts of major multi-sport events: systematic review (1978-2008)

Objective To assess the effects of major multi-sport events on health and socioeconomic determinants of health in the population of the city hosting the event. Design Systematic review. Data sources We searched the following sources without language restrictions for papers published between 1978 and 2008: Applied Social Science Index and Abstracts (ASSIA), British Humanities Index (BHI), Cochrane database of systematic reviews, Econlit database, Embase, Education Resources Information Center (ERIC) database, Health Management Information Consortium (HMIC) database, International Bibliography of the Social Sciences (IBSS), Medline, PreMedline, PsycINFO, Sociological Abstracts, Sportdiscus, Web of Knowledge, Worldwide Political Science Abstracts, and the grey literature. Review methods Studies of any design that assessed the health and socioeconomic impacts of major multi-sport events on the host population were included. We excluded studies that used exclusively estimated data rather than actual data, that investigated host population support for an event or media portrayals of host cities, or that described new physical infrastructure. Studies were selected and critically appraised by two independent reviewers. Results Fifty four studies were included. Study quality was poor, with 69% of studies using a repeat cross-sectional design and 85% of quantitative studies assessed as being below 2+ on the Health Development Agency appraisal scale, often because of a lack of comparison group. Five studies, each with a high risk of bias, reported health related outcomes, which were suicide, paediatric health service demand, presentations for asthma in children (two studies), and problems related to illicit drug use. Overall, the data did not indicate clear negative or positive health impacts of major multi-sport events on host populations. The most frequently reported outcomes were economic outcomes (18 studies). The outcomes used were similar enough to allow us to perform a narrative synthesis, but the overall impact of major multi-sport events on economic growth and employment was unclear. Two thirds of the economic studies reported increased economic growth or employment immediately after the event, but all these studies used some estimated data in their models, failed to account for opportunity costs, or examined only short term effects. Outcomes for transport were also similar enough to allow synthesis of six of the eight studies, which showed that event related interventions—including restricted car use and public transport promotion—were associated with significant short term reductions in traffic volume, congestion, or pollution in four out of five cities. Conclusions The available evidence is not sufficient to confirm or refute expectations about the health or socioeconomic benefits for the host population of previous major multi-sport events. Future events such as the 2012 Olympic Games and Paralympic Games, or the 2014 Commonwealth Games, cannot be expected to automatically provide benefits. Until decision makers include robust, long term evaluations as part of their design and implementation of events, it is unclear how the costs of major multi-sport events can be justified in terms of benefits to the host population.

Building Capacity for System-Level Change in Schools: Lessons from the Gatehouse Project

The Gatehouse Project is an innovative, comprehensive approach to mental health promotion in secondary schools. It sets out to promote student engagement and school connectedness as the way to improve emotional well-being and learning outcomes. The key elements of the whole-school intervention are the establishment and support of a school-based adolescent health team; the identification of risk and protective factors in each school’s social and leaning environment from student surveys; and, through the use of these data, the identification and implementation of effective strategies to address these issues. The project evaluation used a cluster-randomized controlled trial design involving 26 schools with initial results demonstrating considerable success in reducing smoking rates among Year 8 children. This article describes and accounts for how system-level changes have been made in schools through a process of capacity building. This encourages teachers, parents, and students to view the core business of education differently.

Process evaluation in complex public health intervention studies: the need for guidance

Public health interventions aim to improve the health of populations or at-risk subgroups. Problems targeted by such interventions, such as diet and smoking, involve complex multifactorial aetiology. Interventions will often aim to address more than one cause simultaneously, targeting factors at multiple levels (eg, individual, interpersonal, organisational), and comprising several components which interact to affect more than one outcome.1 They will often be delivered in systems which respond in unpredictable ways to the new intervention.2 Recognition is growing that evaluations need to understand this complexity if they are to inform future intervention development, or efforts to apply the same intervention in another setting or population.1 Achieving this will require evaluators to move beyond a ‘does it work?’ focus, towards combining outcomes and process evaluation. There is no such thing as a typical process evaluation, with the term applied to studies which range from a few simple quantitative items on satisfaction, to complex mixed-method studies exploring issues such as the process of implementation, or contextual influences on implementation and outcomes. As recognised within MRC guidance for evaluating complex interventions, process evaluation may be used to ‘assess fidelity and quality of implementation , clarify causal mechanisms and identify contextual factors associated with variation in outcomes’.1 This paper briefly discusses each of these core aims for process evaluation, before describing current Medical Research Council (MRC) Population Health Sciences Research Network (PHSRN) funded work to develop guidance for process evaluations of complex public health interventions. ### Intervention implementation An important role for process evaluations is to examine the quantity and quality of what was actually implemented in practice, and why. This may inform implementation of similar interventions elsewhere, and facilitate interpretation of intervention outcomes. While notions of standardisation are central to implementation assessment, the nature of …

Individual, family, school, and community risk and protective factors for depressive symptoms in adolescents: a comparison of risk profiles for substance use and depressive symptoms.

This study examines the relationship between adolescent depressive symptoms and risk and protective factors identified for substance use. A questionnaire, developed to measure these factors in a young person’s community, family, school, peer group, and individual characteristics for substance use, was used to assess associations with self-reported depressive symptoms. Data were provided by a representative sample of 8984 secondary school students in Victoria, Australia. The prevalence of depressive symptoms was 10.5% (95% CI 9.2,12.0) for males and 21.7% (95% CI 20.3,23.7) for females. Depressive symptoms were associated with factors in all domains, with the strongest associations in the family domain. Strong relationships were found between the number of elevated risk and protective factors and depressive symptoms, maintained after adjusting for substance use. Patterns of associations were similar for users and nonsubstance users. The findings indicate that prevention programs targeting factors for substance use have the potential to impact on depression.