Therese Riley

Complex interventions: how “out of control” can a randomised controlled trial be?

Complex interventions are more than the sum of their parts, and interventions need to be better theorised to reflect this Many people think that standardisation and randomised controlled trials go hand in hand. Having an intervention look the same as possible in different places is thought to be paramount. But this may be why some community interventions have had weak effects. We propose a radical departure from the way large scale interventions are typically conceptualised. This could liberate interventions to be responsive to local context and potentially more effective while still allowing meaningful evaluation in controlled designs. The key lies in looking past the simple elements of a system to embrace complex system functions and processes. The suitability of cluster randomised trials for evaluating interventions directed at whole communities or organisations remains vexed.1 It need not be.2 Some health promotion advocates (including the WHO European working group on health promotion evaluation) believe randomised controlled trials are inappropriate because of the perceived requirement for interventions in different sites to be standardised or look the same.1 3 4 They have abandoned randomised trials because they think context level adaptation, which is essential for interventions to work, is precluded by trial designs. An example of context level adaptation might be adjusting educational materials to suit various local learning styles and literacy levels. Lead thinkers in complex interventions, such as the UKs Medical Research Council, also think that trials of complex interventions must “consistently provide as close to the same intervention as possible” by “standardising the content and delivery of the intervention.”5 By contrast, however, they do not see this as a reason to reject randomised controlled trials. These divergent views have led to problems on two fronts. Firstly, the field of health promotion is being turned away from randomised …

Theorising Interventions as Events in Systems

Conventional thinking about preventive interventions focuses over simplistically on the “package” of activities and/or their educational messages. An alternative is to focus on the dynamic properties of the context into which the intervention is introduced. Schools, communities and worksites can be thought of as complex ecological systems. They can be theorised on three dimensions: (1) their constituent activity settings (e.g., clubs, festivals, assemblies, classrooms); (2) the social networks that connect the people and the settings; and (3) time. An intervention may then be seen as a critical event in the history of a system, leading to the evolution of new structures of interaction and new shared meanings. Interventions impact on evolving networks of person-time-place interaction, changing relationships, displacing existing activities and redistributing and transforming resources. This alternative view has significant implications for how interventions should be evaluated and how they could be made more effective. We explore this idea, drawing on social network analysis and complex systems theory.

Methods for exploring implementation variation and local context within a cluster randomised community intervention trial

Insignificant or modest findings in intervention trials may be attributable to poorly designed or theorised interventions, poorly implemented interventions, or inadequate evaluation methods. The pre-existing context may also account for the effects observed. A combination of qualitative and quantitative methods is outlined that will permit the determination of how context level factors might modify intervention effectiveness, within a cluster randomised community intervention trial to promote the health of mothers with new babies. The methods include written and oral narratives, key informant interviews, impact logs, and inter-organisational network analyses. Context level factors, which may affect intervention uptake, success, and sustainability are the density of inter-organisational ties within communities at the start of the intervention, the centrality of the primary care agencies expected to take a lead with the intervention, the extent of context-level adaptation of the intervention, and the amount of local resources contributed by the participating agencies. Investigation of how intervention effects are modified by context is a new methodological frontier in community intervention trial research.

Contested ground: how should qualitative evidence inform the conduct of a community intervention trial?

This paper presents issues which arose in the conduct of qualitative evaluation research within a cluster-randomized, community-level, preventive intervention trial. The research involved the collection of narratives of practice regarding the intervention by community development officers working in eight communities over a two-year period. The community development officers were largely responsible for implementing the intervention. We discuss the challenges associated with the collection of data as the intervention unfolded, in particular, the disputes over cues to revise and adjust the intervention (i.e. to use the early data formatively). We explore the ethical uncertainties that arise when multiple parties have different views on the legitimacy of types of knowledge and the appropriate role of research and theory in various trial stages. These issues are discussed drawing on the fields of ethnography, community psychology, epidemiology, qualitative methodology and notions of research reflexivity. We conclude by arguing that, in addition to the usual practice of having an outcome data-monitoring committee, community intervention trials also require a process data-monitoring committee as a forum for debate and decision-making. Without such a forum, the relevance, ethics and position of qualitative evaluation research within randomized controlled trials are destined to be a point of contention rather than a source of insight.

Ecological theory in practice: illustrations from a community-based intervention to promote the health of recent mothers.

We present a qualitative case study where we used four principles of ecological theory from community psychology as a template to assess the dynamics about how a preventive community intervention was transacted in eight communities in Victoria, Australia. The principles were cycling of resources, interdependence, adaptation, and succession. Ecological thinking focuses on key resources in communities. That is, the people, events, and settings that are the foundations of thinking about communities as systems. The data set consists of field diaries kept by and serial interviews with nine community development workers over a 2-year period. We found that the analysis highlighted a process-oriented way of representing the intervention, one that sees beyond the interventions technical components (or packaged elements) to the complexities of the cultural and political change processes occurring beneath. The value of this is the attention focussed on likely project sustainability.

A Review of Programs That Targeted Environmental Determinants of Aboriginal and Torres Strait Islander Health

Objective: Effective interventions to improve population and individual health require environmental change as well as strategies that target individual behaviours and clinical factors. This is the basis of implementing an ecological approach to health programs and health promotion. For Aboriginal People and Torres Strait Islanders, colonisation has made the physical and social environment particularly detrimental for health. Methods and Results: We conducted a literature review to identify Aboriginal health interventions that targeted environmental determinants of health, identifying 21 different health programs. Program activities that targeted environmental determinants of health included: Caring for Country; changes to food supply and/or policy; infrastructure for physical activity; housing construction and maintenance; anti-smoking policies; increased workforce capacity; continuous quality improvement of clinical systems; petrol substitution; and income management. Targets were categorised according to Miller’s Living Systems Theory. Researchers using an Indigenous community based perspective more often identified interpersonal and community-level targets than were identified using a Western academic perspective. Conclusions: Although there are relatively few papers describing interventions that target environmental determinants of health, many of these addressed such determinants at multiple levels, consistent to some degree with an ecological approach. Interpretation of program targets sometimes differed between academic and community-based perspectives, and was limited by the type of data reported in the journal articles, highlighting the need for local Indigenous knowledge for accurate program evaluation. Implications: While an ecological approach to Indigenous health is increasingly evident in the health research literature, the design and evaluation of such programs requires a wide breadth of expertise, including local Indigenous knowledge.

The Australian Government's ‘Social Inclusion Agenda’: the intersection between public health and social policy

Social exclusion and social inclusion have been popular policy themes in the UK and Europe, and made more modest appearances in countries such as Canada and New Zealand, for over a decade. In 2007, the Australian Government became the latest country in this trend of structuring social policy around issues of exclusion, launching its ‘Social Inclusion Agenda’ (SIA). The SIA aims to increase social and economic participation through a reorientation of social services and increased attention to issues of equity. At the same time, there has been international consensus on the importance of the social determinants of health (such as education, income and gender) for individuals and populations. The SIA has the potential to make a substantive contribution to the social determinants of health and hence the health and wellbeing of the population. However, much will depend on the extent to which international discourses of inclusion, exclusion, structural inequality and third way politics are taken up, adapted or discarded in the Australian context. At this formative stage of the SIAs development, the public health community is in a unique position to contribute to the development and direction of the SIA to secure the potential health gains it offers. This article outlines the formulation of social inclusion policy in Australia, and discusses the potential promises and pitfalls of a social inclusion approach. Our examination of the Australian experience of social inclusion policy provides an opportunity to reflect on the relationship between social inclusion and health internationally.

Top-Down Approaches to Joined-Up Government: Examining the Unintended Consequences of Weak Implementation

Since forming part of Blair’s modernization agenda in the UK, joined-up government has become a central ambition of governments in many industrialized countries. While there continues to be an absence of core methods and principles for achieving joined-up government, consensus has emerged around the effectiveness of top down approaches. Research has found that joining must happen at multiple levels and be supported by a range of cultural and structural interventions. This article presents findings from a study into the Australian Social Inclusion Agenda and explores the long-term and unintended consequences of joined-up initiatives that take a top-down approach.

In response to Spillane V., Byrne M.C., Byrne M., Leathem C.S., O’Malley M. & Cupples M.E. (2007) Monitoring treatment fidelity in a randomized trial of a complex intervention. Journal of Advanced Nursing60(3), 343–352.

Spillane et al. (2007) outline in their JAN paper the methods they used to monitor (and presumably promote) intervention fidelity within a trial of a ‘complex’ intervention to encourage medication adherence and lifestyle change among heart disease patients. The study involved 48 general practices. The authors misquote our own work in complex, multi-site cluster-randomized trials. Given that the point is germane to their argument, we would like to correct this. We are cited as saying that trials of complex interventions ‘must strive to consistently implement the same intervention by standardizing its content and delivery’. Actually what we said was that something has to be standardized in a complex intervention, for the sake of both internal and external validity (Hawe et al. 2004), but it does not have to be the content of the intervention, nor the way it is delivered. In our paper, we lamented the tendency for investigators to feel that the form of an intervention had to remain the same in every site, mistakenly believing that this was an essential requirement of a randomized trial. We felt that this over-controlled the intervention. In some cases, standardizing interventions by form (e.g. using exactly the same patient education materials in every site) might be going too far in ‘treatment fidelity’, when tailoring to context might be more effective (e.g. allowing materials to be adapted to local cultural styles and literacy levels). In the latter case, the function of an intervention component would remain the same, but the form could be different in different sites. We argued that standardizing by the function that a component plays in an intervention, instead of standardizing by the form it takes, may be more appropriate in many complex interventions. Indeed, the more complex an intervention becomes, the more it is necessary to have rigorous theory about the process and principles of the change process being tested, but to be flexible about the form that this takes in each site. Anyone versed in fields like active listening or community development, for example, would appreciate that it is impossible to prescribe the exact form the intervention takes, but the process, principles and critical sequences ought to be recognizable and replicable. Thus, we would suggest that both the content and the delivery of Spillane’s SPHERE intervention could differ among the 24 interventions sites, but the integrity of the intervention could be fully preserved if the principles of training, delivery and enactment of skills (as they have specified) coincide with the theory the intervention is testing.

Habitus and responsible dog-ownership: reconsidering the health promotion implications of ‘dog-shaped’ holes in people’s lives

Responsible dog ownership has been identified as a point of intervention to promote physical activity, based upon an expectation of dog walking in public space. Nevertheless, quantitative research has found variability among owners in their dog walking. In this study, we explore the implications for health promotion of such variability. We do so by drawing on the concepts of habitus and social capital to analyse qualitative interviews. Participants were recruited from a social network in a cosmopolitan city with a policy framework intended to ensure equitable access to public space for dog walkers. The analysis confirms dog ownership can promote both physical activity and social capital, to the extent of mutual reinforcement. Yet we identified patterns of care in which dogs could influence people’s emotional well-being without promoting physical activity. In particular, some owners were not capable of extensive dog walking but still benefited emotionally from dog ownership and from interpersonal interactions facilitated by dog ownership. Some participants’ dogs, however, could not be walked in public without risking public safety and social sanctions. Responsible dog ownership can therefore also entail not exercising dogs. Contra to the emerging ideal in health promotion, a ‘dog-shaped hole’ in someone’s life does not always take the form of a walking companion.