Hanhua Liu

Evaluation and validation of social and psychological markers in randomised trials of complex interventions in mental health: a methodological research programme.

BACKGROUND The development of the capability and capacity to evaluate the outcomes of trials of complex interventions is a key priority of the National Institute for Health Research (NIHR) and the Medical Research Council (MRC). The evaluation of complex treatment programmes for mental illness (e.g. cognitive-behavioural therapy for depression or psychosis) not only is a vital component of this research in its own right but also provides a well-established model for the evaluation of complex interventions in other clinical areas. In the context of efficacy and mechanism evaluation (EME) there is a particular need for robust methods for making valid causal inference in explanatory analyses of the mechanisms of treatment-induced change in clinical outcomes in randomised clinical trials. OBJECTIVES The key objective was to produce statistical methods to enable trial investigators to make valid causal inferences about the mechanisms of treatment-induced change in these clinical outcomes. The primary objective of this report is to disseminate this methodology, aiming specifically at trial practitioners. METHODS The three components of the research were (1) the extension of instrumental variable (IV) methods to latent growth curve models and growth mixture models for repeated-measures data; (2) the development of designs and regression methods for parallel trials; and (3) the evaluation of the sensitivity/robustness of findings to the assumptions necessary for model identifiability. We illustrate our methods with applications from psychological and psychosocial intervention trials, keeping the technical details to a minimum, leaving the reporting of the more theoretical and mathematically demanding results for publication in appropriate specialist journals. RESULTS We show how to estimate treatment effects and introduce methods for EME. We explain the use of IV methods and principal stratification to evaluate the role of putative treatment effect mediators and therapeutic process measures. These results are extended to the analysis of longitudinal data structures. We consider the design of EME trials. We focus on designs to create convincing IVs, bearing in mind assumptions needed to attain model identifiability. A key area of application that has become apparent during this work is the potential role of treatment moderators (predictive markers) in the evaluation of treatment effect mechanisms for personalised therapies (stratified medicine). We consider the role of targeted therapies and multiarm trials and the use of parallel trials to help elucidate the evaluation of mediators working in parallel. CONCLUSIONS In order to demonstrate both efficacy and mechanism, it is necessary to (1) demonstrate a treatment effect on the primary (clinical) outcome, (2) demonstrate a treatment effect on the putative mediator (mechanism) and (3) demonstrate a causal effect from the mediator to the outcome. Appropriate regression models should be applied for (3) or alternative IV procedures, which account for unmeasured confounding, provided that a valid instrument can be identified. Stratified medicine may provide a setting where such instruments can be designed into the trial. This work could be extended by considering improved trial designs, sample size considerations and measurement properties. FUNDING The project presents independent research funded under the MRC-NIHR Methodology Research Programme (grant reference G0900678).

Mobilizing citizen effort to enhance environmental outcomes: A randomized controlled trial of a door-to-door recycling campaign

This paper uses a randomized controlled trial to test whether doorstep canvassing can raise participation in kerbside recycling. Existing research shows that canvassing can confront negative attitudes, increase understanding and resolve structural obstacles, but there is less known about the longitudinal effects of such interventions, which may fall away over time. 194 streets in Trafford, in the North West of England, UK were randomly assigned into a treatment and a control group. All households in the treatment group were visited by canvassers who were trained to promote and encourage recycling. Recycling participation rates for all households were measured by observing bin set out rates over a three-week period. Measurement was done before and after the canvassing campaign and then again three months later to see if the intervention had been effective in raising participation rates. Random-effects multilevel regression models, controlling for baseline recycling, street size, deprivation and size of ethnic minority population, show that the canvassing raised recycling participation rates for the treatment group compared to the control group, but there was a decline in the impact of the intervention over time. The intervention was more effective on streets with low levels of recycling at baseline.

How Civic is the Civic Culture? Explaining Community Participation Using the 2005 English Citizenship Survey

Governments increasingly seek to involve citizens in public policy and management, often appealing to their civic virtue. But why do people participate in civic and community-based actions? Drawing on theories of interpersonal behaviour, the article sets out four categories of citizen orientation that might influence participation: trust in government institutions, moral motivations, neighbourhood social norms and neighbourhood affect. Using the core sample component of the Home Office Citizenship Survey 2005, the analysis applies structural equation models (SEMs) to identify and explain four types of citizen act: influencing institutions individually, collective civic, citizen governance and community voluntarism. The results show that neighbourhood affect – having positive feelings about the neighbourhood – has a positive effect on civic behaviour. Citizens with low levels of political trust are more likely than others to engage in civic behaviour. Taking into account a range of socio-economic and other factors, there is no significant effect of neighbourhood social norms and moral motivations on civic behaviour.

Integrating biomarker information within trials to evaluate treatment mechanisms and efficacy for personalised medicine

Background The development of personalised (stratified) medicine is intrinsically dependent on an understanding of treatment-effect mechanisms (effects on therapeutic targets that mediate the effect of the treatment on clinical outcomes). There is a need for clinical trial data for the joint evaluation of treatment efficacy, the utility of predictive markers as indicators of treatment efficacy, and the mediational mechanisms proposed as the explanation of these effects. Purpose (1) To review the problem of confounding (common causes) for the drawing of valid inferences concerning treatment-effect mechanisms, even when the data have been generated using a randomised controlled trial, and (2) to suggest and illustrate solutions to this problem of confounding. Results We illustrate the potential of the predictive biomarker stratified design, together with baseline measurement of all known prognostic markers, to enable us to evaluate both the utility of the predictive biomarker in such a stratification and, perhaps more importantly, to estimate how much of the treatment’s effect is actually explained by changes in the putative mediator. The analysis strategy involves the use of instrumental variable (IV) regression, using the treatment by predictive biomarker interaction as an IV – a refined, much more powerful, and (in the present context) subtle use of Mendelian randomisation. Conclusion Personalised (stratified) medicine and treatment-effect mechanisms evaluation are inextricably linked. Stratification without corresponding mechanisms evaluation lacks credibility. In the presence of mediator-outcome confounding, mechanisms evaluation is dependent on stratification for its validity. Both stratification and treatment-effect mediation can be evaluated using a biomarker stratified trial design together with detailed baseline measurement of all known prognostic biomarkers and other prognostic covariates. Direct and indirect (mediated) effects should be estimated through the use of IV methods (the IV being the predictive marker by treatment interaction) together with adjustments for all known prognostic markers (confounders) – the latter adjustments contributing to increased precision (as in a conventional analysis of treatment effects) rather than bias reduction.

Who do treatments work for and why? Understanding treatment-effect mechanisms in stratified medicine

The development of stratified medicine depends on an understanding of treatment-effect mechanisms (effects on therapeutic targets that mediate the effect of the treatment on clinical outcomes). Yet the evaluation of these mechanisms is often absent from the design and analysis of studies for stratified medicine, and even if present, is subject to unmeasured confounding. We review the problem of confounding (common causes) for the drawing of valid inferences concerning treatment-effect mechanisms, even when the data has been generated using a randomised controlled trial. We illustrate the potential of the predictive biomarker-stratified trial design, together with baseline measurement of all known prognostic markers, to enable us to evaluate both the utility of the predictive biomarker in such a stratification and to estimate how much of the treatments effect is actually explained by changes in the putative mediator. We call this a biomarker-stratified efficacy and mechanisms evaluation (BS-EME) trial design. The analysis strategy involves the use of instrumental variable estimation methods, using the treatment by predictive biomarker interaction as an instrumental variable together with adjustments for all know prognostic markers; the latter contributing to increased precision (as in a conventional analysis of treatment effects) rather than bias reduction. The analysis approach provides unbiased estimates even in the presence of unmeasured confounding. We conclude that stratification without corresponding mechanisms evaluation lacks credibility and in the almost certain presence of mediator-outcome confounding, mechanisms evaluation is dependent on stratification for its validity. Our trial design and analysis approach evaluates both stratification and treatment-effect mediation.

An Assessment of Radiation-Associated Risks of Mortality from Circulatory Disease in the Cohorts of Mayak and Sellafield Nuclear Workers

Mortality from circulatory disease (CD), ischemic heart disease (IHD) and cerebrovascular disease (CeVD) was investigated in relationship to cumulative doses of external gamma radiation and internal alpha radiation to the liver from deposited plutonium over long follow-up periods in two large cohorts of nuclear workers: the Russian Mayak Worker Cohort (MWC) and the UK Sellafield Worker Cohort (SWC). The MWC comprised 22,374 workers (74.6% males) with 5,123 CD deaths registered during 842,538 person-years of follow-up, while the SWC comprised 23,443 workers (87.8% males) with 2,322 CD deaths registered during 602,311 person-years of follow-up. Dose estimates for external gamma radiation and internal alpha radiation to the liver were calculated via a common methodology, in accordance with an agreed protocol. The mean cumulative external Hp(10) dose was 0.52 Sv for the MWC and 0.07 Sv for the SWC, while the mean cumulative internal dose was 0.19 Gy for the MWC and 0.01 Gy for the SWC. Categorical relative risks (RR) and excess relative risks (ERR) per unit dose were estimated for each cohort and for the pooled cohort when appropriate. The dose responses for CD, IHD and CeVD in relationship to internal alpha-particle dose did not differ significantly from the null for either the MWC, the SWC or the pooled plutonium worker cohort. The ERR/Sv estimates in relationship to external exposure were significantly raised for both cohorts (marginally so for the MWC) for CD and IHD (but not for CeVD), but differed significantly between the two cohorts, the estimate for the SWC being approximately ten times greater than that for the MWC. Examination of the ERR/Sv estimates for two periods of first employment at the two facilities revealed that the significant heterogeneity was confined to the earlier sub-cohorts, and that the estimates for the later sub-cohorts were compatible. The two sub-cohorts for the later first-employment periods were pooled, producing risk estimates that were raised, but not significantly so: ERR/Sv for CD, IHD and CeVD of 0.22 (95% CI: –0.01, 0.49), 0.22 (95% CI: –0.06, 0.57) and 0.24 (95% CI: –0.17, 0.80), respectively. The reasons for the complex pattern of results found in this study are unclear. Among potential explanations are the influence of differences in background CD mortality rates, an effect of other occupational factors, substantial uncertainties in doses, particularly during earlier periods of operations, as well as confounding and/or modifying factors that were not taken into account in the current analysis.

A review of job-exposure matrix methodology for application to workers exposed to radiation from internally deposited plutonium or other radioactive materials

Any potential health effects of radiation emitted from radionuclides deposited in the bodies of workers exposed to radioactive materials can be directly investigated through epidemiological studies. However, estimates of radionuclide exposure and consequent tissue-specific doses, particularly for early workers for whom monitoring was relatively crude but exposures tended to be highest, can be uncertain, limiting the accuracy of risk estimates. We review the use of job-exposure matrices (JEMs) in peer-reviewed epidemiological and exposure assessment studies of nuclear industry workers exposed to radioactive materials as a method for addressing gaps in exposure data, and discuss methodology and comparability between studies. We identified nine studies of nuclear worker cohorts in France, Russia, the USA and the UK that had incorporated JEMs in their exposure assessments. All these JEMs were study or cohort-specific, and although broadly comparable methodologies were used in their construction, this is insufficient to enable the transfer of any one JEM to another study. Moreover there was often inadequate detail on whether, or how, JEMs were validated. JEMs have become more detailed and more quantitative, and this trend may eventually enable better comparison across, and the pooling of, studies. We conclude that JEMs have been shown to be a valuable exposure assessment methodology for imputation of missing exposure data for nuclear worker cohorts with data not missing at random. The next step forward for direct comparison or pooled analysis of complete cohorts would be the use of transparent and transferable methods.

0200 Creation of a quantitative historical job-exposure matrix for plutonium workers and feasibility of its use with reconstructed occupational histories for epidemiological purposes

Introduction The UK Sellafield workforce is important for studying potential health risks of plutonium (Pu) exposure. However, several hundred early workers, employed during the period 1952–63, have been excluded from epidemiological studies because their urinalysis results were insufficiently reliable to provide accurate exposure assessment. This project aimed to develop a job-exposure matrix (JEM) that would enable future inclusion of these workers in epidemiological studies. Methods 630 plutonium workers without reliable Pu urinalysis data for 1952–63 were identified within fourteen ‘homogeneous’ plutonium exposure groups. For each job/work location/year, ‘exposure analogues’ with reliable urinalysis data were identified (n=330). The JEM was based on 4487 work history records and 6899 urinalysis results. Intake assessments were produced using the ‘PuMA’ plutonium mass assessment code employing the latest conventional assessment methodology. Results The JEM provided estimates for the median plutonium intake in becquerel (Bq) per year for each job/work location/year combination, and ranged from ”no intake” to 175 Bq/yr. Cumulative plutonium intakes for these workers ranged from ”no intake” to 990 Bq. Internal cross-validation indicated moderate-to-good correlations (r>0.4) and relative differences between JEM and validation sample <10%. Probabilistic evaluation indicated robust estimates of cumulative intake. Median cumulative JEM intake was 50 times lower than for conventional assessment methodology and much better aligned with prior expectation. Conclusions The ‘exposure analogues’ methodology in JEM-development is a novel approach and has the potential to be a valuable tool for future epidemiological studies of the risks that may arise from plutonium exposure at Sellafield and potentially other similar cohorts. Declaration of potential conflict of interest: Dr MacGregor, Mrs Wilson, Mr Peace and Mr Herdnan are employed by Sellafield Ltd. Professor Wakeford does consultancy work, including for the UK Compensation Scheme for Radiation-linked Diseases. The authors declare that they otherwise have no actual or potential competing financial interests.