Julie Barber

How should cost data in pragmatic randomised trials be analysed

Pragmatic randomised trials are usually large scale multicentre studies in which interventions or medical policies are compared in a realistic setting.1 The intention is that conclusions from these trials, if accepted, can be adopted directly into medical practice.2 Economic evaluations carried out alongside these trials are increasingly common because it is often important to assess costs and cost effectiveness as well as clinical outcomes.3 Costs are usually derived from information about the quantity of healthcare resources used by each patient in the trial. The quantities of each resource used are multiplied by fixed unit cost values and are then summed over the separate types of resource to give a total cost per patient.4 Distribution of costs from a trial comparing endometrial resection with hysterectomy in women with menorrhagia. Costs are based on health resource use from randomisation to two years; they include preoperative, operative, hospital stay, complications, retreatment, and primary care components5 This information leads to a range of different costs across participants in the trial. As an example, the figure shows the distribution of costs in women with menorrhagia randomised to treatment with endometrial resection or abdominal hysterectomy.5 Such highly skewed distributions are typical of cost data; the long right hand tail reflects the fact that some patients incur high costs because of factors such as medical complications, reoperation, or extended hospital stay. ### Summary points Health economic evaluations are now commonly included in pragmatic clinical trials that inform policy decisions Despite the usual skewness in the distribution of costs, it is the arithmetic mean that is the most informative measure Measures other than the arithmetic mean do not provide information about the cost of treating all patients, which is needed as the basis for healthcare policy decisions Statistical analysis based on transforming cost data or …

Analysis of cost data in randomized trials: an application of the non-parametric bootstrap.

Health economic evaluations are now more commonly being included in pragmatic randomized trials. However a variety of methods are being used for the presentation and analysis of the resulting cost data, and in many cases the approaches taken are inappropriate. In order to inform health care policy decisions, analysis needs to focus on arithmetic mean costs, since these will reflect the total cost of treating all patients with the disease. Thus, despite the often highly skewed distribution of cost data, standard non-parametric methods or use of normalizing transformations are not appropriate. Although standard parametric methods of comparing arithmetic means may be robust to non-normality for some data sets, this is not guaranteed. While the randomization test can be used to overcome assumptions of normality, its use for comparing means is still restricted by the need for similarly shaped distributions in the two groups. In this paper we show how the non-parametric bootstrap provides a more flexible alternative for comparing arithmetic mean costs between randomized groups, avoiding the assumptions which limit other methods. Details of several bootstrap methods for hypothesis tests and confidence intervals are described and applied to cost data from two randomized trials. The preferred bootstrap approaches are the bootstrap-t or variance stabilized bootstrap-t and the bias corrected and accelerated percentile methods. We conclude that such bootstrap techniques can be recommended either as a check on the robustness of standard parametric methods, or to provide the primary statistical analysis when making inferences about arithmetic means for moderately sized samples of highly skewed data such as costs.

Multiple regression of cost data: use of generalised linear models.

Objective: Choosing an appropriate method for regression analyses of cost data is problematic because it must focus on population means while taking into account the typically skewed distribution of the data. In this paper we illustrate the use of generalised linear models for regression analysis of cost data. Methods: We consider generalised linear models with either an identity link function (providing additive covariate effects) or log link function (providing multiplicative effects), and with gaussian (normal), overdispersed poisson, gamma, or inverse gaussian distributions. These are applied to estimate the treatment effects in two randomised trials adjusted for baseline covariates. Criteria for choosing an appropriate model are presented. Results: In both examples considered, the gaussian model fits poorly and other distributions are to be preferred. When there are variables of prognostic importance in the model, using different distributions can materially affect the estimates obtained; it may also be possible to discriminate between additive and multiplicative covariate effects. Conclusions: Generalised linear models are attractive for the regression of cost data because they provide parametric methods of analysis where a variety of non-normal distributions can be specified and the way covariates act can be altered. Unlike the use of data transformation in ordinary least-squares regression, generalised linear models make inferences about the mean cost directly.

Analysis and interpretation of cost data in randomised controlled trials: review of published studies

Objective To review critically the statistical methods used for health economic evaluations in randomised controlled trials where an estimate of cost is available for each patient in the study. Design Survey of published randomised trials including an economic evaluation with cost values suitable for statistical analysis; 45 such trials published in 1995 were identified from Medline. Main outcome measures The use of statistical methods for cost data was assessed interms of the descriptive statistics reported, use of statistical inference, and whether the reported conclusions were justified. Results Although all 45 trials reviewed apparently had cost data for each patient, only 9 (20%) reported adequate measures of variability for these data and only 25 (56%) gave results of statistical tests or a measure of precision for the comparison of costs between the randomised groups. Only 16 (36%) of the articles gave conclusions which were justified on the basis of results presented in the paper. No paper reported sample size calculations for costs Conclusions The analysis and interpretation of cost data from published trials reveal a lack of statistical awareness. Strong and potentially misleading conclusions about the relative costs of alternative therapies have often been reported in the absence of supporting statistical evidence. Improvements in the analysis and reporting of health economic assessments are urgently required. Health economic guidelines need to be revised to incorporate more detailed statistical advice.

Clinical effectiveness of a manual based coping strategy programme (START, STrAtegies for RelaTives) in promoting the mental health of carers of family members with dementia: pragmatic randomised controlled trial

Objective To assess whether a manual based coping strategy compared with treatment as usual reduces depression and anxiety symptoms in carers of family members with dementia. Design Randomised, parallel group, superiority trial. Setting Three mental health community services and one neurological outpatient dementia service in London and Essex, UK. Participants 260 carers of family members with dementia. Intervention A manual based coping intervention comprising eight sessions and delivered by supervised psychology graduates to carers of family members with dementia. The programme consisted of psychoeducation about dementia, carers’ stress, and where to get emotional support; understanding behaviours of the family member being cared for, and behavioural management techniques; changing unhelpful thoughts; promoting acceptance; assertive communication; relaxation; planning for the future; increasing pleasant activities; and maintaining skills learnt. Carers practised these techniques at home, using the manual and relaxation CDs. Main outcome measures Affective symptoms (hospital anxiety and depression total score) at four and eight months. Secondary outcomes were depression and anxiety caseness on the hospital anxiety and depression scale; quality of life of both the carer (health status questionnaire, mental health) and the recipient of care (quality of life-Alzheimer’s disease); and potentially abusive behaviour by the carer towards the recipient of care (modified conflict tactics scale). Results 260 carers were recruited; 173 were randomised to the intervention and 87 to treatment as usual. Mean total scores on the hospital anxiety and depression scale were lower in the intervention group than in the treatment as usual group over the eight month evaluation period: adjusted difference in means −1.80 points (95% confidence interval −3.29 to −0.31; P=0.02) and absolute difference in means −2.0 points. Carers in the intervention group were less likely to have case level depression (odds ratio 0.24, 95% confidence interval 0.07 to 0.76) and there was a non-significant trend towards reduced case level anxiety (0.30, 0.08 to 1.05). Carers’ quality of life was higher in the intervention group (difference in means 4.09, 95% confidence interval 0.34 to 7.83) but not for the recipient of care (difference in means 0.59, −0.72 to 1.89). Carers in the intervention group reported less abusive behaviour towards the recipient of care compared with those in the treatment as usual group (odds ratio 0.47, 95% confidence interval 0.18 to 1.23), although this was not significant. Conclusions A manual based coping strategy was effective in reducing affective symptoms and case level depression in carers of family members with dementia. The carers’ quality of life also improved. Trial registration Current Controlled Trials ISCTRN70017938.

Virtual outreach: economic evaluation of joint teleconsultations for patients referred by their general practitioner for a specialist opinion

Abstract Objectives To test the hypotheses that, compared with conventional outpatient consultations, joint teleconsultation (virtual outreach) would incur no increased costs to the NHS, reduce costs to patients, and reduce absences from work by patients and their carers. Design Cost consequences study alongside randomised controlled trial. Setting Two hospitals in London and Shrewsbury and 29 general practices in inner London and Wales. Participants 3170 patients identified; 2094 eligible for inclusion and willing to participate. 1051 randomised to virtual outreach and 1043 to standard outpatient appointments. Main outcome measures NHS costs, patient costs, health status (SF-12), time spent attending index consultation, patient satisfaction. Results Overall six months costs were greater for the virtual outreach consultations (£724 per patient) than for conventional outpatient appointments (£625): difference in means £99 (

START (STrAtegies for RelaTives) study: a pragmatic randomised controlled trial to determine the clinical effectiveness and cost-effectiveness of a manual-based coping strategy programme in promoting the mental health of carers of people with dementia

162; €138) (95% confidence interval £10 to £187, P=0.03). If the analysis is restricted to resource items deemed “attributable” to the index consultation, six month costs were still greater for virtual outreach: difference in means £108 (£73 to £142, P < 0.0001). In both analyses the index consultation accounted for the excess cost. Savings to patients in terms of costs and time occurred in both centres: difference in mean total patient cost £8 (£5 to £10, P < 0.0001). Loss of productive time was less in the virtual outreach group: difference in mean cost £11 (£10 to £12, P < 0.0001). Conclusion The main hypothesis that virtual outreach would be cost neutral is rejected, but the hypotheses that costs to patients and losses in productivity would be lower are supported.

Cost effectiveness of a manual based coping strategy programme in promoting the mental health of family carers of people with dementia (the START (STrAtegies for RelaTives) study): a pragmatic randomised controlled trial

BACKGROUND Two-thirds of people with dementia live at home, receiving most care from family carers, about 40% of whom have clinically significant depression or anxiety. This impacts on the person with dementia, families and society, predicting care breakdown. There are currently no clinically effective and cost-effective NHS family carer interventions. OBJECTIVES To assess the STrAtegies for RelaTives (START) intervention in the short (4 and 8 months) and long term (1 and 2 years) compared with treatment as usual (TAU). DESIGN Randomised, parallel-group, superiority trial with blinded assessment recruiting participants 2:1 (intervention to TAU) to allow for therapist clustering. SETTING Three UK mental health services and one neurological service. PARTICIPANTS Family carers of people with dementia. INTERVENTION Eight-session manual-based coping intervention delivered by supervised psychology graduates to individuals. MAIN OUTCOME MEASURES Affective symptoms [Hospital Anxiety and Depression Scale-total (HADS-T)] and cost-effectiveness. Secondary measures: anxiety and depression symptoms and caseness, quality of life (QoL), abusive behaviour and long-term care home admission. RESULTS Two hundred and sixty participants were randomised (173 intervention, 87 TAU). We used intention-to-treat analysis in the short term (152 intervention, 77 TAU) and in the long term (140 intervention, 69 TAU). In the short term, the intervention group had lower HADS-T [mean difference -1.80, 95% confidence interval (CI) -3.29 to -0.31; p=0.02] and higher quality-adjusted life-years (QALYs) (mean difference 0.03, 95% CI -0.01 to 0.08). Costs were no different between groups [mean £ 252 (95% CI -£ 28 to £ 565) for intervention group]. The cost-effectiveness acceptability curve showed a greater than 99% chance of being cost-effectiveness at a £ 30,000/QALY willingness-to-pay threshold and a high probability of cost-effectiveness based on the HADS-T score. Carers in the intervention group had less case-level depression [odds ratio (OR) 0.24, 95% CI 0.07 to 0.76], a trend towards reduced case-level anxiety (OR 0.30, 95% CI 0.08 to 1.05), lower Hospital Anxiety and Depression Scale-anxiety (HADS-A) (-0.91, 95% CI -1.76 to -0.07; p = 0.03) and Hospital Anxiety and Depression Scale-depression (HADS-D) (-0.91, 95% CI -1.71 to -0.10; p = 0.03) and higher Health Status Questionnaire (HSQ) QoL (mean difference 4.09, 95% CI 0.34 to 7.83). Group differences in abusive behaviour (OR 0.48, 95% CI 0.18 to 1.27) and the person with dementias quality of life-Alzheimers disease (QoL-AD) (mean increase 0.59, 95% CI -0.72 to 1.89) were not significant. In the long term, the intervention group had lower HADS-T (mean difference -2.58, 95% CI -4.26 to -0.90; p = 0.03) and higher QALYs (mean difference 0.03, 95% CI -0.01 to 0.06). Carers in the intervention group had less case-level depression (OR 0.14, 95% CI 0.04 to 0.53), a trend towards reduced case-level anxiety (OR 0.57, 95% CI 0.26 to 1.24), lower HADS-A (-1.16, 95% CI -2.15 to -0.18) and HADS-D (1.45, 95% CI -2.32 to -0.57), and higher HSQ (mean difference 7.47, 95% CI 2.87 to 12.08). Thirty-two (18.7%) people with dementia in the intervention group and 17 (20.2%) in TAU were admitted to a care home (hazard ratio 0.83, 95% CI 0.44 to 1.56; p = 0.56). There were no significant differences between groups in abusive behaviour (OR 0.83, 95% CI 0.36 to 1.94), the person with dementias QoL-AD (0.17, 95% CI -1.37 to 1.70) or costs (£ 336, 95% CI -£ 223 to £ 895) for intervention group. The probability that the intervention would be seen as cost-effective at £ 30,000/QALY threshold and cost-effectiveness on the HADS-T remained high. CONCLUSIONS The START intervention was clinically effective and cost-effective in the short and longer term. The results are robust to the sensitivity analyses performed. Future work is needed to consider mechanism of action; the effects on people with dementia in clinical terms (cognition, neuropsychiatric symptoms, longer-term care home admission); and on health and social care costs. In addition, we will explore the effects of carer abusive behaviour on the care recipients care home admission and if this then reduces abusive behaviour. We would also like to implement START and evaluate this implementation in clinical practice. TRIAL REGISTRATION Current Controlled Trials ISCTRN70017938.

Design and performance of a multi-centre randomised controlled trial and economic evaluation of joint tele-consultations (ISRCTN54264250)

Objective To assess whether the START (STrAtegies for RelatTives) intervention added to treatment as usual is cost effective compared with usual treatment alone. Design Cost effectiveness analysis nested within a pragmatic randomised controlled trial. Setting Three mental health and one neurological outpatient dementia service in London and Essex, UK. Participants Family carers of people with dementia. Intervention Eight session, manual based, coping intervention delivered by supervised psychology graduates to family carers of people with dementia added to usual treatment, compared with usual treatment alone. Primary outcome measures Costs measured from a health and social care perspective were analysed alongside the Hospital Anxiety and Depression Scale total score (HADS-T) of affective symptoms and quality adjusted life years (QALYs) in cost effectiveness analyses over eight months from baseline. Results Of the 260 participants recruited to the study, 173 were randomised to the START intervention, and 87 to usual treatment alone. Mean HADS-T scores were lower in the intervention group than the usual treatment group over the 8 month evaluation period (mean difference −1.79 (95% CI −3.32 to −0.33)), indicating better outcomes associated with the START intervention. There was a small improvement in health related quality of life as measured by QALYs (0.03 (−0.01 to 0.08)). Costs were no different between the intervention and usual treatment groups (£252 (−28 to 565) higher for START group). The cost effectiveness calculations suggested that START had a greater than 99% chance of being cost effective compared with usual treatment alone at a willingness to pay threshold of £30 000 per QALY gained, and a high probability of cost effectiveness on the HADS-T measure. Conclusions The manual based coping intervention START, when added to treatment as usual, was cost effective compared with treatment as usual alone by reference to both outcome measures (affective symptoms for family carers, and carer based QALYs). Trial Registration ISCTRN 70017938

Development and validation of the BRIGHTLIGHT Survey, a patient-reported experience measure for young people with cancer

BackgroundAppropriate information flow is crucial to the care of patients, particularly at the interface between primary and secondary care. Communication problems can result from inadequate organisation and training, There is a major expectation that information and communication technologies may offer solutions, but little reliable evidence. This paper reports the design and performance of a multi-centre randomised controlled trial (RCT), unparalleled in telemedicine research in either scale or range of outcomes. The study investigated the effectiveness and cost implications in rural and inner-city settings of using videoconferencing to perform joint tele-consultations as an alternative to general practitioner referral to the hospital specialist in the outpatient clinic.MethodsJoint tele-consultation services were established in both the Royal Free Hampstead NHS Trust in inner London, and the Royal Shrewsbury Hospitals Trust, in Shropshire. All the patients who gave consent to participate were randomised either to joint tele-consultation or to a routine outpatients appointment. The principal outcome measures included the frequency of decision by the specialist to offer a follow-up outpatient appointment, patient satisfaction (Ware Specific Questionnaire), wellbeing (SF12) and enablement (PEI), numbers of tests, investigations, procedures and treatments.ResultsA total of 134 general practitioners operating from 29 practices participated in the trial, referring a total of 3170 patients to 20 specialists in ENT medicine, general medicine (including endocrinology, and rheumatology), gastroenterology, orthopaedics, neurology and urology. Of these, 2094 patients consented to participate in the study and were correctly randomised. There was a 91% response rate to the initial assessment questionnaires, and analysis showed equivalence for all key characteristics between the treatment and control groups.ConclusionWe have designed and performed a major multi-centre trial of teleconsultations in two contrasting centres. Many problems were overcome to enable the trial to be carried out, with a considerable development and learning phase. A lengthier development phase might have enabled us to improve the patient selection criteria, but there is a window of opportunity for these developments, and we believe that our approach was appropriate, allowing the evaluation of the technology before its widespread implementation.