Gareth Ambler

A comparison of imputation techniques for handling missing predictor values in a risk model with a binary outcome

Risk models that aim to predict the future course and outcome of disease processes are increasingly used in health research, and it is important that they are accurate and reliable. Most of these risk models are fitted using routinely collected data in hospitals or general practices. Clinical outcomes such as short-term mortality will be near-complete, but many of the predictors may have missing values. A common approach to dealing with this is to perform a complete-case analysis. However, this may lead to overfitted models and biased estimates if entire patient subgroups are excluded. The aim of this paper is to investigate a number of methods for imputing missing data to evaluate their effect on risk model estimation and the reliability of the predictions. Multiple imputation methods, including hotdecking and multiple imputation by chained equations (MICE), were investigated along with several single imputation methods. A large national cardiac surgery database was used to create simulated yet realistic datasets. The results suggest that complete case analysis may produce unreliable risk predictions and should be avoided. Conditional mean imputation performed well in our scenario, but may not be appropriate if using variable selection methods. MICE was amongst the best performing multiple imputation methods with regards to the quality of the predictions. Additionally, it produced the least biased estimates, with good coverage, and hence is recommended for use in practice.

Intravenous immunoglobulin treatment of the complex regional pain syndrome: a randomized trial.

BACKGROUND Treatment of long-standing complex regional pain syndrome (CRPS) is empirical and often of limited efficacy. Preliminary data suggest that the immune system is involved in sustaining this condition and that treatment with low-dose intravenous immunoglobulin (IVIG) may substantially reduce pain in some patients. OBJECTIVE To evaluate the efficacy of IVIG in patients with longstanding CRPS under randomized, controlled conditions. DESIGN A randomized, double-blind, placebo-controlled crossover trial. (National Research Registry number: N0263177713; International Standard Randomised Controlled Trial Number Registry: 63918259) SETTING University College London Hospitals Pain Management Centre. PATIENTS Persons who had pain intensity greater than 4 on an 11-point (0 to 10) numerical rating scale and had CRPS for 6 to 30 months that was refractory to standard treatment. INTERVENTION IVIG, 0.5 g/kg, and normal saline in separate treatments, divided by a washout period of at least 28 days. MEASUREMENTS The primary outcome was pain intensity 6 to 19 days after the initial treatment and the crossover treatment. RESULTS 13 eligible participants were randomly assigned between November 2005 and May 2008; 12 completed the trial. The average pain intensity was 1.55 units lower after IVIG treatment than after saline (95% CI, 1.29 to 1.82; P < 0.001). In 3 patients, pain intensity after IVIG was less than after saline by 50% or more. No serious adverse reactions were reported. LIMITATION The trial was small, and recruitment bias and chance variation could have influenced results and their interpretation. CONCLUSION IVIG, 0.5 g/kg, can reduce pain in refractory CRPS. Studies are required to determine the best immunoglobulin dose, the duration of effect, and when repeated treatments are needed. PRIMARY FUNDING SOURCE Association of Anaesthetists of Great Britain and Ireland, University College London Hospitals Charity, and CSL-Behring.

Off-pump Coronary Artery Bypass (OPCAB) surgery reduces risk-stratified morbidity and mortality: A United Kingdom multi-center comparative analysis of early clinical outcome

Objective—Off-Pump Coronary Artery Bypass (OPCAB) surgery is gaining more popularity worldwide. The aim of this United Kingdom (UK) multi-center study was to assess the early clinical outcome of the OPCAB technique and perform a risk-stratified comparison with the conventional Coronary Artery Bypass Grafting (CABG) using the Cardio-Pulmonary Bypass (CPB) technique. Methods—Data were collected on 5,163 CPB patients from the database of the National Heart and Lung institute, Imperial College, University of London, and on 2,223 OPCAB patients from eight UK cardiac surgical centers, which run established OPCAB surgery programs. All patients had undergone primary isolated CABG for multi-vessel disease through a midline sternotomy approach, between January 1997 and April 2001. Postoperative morbidity and mortality were compared between the CPB and OPCAB patients after adjusting for case-mix. The mortality of the OPCAB patients was also compared, using risk stratification, to the mortality figures reported by the Society of Cardiothoracic Surgeons of Great Britain and Ireland (SCTS) based on 28,018 patients in the national database who were operated on between January 1996 and December 1999. Results—Morbidity and mortality were significantly lower in the OPCAB patients compared with the CPB patients and the UK national database of CABG patients, over the same period of time, after adjusting for case-mix. Conclusion—This study demonstrates that risk stratified morbidity and mortality are significantly lower in OPCAB patients than CPB patients and patients in the UK national database.

Social variations in access to hospital care for patients with colorectal, breast, and lung cancer between 1999 and 2006: retrospective analysis of hospital episode statistics

Objectives To determine the extent to which type of hospital admission (emergency compared with elective) and surgical procedure varied by socioeconomic circumstances, age, sex, and year of admission for colorectal, breast, and lung cancer. Design Repeated cross sectional study with data from individual patients, 1 April 1999 to 31 March 2006. Setting Hospital episode statistics (HES) dataset. Participants 564 821 patients aged 50 and over admitted with a diagnosis of colorectal, breast, or lung cancer. Main outcome measures Proportion of patients admitted as emergencies, and the proportion receiving the recommended surgical treatment. Results Patients from deprived areas, older people, and women were more likely to be admitted as emergencies. For example, the adjusted odds ratio for patients with breast cancer in the least compared with most deprived fifth of deprivation was 0.63 (95% confidence interval 0.60 to 0.66) and the adjusted odds ratio for patients with lung cancer aged 80-89 compared with those aged 50-59 was 3.13 (2.93 to 3.34). There were some improvements in disparities between age groups but not for patients living in deprived areas over time. Patients from deprived areas were less likely to receive preferred procedures for rectal, breast, and lung cancer. These findings did not improve with time. For example, 67.4% (3529/5237) of patients in the most deprived fifth of deprivation had anterior resection for rectal cancer compared with 75.5% (4497/5959) of patients in the least deprived fifth (1.34, 1.22 to 1.47). Over half (54.0%, 11 256/20 849) of patients in the most deprived fifth of deprivation had breast conserving surgery compared with 63.7% (18 445/28 960) of patients in the least deprived fifth (1.21, 1.16 to 1.26). Men were less likely than women to undergo anterior resection and lung cancer resection and older people were less likely to receive breast conserving surgery and lung cancer resection. For example, the adjusted odds ratio for lung cancer patients aged 80-89 compared with those aged 50-59 was 0.52 (0.46 to 0.59). Conclusions Despite the implementation of the NHS Cancer Plan, social factors still strongly influence access to and the provision of care.

How to develop a more accurate risk prediction model when there are few events.

When the number of events is low relative to the number of predictors, standard regression could produce overfitted risk models that make inaccurate predictions. Use of penalised regression may improve the accuracy of risk prediction

An evaluation of existing risk stratification models as a tool for comparison of surgical performances for coronary artery bypass grafting between institutions

OBJECTIVE Risk stratification systems are used in cardiac surgery to estimate mortality risk for individual patients and to compare surgical performance between institutions or surgeons. This study investigates the suitability of six existing risk stratification systems for these purposes. METHODS Data on 5471 patients who underwent isolated coronary artery bypass grafting at two UK cardiac centres between 1993 and 1999 were extracted from a prospective computerised clinical data base. Of these patients, 184 (3.3%) died in hospital. In-hospital mortality risk scores were calculated for each patient using the Parsonnet score, the EuroSCORE, the ACC/AHA score and three UK Bayes models (old, new complex and new simple). The accuracy for predicting mortality at an institutional level was assessed by comparing total observed and predicted mortality. The accuracy of the risk scores for predicting mortality for a patient was assessed by the Hosmer-Lemeshow test. The receiver operating characteristic (ROC) curve was used to evaluate how well a system ranks the patient with respect to their risk of mortality and can be useful for patient management. RESULTS Both EuroSCORE and the simple Bayes model were reasonably accurate at predicting overall mortality. However predictive accuracy at the patient level was poor for all systems, although EuroSCORE was accurate for low to medium risk patients. Discrimination was fair with the following ROC areas: Parsonnet 0.73, EuroSCORE 0.76, ACC/AHA system 0.76, old Bayes 0.77, complex Bayes 0.76, simple Bayes 0.76. CONCLUSIONS This study suggests that two of the scores may be useful in comparing institutions. None of the risk scores provide accurate risk estimates for individual patients in the two hospitals studied although EuroSCORE may have some utility for certain patients. All six systems perform moderately at ranking the patients and so may be useful for patient management. More results are needed from other institutions to confirm that the EuroSCORE and the simple Bayes model are suitable for institutional risk-adjusted comparisons.

Volume and functional outcome of intracerebral hemorrhage according to oral anticoagulant type.

Objective: To compare intracerebral hemorrhage (ICH) volume and clinical outcome of non–vitamin K oral anticoagulants (NOAC)–associated ICH to warfarin-associated ICH. Methods: In this multicenter cross-sectional observational study of patients with anticoagulant-associated ICH, consecutive patients with NOAC-ICH were compared to those with warfarin-ICH selected from a population of 344 patients with anticoagulant-associated ICH. ICH volume was measured by an observer blinded to clinical details. Outcome measures were ICH volume and clinical outcome adjusted for confounding factors. Results: We compared 11 patients with NOAC-ICH to 52 patients with warfarin-ICH. The median ICH volume was 2.4 mL (interquartile range [IQR] 0.3–5.4 mL) for NOAC-ICH vs 8.9 mL (IQR 4.0–21.3 mL) for warfarin-ICH (p = 0.0028). In univariate linear regression, use of warfarin (difference in cube root volume 1.61; 95% confidence interval [CI] 0.69 to 2.53) and lobar ICH location (compared with nonlobar ICH; difference in cube root volume 1.52; 95% CI 2.20 to 0.85) were associated with larger ICH volumes. In multivariable linear regression adjusting for confounding factors (sex, hypertension, previous ischemic stroke, white matter disease burden, and premorbid modified Rankin Scale score [mRS]), warfarin use remained independently associated with larger ICH (cube root) volumes (coefficient 0.64; 95% CI 0.24 to 1.25; p = 0.042). Ordered logistic regression showed an increased odds of a worse clinical outcome (as measured by discharge mRS) in warfarin-ICH compared with NOAC-ICH: odds ratio 4.46 (95% CI 1.10 to 18.14; p = 0.037). Conclusions: In this small prospective observational study, patients with NOAC-associated ICH had smaller ICH volumes and better clinical outcomes compared with warfarin-associated ICH.

An evaluation of penalised survival methods for developing prognostic models with rare events.

Prognostic models for survival outcomes are often developed by fitting standard survival regression models, such as the Cox proportional hazards model, to representative datasets. However, these models can be unreliable if the datasets contain few events, which may be the case if either the disease or the event of interest is rare. Specific problems include predictions that are too extreme, and poor discrimination between low-risk and high-risk patients. The objective of this paper is to evaluate three existing penalised methods that have been proposed to improve predictive accuracy. In particular, ridge, lasso and the garotte, which use penalised maximum likelihood to shrink coefficient estimates and in some cases omit predictors entirely, are assessed using simulated data derived from two clinical datasets. The predictions obtained using these methods are compared with those from Cox models fitted using standard maximum likelihood. The simulation results suggest that Cox models fitted using maximum likelihood can perform poorly when there are few events, and that significant improvements are possible by taking a penalised modelling approach. The ridge method generally performed the best, although lasso is recommended if variable selection is required.

Sample size calculation for a stepped wedge trial

BackgroundStepped wedge trials (SWTs) can be considered as a variant of a clustered randomised trial, although in many ways they embed additional complications from the point of view of statistical design and analysis. While the literature is rich for standard parallel or clustered randomised clinical trials (CRTs), it is much less so for SWTs. The specific features of SWTs need to be addressed properly in the sample size calculations to ensure valid estimates of the intervention effect.MethodsWe critically review the available literature on analytical methods to perform sample size and power calculations in a SWT. In particular, we highlight the specific assumptions underlying currently used methods and comment on their validity and potential for extensions. Finally, we propose the use of simulation-based methods to overcome some of the limitations of analytical formulae. We performed a simulation exercise in which we compared simulation-based sample size computations with analytical methods and assessed the impact of varying the basic parameters to the resulting sample size/power, in the case of continuous and binary outcomes and assuming both cross-sectional data and the closed cohort design.ResultsWe compared the sample size requirements for a SWT in comparison to CRTs based on comparable number of measurements in each cluster. In line with the existing literature, we found that when the level of correlation within the clusters is relatively high (for example, greater than 0.1), the SWT requires a smaller number of clusters. For low values of the intracluster correlation, the two designs produce more similar requirements in terms of total number of clusters. We validated our simulation-based approach and compared the results of sample size calculations to analytical methods; the simulation-based procedures perform well, producing results that are extremely similar to the analytical methods. We found that usually the SWT is relatively insensitive to variations in the intracluster correlation, and that failure to account for a potential time effect will artificially and grossly overestimate the power of a study.ConclusionsWe provide a framework for handling the sample size and power calculations of a SWT and suggest that simulation-based procedures may be more effective, especially in dealing with the specific features of the study at hand. In selected situations and depending on the level of intracluster correlation and the cluster size, SWTs may be more efficient than comparable CRTs. However, the decision about the design to be implemented will be based on a wide range of considerations, including the cost associated with the number of clusters, number of measurements and the trial duration.

Validation of four different risk stratification systems in patients undergoing off-pump coronary artery bypass surgery: a UK multicentre analysis of 2223 patients

Background: Various risk stratification systems have been developed in coronary artery bypass graft surgery (CABG), based mainly on patients undergoing procedures with cardiopulmonary bypass. Objective: To assess the validity and applicability of the Parsonnet score, the EuroSCORE, the American College of Cardiology/American Heart Association (ACC/AHA) system, and the UK CABG Bayes model in patients undergoing off-pump coronary artery bypass surgery (OPCAB) in the UK. Methods: Data on 2223 patients who underwent OPCAB in eight cardiac surgical centres were collected. Predicted mortality risk scores were calculated using the four systems and compared with observed mortality. Calibration was assessed by the Hosmer–Lemeshow (HL) test. Discrimination was assessed using the receiver operating characteristic (ROC) curve area. Results: 30 of 2223 patients (1.3%) died in hospital. For the Parsonnet score the HL test was significant (p < 0.001) and the receiver operating characteristic curve (ROC) area was 0.74. For the EuroSCORE the HL test was also significant (p = 0.008) and the ROC area was 0.75. For the ACC/AHA system the HL test was non-significant (p = 0.7) and the ROC area was 0.75. For the UK CABG Bayes model the HL test was also non-significant (p = 0.3) and the ROC area was 0.81. Conclusions: The UK CABG Bayes model is reasonably well calibrated and provides good discrimination when applied to OPCAB patients in the UK. Among the other three systems, the ACC/AHA system is well calibrated but its discrimination power was less than for the UK CABG Bayes model. These data suggest that the UK CABG Bayes model could be an appropriate risk stratification system to use for patients undergoing OPCAB in the UK.