Peter McCulloch

Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide

Without a complete published description of interventions, clinicians and patients cannot reliably implement interventions that are shown to be useful, and other researchers cannot replicate or build on research findings. The quality of description of interventions in publications, however, is remarkably poor. To improve the completeness of reporting, and ultimately the replicability, of interventions, an international group of experts and stakeholders developed the Template for Intervention Description and Replication (TIDieR) checklist and guide. The process involved a literature review for relevant checklists and research, a Delphi survey of an international panel of experts to guide item selection, and a face to face panel meeting. The resultant 12 item TIDieR checklist (brief name, why, what (materials), what (procedure), who provided, how, where, when and how much, tailoring, modifications, how well (planned), how well (actual)) is an extension of the CONSORT 2010 statement (item 5) and the SPIRIT 2013 statement (item 11). While the emphasis of the checklist is on trials, the guidance is intended to apply across all evaluative study designs. This paper presents the TIDieR checklist and guide, with an explanation and elaboration for each item, and examples of good reporting. The TIDieR checklist and guide should improve the reporting of interventions and make it easier for authors to structure accounts of their interventions, reviewers and editors to assess the descriptions, and readers to use the information.

No surgical innovation without evaluation: the IDEAL recommendations

Surgery and other invasive therapies are complex interventions, the assessment of which is challenged by factors that depend on operator, team, and setting, such as learning curves, quality variations, and perception of equipoise. We propose recommendations for the assessment of surgery based on a five-stage description of the surgical development process. We also encourage the widespread use of prospective databases and registries. Reports of new techniques should be registered as a professional duty, anonymously if necessary when outcomes are adverse. Case series studies should be replaced by prospective development studies for early technical modifications and by prospective research databases for later pre-trial evaluation. Protocols for these studies should be registered publicly. Statistical process control techniques can be useful in both early and late assessment. Randomised trials should be used whenever possible to investigate efficacy, but adequate pre-trial data are essential to allow power calculations, clarify the definition and indications of the intervention, and develop quality measures. Difficulties in doing randomised clinical trials should be addressed by measures to evaluate learning curves and alleviate equipoise problems. Alternative prospective designs, such as interrupted time series studies, should be used when randomised trials are not feasible. Established procedures should be monitored with prospective databases to analyse outcome variations and to identify late and rare events. Achievement of improved design, conduct, and reporting of surgical research will need concerted action by editors, funders of health care and research, regulatory bodies, and professional societies.

Randomised trials in surgery: problems and possible solutions

The quality and quantity of randomised trials of surgical techniques is acknowledged to be limited. According to Peter McCulloch and colleagues, however, some aspects of surgery present special difficulties for randomised trials. In this article they analyse what these difficulties are and propose some solutions for improving the standards of clinical research in surgery.

When are randomised trials unnecessary? Picking signal from noise.

Although randomised trials are widely accepted as the ideal way of obtaining unbiased estimates of treatment effects, some treatments have dramatic effects that are highly unlikely to reflect inadequately controlled biases. We compiled a list of historical examples of such effects and identified the features of convincing inferences about treatment effects from sources other than randomised trials. A unifying principle is the size of the treatment effect (signal) relative to the expected prognosis (noise) of the condition. A treatment effect is inferred most confidently when the signal to noise ratio is large and its timing is rapid compared with the natural course of the condition. For the examples we considered in detail the rate ratio often exceeds 10 and thus is highly unlikely to reflect bias or factors other than a treatment effect. This model may help to reduce controversy about evidence for treatments whose effects are so dramatic that randomised trials are unnecessary. The relation between a treatment and its effect is sometimes so dramatic that bias can be ruled out as an explanation. Paul Glasziouand colleagues suggest how to determine when observations speak for themselves

Evaluation and stages of surgical innovations

Surgical innovation is an important part of surgical practice. Its assessment is complex because of idiosyncrasies related to surgical practice, but necessary so that introduction and adoption of surgical innovations can derive from evidence-based principles rather than trial and error. A regulatory framework is also desirable to protect patients against the potential harms of any novel procedure. In this first of three Series papers on surgical innovation and evaluation, we propose a five-stage paradigm to describe the development of innovative surgical procedures.

Mortality and morbidity in gastro-oesophageal cancer surgery: initial results of ASCOT multicentre prospective cohort study

Abstract Objective To evaluate the effect of comorbidity and other risk factors on postoperative mortality and morbidity in patients undergoing major oesophageal and gastric surgery. Design Multicentre cohort study with data on postoperative mortality and morbidity in hospital. Data source and methods The ASCOT prospective database, comprising 2087 patients with newly diagnosed oesophageal and gastric cancer in 24 hospitals in England and Wales between 1 January 1999 and 31 December 2002. Multivariate logistic regression analysis was used to model the risk of death and postoperative complications. Results 955 patients underwent oesophagectomy or gastrectomy. Of these, 253 (27%) were graded ASA III or IV, and 187 (20%) had a high physiological POSSUM score (≥ 20). Operative mortality was 12% (111/955). Physiological POSSUM score, surgeons assessment, type of operation, hospital case volume, and tumour stage independently predicted operative mortality. Medical complications were associated with higher physiological POSSUM scores and ASA grade, oesophagectomy or total gastrectomy, thoracotomy, and radical nodal dissection. Stage and additional organ resection predicted surgical (technical) complications. Conclusions Many patients undergoing surgery for gastro-oesophageal cancer have major comorbid disease, which strongly influences their risk of postoperative death. Technical complications do not seem to be influenced by preoperative factors but reflect the extent of surgery and perhaps surgical judgment. Detailed prospective multicentre cooperative audit, with appropriate risk adjustment, is fundamental in the evaluation of cancer care and must be properly resourced.

The effects of aviation-style non-technical skills training on technical performance and outcome in the operating theatre

Unintended harm to patients in operating theatres is common. Correlations have been demonstrated between teamwork skills and error rates in theatres. This was a single-institution uncontrolled before–after study of the effects of “non-technical” skills training on attitudes, teamwork, technical performance and clinical outcome in laparoscopic cholecystectomy (LC) and carotid endarterectomy (CEA) operations. The setting was the theatre suite of a UK teaching hospital. Attitudes were measured using the Safety Attitudes Questionnaire (SAQ). Teamwork was scored using the Oxford Non-Technical Skills (NOTECHS) method. Operative technical errors (OTEs), non-operative procedural errors (NOPEs), complications, operating time and length of hospital stay (LOS) were recorded. A 9 h classroom non-technical skills course based on aviation “Crew Resource Management” (CRM) was offered to all staff, followed by 3 months of twice-weekly coaching from CRM experts. Forty-eight procedures (26 LC and 22 CEA) were studied before intervention, and 55 (32 and 23) afterwards. Non-technical skills and attitudes improved after training (NOTECHS increase 37.0 to 38.7, t = −2.35, p = 0.021, SAQ teamwork climate increase 64.1 to 69.2, t = −2.95, p = 0.007). OTEs declined from 1.73 to 0.98 (u = 1071, p = 0.009), and NOPEs from 8.48 to 5.16 per operation (t = 4.383, p<0.001). These effects were stronger in the LC group than in CEA procedures. The operating time was unchanged, and a non-significant reduction in LOS was observed. Non-technical skills training improved technical performance in theatre, but the effects varied between teams. Considerable cultural resistance to adoption was encountered, particularly among medical staff. Debriefing and challenging authority seemed more difficult to introduce than other parts of the training. Further studies are needed to define the optimal training package, explain variable responses and confirm clinical benefit.

Teamwork and Error in the Operating Room : Analysis of Skills and Roles

Objective:To analyze the effects of surgical, anesthetic, and nursing teamwork skills on technical outcomes. Summary Background Data:The value of team skills in reducing adverse events in the operating room is presently receiving considerable attention. Current work has not yet identified in detail how the teamwork and communication skills of surgeons, anesthetists, and nurses affect the course of an operation. Methods:Twenty-six laparoscopic cholecystectomies and 22 carotid endarterectomies were studied using direct observation methods. For each operation, teams’ skills were scored for the whole team, and for nursing, surgical, and anesthetic subteams on 4 dimensions (leadership and management [LM]; teamwork and cooperation; problem solving and decision making; and situation awareness). Operating time, errors in surgical technique, and other procedural problems and errors were measured as outcome parameters for each operation. The relationships between teamwork scores and these outcome parameters within each operation were examined using analysis of variance and linear regression. Results:Surgical (F(2,42) = 3.32, P = 0.046) and anesthetic (F(2,42) = 3.26, P = 0.048) LM had significant but opposite relationships with operating time in each operation: operating time increased significantly with higher anesthetic but decreased with higher surgical LM scores. Errors in surgical technique had a strong association with surgical situation awareness (F(2,42) = 7.93, P < 0.001) in each operation. Other procedural problems and errors were related to the intraoperative LM skills of the nurses (F(5,1) = 3.96, P = 0.027). Conclusions:Detailed analysis of team interactions and dimensions is feasible and valuable, yielding important insights into relationships between nontechnical skills, technical performance, and operative duration. These results support the concept that interventions designed to improve teamwork and communication may have beneficial effects on technical performance and patient outcome.

Marimastat as maintenance therapy for patients with advanced gastric cancer: a randomised trial

This randomised, double-blind, placebo-controlled study was designed to evaluate the ability of the orally administered matrix metalloproteinase inhibitor, marimastat, to prolong survival in patients with non-resectable gastric and gastro-oesophageal adenocarcinoma. Three hundred and sixty-nine patients with histological proof of adenocarcinoma, who had received no more than a single regimen of 5-fluorouracil-based chemotherapy, were randomised to receive either marimastat (10 mg b.d.) or placebo. Patients were treated for as long as was tolerable. The primary endpoint was overall survival with secondary endpoints of time to disease progression and quality of life. At the point of protocol-defined study completion (85% mortality in the placebo arm) there was a modest difference in survival in the intention-to-treat population in favour of marimastat (P=0.07 log-rank test, hazard ratio=1.23 (95% confidence interval 0.98–1.55)). This survival benefit was maintained over a further 2 years of follow-up (P=0.024, hazard ratio=1.27 (1.03–1.57)). The median survival was 138 days for placebo and 160 days for marimastat, with 2-year survival of 3% and 9% respectively. A significant survival benefit was identified at study completion in the pre-defined sub-group of 123 patients who had received prior chemotherapy (P=0.045, hazard ratio=1.53 (1.00–2.34)). This benefit increased with 2 years additional follow-up (P=0.006, hazard ratio=1.68 (1.16–2.44)), with 2-year survival of 5% and 18% respectively. Progression-free survival was also significantly longer for patients receiving marimastat compared to placebo (P=0.009, hazard ratio=1.32 (1.07–1.63)). Marimastat treatment was associated with the development of musculoskeletal pain and inflammation. Events of anaemia, abdominal pain, jaundice and weight loss were more common in the placebo arm. This is one of the first demonstrations of a therapeutic benefit for a matrix metalloproteinase inhibitor in cancer patients. The greatest benefit was observed in patients who had previously received chemotherapy. A further randomised study of marimastat in these patients is warranted.

The Oxford NOTECHS System: reliability and validity of a tool for measuring teamwork behaviour in the operating theatre

Introduction: The frequency of adverse events in the operating theatre has been linked to the quality of teamwork and communication. Developing suitable measures of teamwork may play a role in reducing errors in surgery. This study reports on the development and evaluation of a method for measuring operating-theatre teamwork quality. Methods: The Oxford Non-Technical Skills (NOTECHS) scale was developed from an aviation instrument for assessment of non-technical skills. Consultation with experts and task analysis led to modifications reflecting the complexities of the theatre teamwork, particularly the coexistence of three subteams (surgeons, anaesthetists and nurses). The scale was then evaluated using teams performing laparoscopic cholecystectomy (n = 65) before and after teamwork training. Attitudes to teamwork and surgical error rates were assessed by questionnaire and direct observation methods, and used to assess the reliability and validity of the Oxford NOTECHS scale. Results: The interobserver reliability was excellent in 24 operations independently assessed by two observers (Rwg = 0.99), confirmed by a third observer in 11 cases (Rwg = 0.99). Validity was demonstrated through improved scores after teamwork training (t = −3.019, p = 0.005), concurrent with improved attitudes to teamwork after training; inverse correlation between NOTECHS scores and surgical errors (ρ = −0.267, p = 0.046); strong inverse correlation between surgical subteam score and surgical errors (ρ = −0.412, n = 65, p = 0.001); and strong correlation with teamwork scores from an alternative system (n = 5, r = 0.886, p = 0.046) Conclusion: The Oxford NOTECHS scale appears to be a reliable and valid instrument for assessing teamwork in the operating theatre, and is ready for further application.