Steven A. Julious

Reducing waste from incomplete or unusable reports of biomedical research

Research publication can both communicate and miscommunicate. Unless research is adequately reported, the time and resources invested in the conduct of research is wasted. Reporting guidelines such as CONSORT, STARD, PRISMA, and ARRIVE aim to improve the quality of research reports, but all are much less adopted and adhered to than they should be. Adequate reports of research should clearly describe which questions were addressed and why, what was done, what was shown, and what the findings mean. However, substantial failures occur in each of these elements. For example, studies of published trial reports showed that the poor description of interventions meant that 40-89% were non-replicable; comparisons of protocols with publications showed that most studies had at least one primary outcome changed, introduced, or omitted; and investigators of new trials rarely set their findings in the context of a systematic review, and cited a very small and biased selection of previous relevant trials. Although best documented in reports of controlled trials, inadequate reporting occurs in all types of studies-animal and other preclinical studies, diagnostic studies, epidemiological studies, clinical prediction research, surveys, and qualitative studies. In this report, and in the Series more generally, we point to a waste at all stages in medical research. Although a more nuanced understanding of the complex systems involved in the conduct, writing, and publication of research is desirable, some immediate action can be taken to improve the reporting of research. Evidence for some recommendations is clear: change the current system of research rewards and regulations to encourage better and more complete reporting, and fund the development and maintenance of infrastructure to support better reporting, linkage, and archiving of all elements of research. However, the high amount of waste also warrants future investment in the monitoring of and research into reporting of research, and active implementation of the findings to ensure that research reports better address the needs of the range of research users.

Estimating sample sizes for binary, ordered categorical, and continuous outcomes in two group comparisons.

Sample size calculations are now mandatory for many research protocols, but the ones useful in common situations are not all easily accessible. This paper outlines the ways of calculating sample sizes in two group studies for binary, ordered categorical, and continuous outcomes. Formulas and worked examples are given. Maximum power is usually achieved by having equal numbers in the two groups. However, this is not always possible and calculations for unequal group sizes are given.

Chronic Chlamydia pneumoniae infection and asthma exacerbations in children.

This study was undertaken to investigate the reported association between Chlamydia pneumoniae and Mycoplasma pneumoniae infection and the expression of asthma-related symptoms. One hundred and eight children with asthma symptoms, aged 9-11 yrs, completed a 13 month longitudinal study. The children maintained a daily diary of respiratory symptoms and peak flow rates. When respiratory symptoms were reported an investigator was called and a nasal aspirate obtained. In total 292 episodes were reported. After the study 65 children provided samples when asymptomatic. The presence of infection was investigated by the polymerase chain reaction for C. pneumoniae and M. pneumoniae and C. pneumoniae secretory immunoglobulin A (IgA) was detected by amplified enzyme immunoassay. C. pneumoniae detections were similar between the symptomatic and asymptomatic episodes (23 versus 28%, respectively). Children who reported multiple episodes also tended to remain PCR positive for C. pneumoniae suggesting chronic infection (p< 0.02). C. pneumoniae-specific secretory-IgA antibodies were more than seven times greater in subjects who reported four or more exacerbations in the study compared to those who reported just one (p<0.02). M. pneumoniae was found in two of 292 reports and in two of 65 asymptomatic samples. In conclusion, chronic Chlamydia pneumoniae infection is common in schoolage children and immune responses to C. pneumoniae are positively associated with frequency of asthma exacerbations. We suggest that the immune response to chronic C. pneumoniae infection may interact with allergic inflammation to increase asthma symptoms. In contrast Mycoplasma pneumoniae was not found to be important in this study.

A reinvestigation of recruitment to randomised, controlled, multicenter trials: a review of trials funded by two UK funding agencies.

BackgroundRandomised controlled trials (RCTs) are the gold standard assessment for health technologies. A key aspect of the design of any clinical trial is the target sample size. However, many publicly-funded trials fail to reach their target sample size. This study seeks to assess the current state of recruitment success and grant extensions in trials funded by the Health Technology Assessment (HTA) program and the UK Medical Research Council (MRC).MethodsData were gathered from two sources: the National Institute for Health Research (NIHR) HTA Journal Archive and the MRC subset of the International Standard Randomised Controlled Trial Number (ISRCTN) register. A total of 440 trials recruiting between 2002 and 2008 were assessed for eligibility, of which 73 met the inclusion criteria. Where data were unavailable from the reports, members of the trial team were contacted to ensure completeness.ResultsOver half (55%) of trials recruited their originally specified target sample size, with over three-quarters (78%) recruiting 80% of their target. There was no evidence of this improving over the time of the assessment. Nearly half (45%) of trials received an extension of some kind. Those that did were no more likely to successfully recruit. Trials with 80% power were less likely to successfully recruit compared to studies with 90% power.ConclusionsWhile recruitment appears to have improved since 1994 to 2002, publicly-funded trials in the UK still struggle to recruit to their target sample size, and both time and financial extensions are often requested. Strategies to cope with such problems should be more widely applied. It is recommended that where possible studies are planned with 90% power.

Acupuncture in the prevention of postoperative nausea and vomiting

The efficacy of intra‐operative acupuncture at the PC6 point in the prevention of postoperative nausea or vomiting was studied. A double‐blind randomised controlled study of acupuncture versus placebo was performed in 81 patients scheduled for day case gynaecological laparoscopic surgery. Failure of treatment was defined as the occurrence of nausea or vomiting prior to or within 24 h of discharge. The use of acupuncture reduced the incidence of postoperative nausea or vomiting in hospital from 65% to 35% compared with placebo and after discharge from 69% to 31% compared with placebo.

Confounding and Simpson's paradox

A common problem when analysing clinical data is that of confounding. This occurs when the association between an exposure and an outcome is investigated but the exposure and outcome are strongly associated with a third variable. An extreme example of this is Simpsons paradox, in which this third factor reverses the effect first observed.1 This phenomenon has long been recognised as a theoretical possibility but few real examples have been presented. Charig et al undertook a historical comparison of success rates in removing kidney stones.2 Open surgery (1972-80) had a success rate of 78% (273/350) while percutaneous nephrolithotomy (1980-5) had a success rate of 83% (289/350), an improvement over the use of open surgery. However, the success rates looked rather different when stone diameter was taken into account. This showed …

An audit of sample sizes for pilot and feasibility trials being undertaken in the United Kingdom registered in the United Kingdom Clinical Research Network database

BackgroundThere is little published guidance as to the sample size required for a pilot or feasibility trial despite the fact that a sample size justification is a key element in the design of a trial. A sample size justification should give the minimum number of participants needed in order to meet the objectives of the trial. This paper seeks to describe the target sample sizes set for pilot and feasibility randomised controlled trials, currently running within the United Kingdom.MethodsData were gathered from the United Kingdom Clinical Research Network (UKCRN) database using the search terms ‘pilot’ and ‘feasibility’. From this search 513 studies were assessed for eligibility of which 79 met the inclusion criteria. Where the data summary on the UKCRN Database was incomplete, data were also gathered from: the International Standardised Randomised Controlled Trial Number (ISRCTN) register; the clinicaltrials.gov website and the website of the funders. For 62 of the trials, it was necessary to contact members of the research team by email to ensure completeness.ResultsOf the 79 trials analysed, 50 (63.3%) were labelled as pilot trials, 25 (31.6%) feasibility and 14 were described as both pilot and feasibility trials. The majority had two arms (n = 68, 86.1%) and the two most common endpoints were continuous (n = 45, 57.0%) and dichotomous (n = 31, 39.2%). Pilot trials were found to have a smaller sample size per arm (median = 30, range = 8 to 114 participants) than feasibility trials (median = 36, range = 10 to 300 participants). By type of endpoint, across feasibility and pilot trials, the median sample size per arm was 36 (range = 10 to 300 participants) for trials with a dichotomous endpoint and 30 (range = 8 to 114 participants) for trials with a continuous endpoint. Publicly funded pilot trials appear to be larger than industry funded pilot trials: median sample sizes of 33 (range = 15 to 114 participants) and 25 (range = 8 to 100 participants) respectively.ConclusionAll studies should have a sample size justification. Not all studies however need to have a sample size calculation. For pilot and feasibility trials, while a sample size justification is important, a formal sample size calculation may not be appropriate. The results in this paper describe the observed sample sizes in feasibility and pilot randomised controlled trials on the UKCRN Database.

A comparison of intravenous and subcutaneous hydration in elderly acute stroke patients.

The aim of this study was to compare the effectiveness of subcutaneous and intravenous fluid therapy in hydrating, elderly acute stroke patients. Thirty-four such patients, needing parenteral fluids because of impaired consciousness or dysphagia, were randomly allocated to receive either subcutaneous or intravenous fluids (2 litres of dextrose-saline/24 hours). Serum osmolality was measured before starting fluid therapy (Day 1) and on Days 2 and 3. An analysis of covariance of the osmolalities showed no statistical difference between the two groups (P = 0.12). The total cost of cannulae used over the 3 days for the subcutaneous route was approximately a third of that for the intravenous route. Complication rates were similar for the two groups. The results suggest that subcutaneous fluid therapy is an effective alternative to the intravenous route.

Problems with the performance of the SF-36 among people with type 2 diabetes in general practice.

Objective: To validate the short form-36 (SF-36) among people with type 2 diabetes in general practice, and to make comparisons with the Audit of Diabetis Dependent Quality of Life (ADDQoL). Design: Postal survey with one reminder. Setting: Four general practices. Patients: One hundred and eighty-four eligible patients (30–70 years) with type 2 diabetes on 14 general practitioner lists. Measures: SF-36 response rates, distribution of dimension scores and internal consistency. Median scores in relation to sociodemography and self-reported health. Comparisons with ADDQoL scores. Results: One hundred and thirty-one patients responded (71%). Distributions of SF-36 dimension scores were mostly skewed. Internal consistency and construct validity were acceptable, with predictable sociodemographic trends. People with illness related to or unrelated to diabetes scored significantly lower on most dimensions. SF-36 dimension scores correlated best with relevant diabetes-specific ADDQoL scores amongst respondents reporting no comorbidity. Conclusions: Although valid and reliable, SF-36 scores are strongly affected by non-diabetic comorbidity in type 2 diabetes, supporting the complementary use of a diabetes-specific measure, providing information about the impact of diabetes specifically.

Computer Therapy Compared With Usual Care for People With Long-Standing Aphasia Poststroke A Pilot Randomized Controlled Trial

Background and Purpose— The purpose of this study was to test the feasibility of conducting a randomized controlled trial to study the effectiveness of self-managed computer treatment for people with long-standing aphasia after stroke. Method— In this pilot single-blinded, parallel-group, randomized controlled trial participants with aphasia were allocated to self-managed computer treatment with volunteer support or usual care (everyday language activity). The 5-month intervention period was followed by 3 months without intervention to investigate treatment maintenance. Results— Thirty-four participants were recruited. Seventeen participants were allocated to each group. Thirteen participants from the usual care group and 15 from the computer treatment group were followed up at 5 months. An average of 4 hours 43 minutes speech and language therapy time and 4 hours volunteer support time enabled an average of 25 hours of independent practice. The difference in percentage change in naming ability from baseline at 5 months between groups was 19.8% (95% CI, 4.4–35.2; P=0.014) in favor of the treatment group. Participants with more severe aphasia showed little benefit. Results demonstrate early indications of cost-effectiveness of self-managed computer therapy. Conclusion— This pilot trial indicates that self-managed computer therapy for aphasia is feasible and that it will be practical to recruit sufficient participants to conduct an appropriately powered clinical trial to investigate the effectiveness of self-managed computer therapy for people with long-standing aphasia. Clinical Trial Registration— www.controlled-trials.com. Unique identifier: ISRCTN91534629.