Jane L. Hutton

Survey of the quality of experimental design, statistical analysis and reporting of research using animals.

For scientific, ethical and economic reasons, experiments involving animals should be appropriately designed, correctly analysed and transparently reported. This increases the scientific validity of the results, and maximises the knowledge gained from each experiment. A minimum amount of relevant information must be included in scientific publications to ensure that the methods and results of a study can be reviewed, analysed and repeated. Omitting essential information can raise scientific and ethical concerns. We report the findings of a systematic survey of reporting, experimental design and statistical analysis in published biomedical research using laboratory animals. Medline and EMBASE were searched for studies reporting research on live rats, mice and non-human primates carried out in UK and US publicly funded research establishments. Detailed information was collected from 271 publications, about the objective or hypothesis of the study, the number, sex, age and/or weight of animals used, and experimental and statistical methods. Only 59% of the studies stated the hypothesis or objective of the study and the number and characteristics of the animals used. Appropriate and efficient experimental design is a critical component of high-quality science. Most of the papers surveyed did not use randomisation (87%) or blinding (86%), to reduce bias in animal selection and outcome assessment. Only 70% of the publications that used statistical methods described their methods and presented the results with a measure of error or variability. This survey has identified a number of issues that need to be addressed in order to improve experimental design and reporting in publications describing research using animals. Scientific publication is a powerful and important source of information; the authors of scientific publications therefore have a responsibility to describe their methods and results comprehensively, accurately and transparently, and peer reviewers and journal editors share the responsibility to ensure that published studies fulfil these criteria.

Maximising response to postal questionnaires – A systematic review of randomised trials in health research

BackgroundPostal self-completion questionnaires offer one of the least expensive modes of collecting patient based outcomes in health care research. The purpose of this review is to assess the efficacy of methods of increasing response to postal questionnaires in health care studies on patient populations.MethodsThe following databases were searched: Medline, Embase, CENTRAL, CDSR, PsycINFO, NRR and ZETOC. Reference lists of relevant reviews and relevant journals were hand searched. Inclusion criteria were randomised trials of strategies to improve questionnaire response in health care research on patient populations. Response rate was defined as the percentage of questionnaires returned after all follow-up efforts. Study quality was assessed by two independent reviewers. The Mantel-Haenszel method was used to calculate the pooled odds ratios.ResultsThirteen studies reporting fifteen trials were included. Implementation of reminder letters and telephone contact had the most significant effect on response rates (odds ratio 3.7, 95% confidence interval 2.30 to 5.97 p = <0.00001). Shorter questionnaires also improved response rates to a lesser degree (odds ratio 1.4, 95% confidence interval 1.19 to 1.54). No evidence was found that incentives, re-ordering of questions or including an information brochure with the questionnaire confer any additional advantage.ConclusionImplementing repeat mailing strategies and/or telephone reminders may improve response to postal questionnaires in health care research. Making the questionnaire shorter may also improve response rates. There is a lack of evidence to suggest that incentives are useful. In the context of health care research all strategies to improve response to postal questionnaires require further evaluation.

Life expectancy in children with cerebral palsy.

Abstract Objective : To determine life expectancy of children with cerebral palsy. Design : Cohort analysis, by means of register compiled from multiple sources of ascertainment, of all children with cerebral palsy born during 1966-84 to mothers resident in Mersey region. Status of children was determined by flagging through NHS central register. Subjects : 1258 subjects with idiopathic cerebral palsy, of whom 1251 were traced and included in analysis. Main outcome measures : Effect of functional ability (ambulation, manual dexterity, and mental ability), sex, birth weight, and gestational age on survival. Results : 20 year survival for whole cohort was 89.3% for females and 86.9% for males. For subjects with no severe functional disabilities 20 year survival was 99% (95% confidence interval 98% to 100%), while subjects severely disabled in all three functional groups had 20 year survival of 50% (42% to 58%). Subjects with birth weight <=2500 g had 20 year survival of 92% (89% to 95%), while those with birth weight <2500 g had survival of 87% (84% to 89%). Subjects with gestational age of <37 weeks had 20 year survival of 93% (91% to 96%), while those with gestational age >=37 weeks had survival of 85% (83% to 88%). Birth weight and gestational age were less predictive of survival than functional disability. Best statistical model used gestational age and number of severe functional disabilities as predictors. Conclusions : Life expectancy of this cohort of children with cerebral palsy was greater than has been suggested in some previous studies. This has important implications for social, educational, and health services.

Number needed to treat and number needed to harm are not the best way to report and assess the results of randomised clinical trials.

The inverse of the difference between rates, called the ‘number needed to treat’ (NNT), was suggested 20 years ago as a good way to present the results of comparisons of success or failure under different therapies. Such comparisons usually arise in randomised controlled trials and meta‐analysis. This article reviews the claims made about this statistic, and the problems associated with it. Methods that have been proposed for confidence intervals are evaluated, and shown to be erroneous. We suggest that giving the baseline risk, and the difference in success or event rates, the ‘absolute risk reduction’, is preferable to the number needed to treat, for both theoretical and practical reasons.

Differential effects of preterm birth and small gestational age on cognitive and motor development

AIMS To determine the differential effects of preterm birth and being small for gestational age on the cognitive and motor ability of the child. METHODS A longitudinal cohort of all infants of gestational age ≤ 32 weeks born to mothers resident in the counties of Cheshire and Merseyside in 1980-1 was studied. The children were assessed at the age of 8 to 9 years using the Wechsler Intelligence Scale for Children, the Neale analysis of reading ability, and the Stott-Moyes-Henderson test of motor impairment. Adequacy of fetal growth was determined by the birthweight ratio—that is, the ratio of the observed birthweight to the expected birthweight for a given gestational age. Children with clinically diagnosed motor, learning or sensory disabilities were excluded. Information on social variables was obtained by a questionnaire completed by the parents. Of the 182 children, 158 were assessed. RESULTS IQ was positively correlated with birthweight ratio but not with birthweight or gestational age. Motor ability was associated with birthweight, gestational age, and birthweight ratio. Reading comprehension was associated with birthweight ratio, but reading rate and accuracy were best explained by social variables and sex. IQ remained associated with birthweight ratio, after adjusting for maternal education, housing status, and number of social service benefits received. Reading ability was related to these social variables but motor ability was not. CONCLUSIONS The effects of SGA and preterm birth differed: SGA was associated with cognitive ability, as measured by IQ and reading comprehension; motor ability was additionally associated with preterm birth. Reading rate and accuracy were not associated with SGA or preterm birth but were socially determined. Key points Preterm birth and being small for gestational age differ in their effects on motor and cognitive development Cognitive ability, as measured by IQ and reading comprehension, was negatively associated with the degree of fetal growth retardation Motor ability was positively associated with gestational age and negatively associated with the degree of fetal growth retardation Reading rate and reading accuracy were not associated with either gestational age or the degree of fetal growth retardation, but were socially determined

Mechanical supports for acute, severe ankle sprain: a pragmatic, multicentre, randomised controlled trial.

BACKGROUND Severe ankle sprains are a common presentation in emergency departments in the UK. We aimed to assess the effectiveness of three different mechanical supports (Aircast brace, Bledsoe boot, or 10-day below-knee cast) compared with that of a double-layer tubular compression bandage in promoting recovery after severe ankle sprains. METHODS We did a pragmatic, multicentre randomised trial with blinded assessment of outcome. 584 participants with severe ankle sprain were recruited between April, 2003, and July, 2005, from eight emergency departments across the UK. Participants were provided with a mechanical support within the first 3 days of attendance by a trained health-care professional, and given advice on reducing swelling and pain. Functional outcomes were measured over 9 months. The primary outcome was quality of ankle function at 3 months, measured using the Foot and Ankle Score; analysis was by intention to treat. This study is registered as an International Standard Randomised Controlled Trial, number ISRCTN37807450. RESULTS Patients who received the below-knee cast had a more rapid recovery than those given the tubular compression bandage. We noted clinically important benefits at 3 months in quality of ankle function with the cast compared with tubular compression bandage (mean difference 9%; 95% CI 2.4-15.0), as well as in pain, symptoms, and activity. The mean difference in quality of ankle function between Aircast brace and tubular compression bandage was 8%; 95% CI 1.8-14.2, but there were little differences for pain, symptoms, and activity. Bledsoe boots offered no benefit over tubular compression bandage, which was the least effective treatment throughout the recovery period. There were no significant differences between tubular compression bandage and the other treatments at 9 months. Side-effects were rare with no discernible differences between treatments. Reported events (all treatments combined) were cellulitis (two cases), pulmonary embolus (two cases), and deep-vein thrombosis (three cases). INTERPRETATION A short period of immobilisation in a below-knee cast or Aircast results in faster recovery than if the patient is only given tubular compression bandage. We recommend below-knee casts because they show the widest range of benefit. FUNDING National Co-ordinating Centre for Health Technology Assessment.

Bias in meta‐analysis due to outcome variable selection within studies

Although bias in meta-analysis arising from selective publication has been studied, within-study selection has received little attention. Chronic diseases often have several possible outcome variables. Methods based on the size of the effect allow results from studies with different outcomes to be combined. However, the possibility of selective reporting of outcomes must be considered. The effect of selective reporting on estimates of the size of the effect and significance levels is presented, and sensitivity analyses are suggested. Substantial publication bias could arise from multiple testing of outcomes in a study, followed by selective reporting. Two meta-analyses, on anthelminth therapy and a treatment for incontinence, are reassessed allowing for within-study selection, as it is clear that more outcomes had been measured than were reported. The sensitivity analyses show that the robustness of the anthelminth results is dependent on what assumption one makes about the reporting strategy for the largest trial. The possible influence of correlation between within-child measurements was such that the conclusions could easily be reversed. The effect of a mild assumption on within-trial selection alone could alter general recommendations about the treatment for incontinence.

Outcome selection bias in meta-analysis.

Publication bias has been previously identified as a threat to the validity of a meta-analysis. Recently, new evidence has documented an additional threat to validity, the selective reporting of trial outcomes within published studies. Several diseases have several possible measures of outcome. Some articles might report only a selection of those outcomes, perhaps those with statistically significant results. In this article, we review this problem while addressing the questions: what is within-study selective reporting? how common is it? why is it done? how can it mislead? how can it be detected?, and finally, what is the solution? We recommend that both publication bias and selective reporting should be routinely investigated in systematic reviews.

Long‐term survival for a cohort of adults with cerebral palsy

The aim of this study was to investigate long‐term survival and examine causes of death in adult patients with cerebral palsy (CP). A 1940–1950 birth cohort based on paediatric case referral allows for long‐term survival follow‐up. Survival is analyzed by birth characteristics and severity of disability from age 20 years (and age 2y for a subset of the data). Survival outcome compared with that expected in the general population based on English life tables. The main cohort consisted of 341 individuals, with 193 males and 148 females. Conditional on surviving to age 20 years, almost 85% of the cohort survived to age 50 years (a comparable estimate for the general population is 96%). Very few deaths were attributed to CP for those people dying over 20 years of age. Females survived better than males. However, females faced a greater increase in risk relative to the general population than did males. We conclude that survival outlook is good though lower than in the general population. The relative risk of death compared with the UK population decreases with age, although it shows some indication of rising again after age 50 years. Many more deaths were caused by diseases of the respiratory system among those dying in their 20s and 30s than would be expected in the general population. Many fewer deaths than expected in this age group are caused by injuries and accidents. For those people who die in their 40s and 50s, an increase in deaths due to diseases of the circulatory system and neoplasms is observed. More deaths than expected in this age group are due to diseases of the nervous system.

Transfusion of ABO‐mismatched platelets leads to early platelet refractoriness

Summary. Forty‐three consecutive patients previously unexposed to platelets and undergoing treatment for acute leukaemia or autografting for relapsed Hodgkins lymphoma were randomized to receive transfused platelets of either their own ABO group (OG) or of a major mismatched group (MMG). The 26 evaluable patients were equally distributed between the two study groups.