James Penston

Statistics-based research--a pig in a poke?

Much of medical research involves large-scale randomized controlled trials designed to detect small differences in outcome between the study groups. This approach is believed to produce reliable evidence on which the management of patients is based. But can we be sure that the demonstration of a small, albeit statistically significant, difference is sufficient to infer the presence of a causal relationship between the drug and the outcome? A study is claimed to have internal validity when other explanations for the observed difference - namely, inequalities between the groups, bias in the assessment of the outcome and chance - have been excluded. Despite the various processes that are put into place - including, for example, randomization, allocation concealment, double-blinding and intention-to-treat analysis - it remains doubtful whether the groups are equal in terms of all factors relevant to the outcome and whether bias has been excluded. As for the exclusion of chance, not only may inappropriate statistical tests be used, but also frequentist statistics has been subjected to serious criticisms in recent years that further bring internal validity into question. But the problems do not end with the flaws in internal validity. The philosophical basis of large-scale randomized controlled trials and epidemiological studies is unsound. When examined closely, many obstacles emerge that threaten the inference from a small, statistically significant difference to the presence of a causal relationship between the drug and the outcome. Given the influence of statistics-based research on the practice of medicine, it is of the utmost importance that the flaws in this methodology are brought to the fore.Much of medical research involves large-scale randomized controlled trials designed to detect small differences in outcome between the study groups. This approach is believed to produce reliable evidence on which the management of patients is based. But can we be sure that the demonstration of a small, albeit statistically significant, difference is sufficient to infer the presence of a causal relationship between the drug and the outcome? A study is claimed to have internal validity when other explanations for the observed difference – namely, inequalities between the groups, bias in the assessment of the outcome and chance – have been excluded. Despite the various processes that are put into place – including, for example, randomization, allocation concealment, double-blinding and intention-to-treat analysis – it remains doubtful whether the groups are equal in terms of all factors relevant to the outcome and whether bias has been excluded. As for the exclusion of chance, not only may inappropriate statistical tests be used, but also frequentist statistics has been subjected to serious criticisms in recent years that further bring internal validity into question. But the problems do not end with the flaws in internal validity. The philosophical basis of large-scale randomized controlled trials and epidemiological studies is unsound. When examined closely, many obstacles emerge that threaten the inference from a small, statistically significant difference to the presence of a causal relationship between the drug and the outcome. Given the influence of statistics-based research on the practice of medicine, it is of the utmost importance that the flaws in this methodology are brought to the fore.

Why is alternative medicine alone under censure

Moynihan’s article and the ensuing rapid responses reveal a visceral dislike of alternative medicine coupled with a strong streak of intolerance among some healthcare professionals.1 Why are the opponents of alternative medicine so driven to eradicate it? Is it because they …

Thrombosis of the portal venous system following blunt abdominal trauma

A 60-year-old man presented to the accident and emergency department with a 4-day history of abdominal pain following blunt abdominal trauma. An initial CT scan showed thickened walls of the proximal jejunum and thromboses in the portal, splenic and superior mesenteric veins. He was given warfarin and the abdominal pain resolved. A repeat CT scan 1 week later revealed significant resolution of the mural thickening and the portal vein thrombosis. A subsequent thrombophilia screen was negative and he continued taking oral anticoagulants for a total of 6 months. A repeat CT scan 3 months after presentation revealed complete recanalisation the portal venous system.

Beware next wave of propaganda

Flexible sigmoidoscopy is recommended to be added to the national bowel cancer screening programme.1 According to the director of programmes, “The recommendation to implement this new programme is based on robust research findings that meet the committee’s well respected criteria. We are confident that the right decision has …

Kenneth Geoffrey Wormsley

Kenneth Geoffrey Wormsley (“Ken”) was born in Bielefeld, Westphalia. When he was 9 years old, his family emigrated to England. He was educated at Leighton Park School in Reading, a public school with a reputation for academic excellence that had been founded by Quakers in 1890. In 1945 he obtained a place at Guy’s Hospital Medical School, London, where he soon established himself as a talented medical student, winning the junior proficiency prize, the Sand-Cox prize in physiology, the Michael Harris prize in anatomy, and the Beaney prize in pathology. Three years later, he gained a BSc (Hons) in chemistry before qualifying in medicine. After house jobs in Romford, Essex, and Enfield, Ken spent his two year period of national service in British military hospitals in Wuppertal, Hostert, and Munster, Germany. On his return to England, in 1955, he was appointed a junior lecturer to Professor W R Spurrell at Guy’s, who was actively involved in the study of gastric …

The grand scheme of public disclosure

Although Tavare presents evidence for both sides of the argument on the quality of performance data, he still gives a positive slant to public disclosure,1 probably because of bias in selecting opinions from those with a vested interest in the venture. For example, the names of the founders of Dr Foster appear …

Might BMJ lead crusade against reporting only relative risks

Surely enough has been written over the past 20 years to show that the use of relative risk reductions in research publications is nothing but a deception? Yet this misleading measure of efficacy continues to sully the pages of medical journals: McCartney’s article on the polypill draws attention to the …

Claims do not hold water

Bolland and colleagues claim that their reanalysis of the Women’s Health Initiative calcium and vitamin D study shows that long term treatment with calcium and vitamin D increases the risk of cardiovascular events in women not taking calcium supplements at randomisation.1 But is this claim supported …

Gastric lipoma: a rare cause of iron-deficiency anaemia.

A 60-year-old man presented with asymptomatic iron deficiency anaemia. Gastroscopy revealed a large submucosal mass involving the antrum and duodenal bulb with a deep ulcer in the gastric component. Computed tomography scan showed the lesion to be a rare gastric lipoma.

Secondary prevention of heart disease with statins: Groups were as different as chalk and cheese

Editor—Wei et al say that the results of their observational study of treatment with statins in routine clinical practice in Tayside are consistent with, and similar to, those reported in clinical trials.1 This conclusion is hardly justified. A cohort of patients, defined by a discharge diagnosis of myocardial infarction between January 1993 and December 2001, was studied to compare the outcomes in those treated with statins and those not receiving these drugs. The data clearly show that the two groups differed substantially in terms of age, sex, comorbidity, and other cardiovascular drug treatment—all of these factors were biased in favour of a poorer outcome in patients not given statins. Furthermore, the two groups belonged to different time periods, most of the untreated patients to the earlier part and most of those given statins to the later part of the study. Given the changes between 1993 and 2001—including, for example, those related to the management of acute myocardial infarction and the criteria for diagnosis—it is only to be expected that the untreated group would have a higher risk of subsequent cardiovascular events. Multivariate analysis was reported as showing that statins reduced all cause mortality by 31% and the composite end point of myocardial infarction plus cardiovascular mortality by 18%. These results should be greeted with scepticism. Firstly, many trials of statins have failed to show any reduction whatsoever in all cause mortality.2-5 Secondly, as observed in other studies, any reduction in all cause mortality would be expected to be much less than that for composite cardiovascular end points.2,4,5 The two groups in this study were as different as chalk and cheese. It should come as no surprise, therefore, that the results were anomalous. In such circumstances, no amount of statistical trickery could be expected to furnish anything of value.