Jeremy Howick

Evidence based medicine: a movement in crisis?

Trisha Greenhalgh and colleagues argue that, although evidence based medicine has had many benefits, it has also had some negative unintended consequences. They offer a preliminary agenda for the movement’s renaissance, refocusing on providing useable evidence that can be combined with context and professional expertise so that individual patients get optimal treatment

The evolution of evidence hierarchies: what can Bradford Hill's 'guidelines for causation' contribute?

High quality randomized controlled trials (RCTs) (concealed allocation, relevant groups blinded and sufficiently powered, etc.) will usually provide sufficient evidence to establish that a particular treatment caused an outcome. Yet sufficiently well‐conducted RCTs are rare.1 Trials can be under‐powered,2 or unsuccessfully blinded,3,4 and often suffer from many undetected biases. The results of most RCTs are therefore often insufficient to establish causation. At the same time, RCTs are often not required to establish causation.5 Treatments including the Heimlich manoeuvre, cardiac defibrillation and parachutes to prevent death6 have never been tested in RCTs, yet their effectiveness is surely strongly supported by evidence. Evidence‐grading systems that place randomized trials at the top of a hierarchy7–13 will deliver misleading conclusions in cases where RCTs are insufficient or unnecessary. According to these hierarchies, trails of homeopathy – often generating positive results and generally of higher quality than RCTs of conventional treatments14 – will be considered to provide strong evidence, whereas the evidence base for the Heimlich manoeuvre to unblock airways and parachutes to prevent death will be judged as less strongly supported by evidence. Sir Austin Bradford Hill, in a widely‐cited ‘pre‐EBM’ system for appraising evidence, suggested that several relevant factors must be considered before concluding causation. We investigated and revised the Bradford Hill ‘guidelines for causation’, in order to refine our intuitions about whether to believe that intervention is effective. Our intention is not to debunk previous attempts to grade evidence, but rather to contribute to their natural evolution and development.

Current and future use of point-of-care tests in primary care: an international survey in Australia, Belgium, The Netherlands, the UK and the USA

Objective Despite the growing number of point-of-care (POC) tests available, little research has assessed primary care clinician need for such tests. We therefore aimed to determine which POC tests they actually use or would like to use (if not currently available in their practice). Design Cross-sectional survey. Setting Primary care in Australia, Belgium (Flanders region only), the Netherlands, the UK and the USA. Participants Primary care doctors (general practitioners, family physicians). Main measures We asked respondents to (1) identify conditions for which a POC test could help inform diagnosis, (2) from a list of tests provided: evaluate which POC tests they currently use (and how frequently) and (3) determine which tests (from that same list) they would like to use in the future (and how frequently). Results 2770 primary care clinicians across five countries responded. Respondents in all countries wanted POC tests to help them diagnose acute conditions (infections, acute cardiac disease, pulmonary embolism/deep vein thrombosis), and some chronic conditions (diabetes, anaemia). Based on the list of POC tests provided, the most common tests currently used were: urine pregnancy, urine leucocytes or nitrite and blood glucose. The most commonly reported tests respondents expressed a wish to use in the future were: D-dimer, troponin and chlamydia. The UK and the USA reported a higher actual and desired use for POC tests than Australia, Belgium and the Netherlands. Our limited data suggest (but do not confirm) representativeness. Conclusions Primary care clinicians in all five countries expressed a desire for POC tests to help them diagnose a range of acute and chronic conditions. Rates of current reported use and desired future use were generally high for a small selection of POC tests, but varied across countries. Future research is warranted to explore how specific POC tests might improve primary care.

The need for randomization in animal trials: an overview of systematic reviews.

Background and Objectives Randomization, allocation concealment, and blind outcome assessment have been shown to reduce bias in human studies. Authors from the Collaborative Approach to Meta Analysis and Review of Animal Data from Experimental Studies (CAMARADES) collaboration recently found that these features protect against bias in animal stroke studies. We extended the scope the work from CAMARADES to include investigations of treatments for any condition. Methods We conducted an overview of systematic reviews. We searched Medline and Embase for systematic reviews of animal studies testing any intervention (against any control) and we included any disease area and outcome. We included reviews comparing randomized versus not randomized (but otherwise controlled), concealed versus unconcealed treatment allocation, or blinded versus unblinded outcome assessment. Results Thirty-one systematic reviews met our inclusion criteria: 20 investigated treatments for experimental stroke, 4 reviews investigated treatments for spinal cord diseases, while 1 review each investigated treatments for bone cancer, intracerebral hemorrhage, glioma, multiple sclerosis, Parkinsons disease, and treatments used in emergency medicine. In our sample 29% of studies reported randomization, 15% of studies reported allocation concealment, and 35% of studies reported blinded outcome assessment. We pooled the results in a meta-analysis, and in our primary analysis found that failure to randomize significantly increased effect sizes, whereas allocation concealment and blinding did not. In our secondary analyses we found that randomization, allocation concealment, and blinding reduced effect sizes, especially where outcomes were subjective. Conclusions Our study demonstrates the need for randomization, allocation concealment, and blind outcome assessment in animal research across a wide range of outcomes and disease areas. Since human studies are often justified based on results from animal studies, our results suggest that unduly biased animal studies should not be allowed to constitute part of the rationale for human trials.

Placebo Use in the United Kingdom: Results from a National Survey of Primary Care Practitioners

Objectives Surveys in various countries suggest 17% to 80% of doctors prescribe ‘placebos’ in routine practice, but prevalence of placebo use in UK primary care is unknown. Methods We administered a web-based questionnaire to a representative sample of UK general practitioners. Following surveys conducted in other countries we divided placebos into ‘pure’ and ‘impure’. ‘Impure’ placebos are interventions with clear efficacy for certain conditions but are prescribed for ailments where their efficacy is unknown, such as antibiotics for suspected viral infections. ‘Pure’ placebos are interventions such as sugar pills or saline injections without direct pharmacologically active ingredients for the condition being treated. We initiated the survey in April 2012. Two reminders were sent and electronic data collection closed after 4 weeks. Results We surveyed 1715 general practitioners and 783 (46%) completed our questionnaire. Our respondents were similar to those of all registered UK doctors suggesting our results are generalizable. 12% (95% CI 10 to 15) of respondents used pure placebos while 97% (95% CI 96 to 98) used impure placebos at least once in their career. 1% of respondents used pure placebos, and 77% (95% CI 74 to 79) used impure placebos at least once per week. Most (66% for pure, 84% for impure) respondents stated placebos were ethical in some circumstances. Conclusion and implications Placebo use is common in primary care but questions remain about their benefits, harms, costs, and whether they can be delivered ethically. Further research is required to investigate ethically acceptable and cost-effective placebo interventions.

Understanding GRADE: an introduction.

Grading of recommendations, assessment, development, and evaluations (GRADE) is arguably the most widely used method for appraising studies to be included in systematic reviews and guidelines. In order to use the GRADE system or know how to interpret it when reading reviews, reading several articles and attending a workshop are required. Moreover, the GRADE system is not covered in standard medical textbooks. Here, we explain GRADE concisely with the use of examples so that students and other researchers can understand it.

The importance of values in evidence-based medicine.

BackgroundEvidence-based medicine (EBM) has always required integration of patient values with ‘best’ clinical evidence. It is widely recognized that scientific practices and discoveries, including those of EBM, are value-laden. But to date, the science of EBM has focused primarily on methods for reducing bias in the evidence, while the role of values in the different aspects of the EBM process has been almost completely ignored.DiscussionIn this paper, we address this gap by demonstrating how a consideration of values can enhance every aspect of EBM, including: prioritizing which tests and treatments to investigate, selecting research designs and methods, assessing effectiveness and efficiency, supporting patient choice and taking account of the limited time and resources available to busy clinicians. Since values are integral to the practice of EBM, it follows that the highest standards of EBM require values to be made explicit, systematically explored, and integrated into decision making.SummaryThrough ‘values based’ approaches, EBM’s connection to the humanitarian principles upon which it was founded will be strengthened.

Are treatments more effective than placebos? A systematic review and meta-analysis.

Background Placebos are widely used in clinical practice in spite of ethical restrictions. Whether such use is justified depends in part on the relative benefit of placebos compared to ‘active’ treatments. A direct test for differences between placebo and ‘active’ treatment effects has not been conducted. Objectives We aimed to test for differences between treatment and placebo effects within similar trial populations. Data Sources A Cochrane Review compared placebos with no treatment in three-armed trials (no treatment, placebo, and treatment). We added an analysis of treatment and placebo differences within the same trials. Synthesis Methods For continuous outcomes we compared mean differences between placebo and no treatment with mean differences between treatment and placebo. For binary outcomes we compared the risk ratio for treatment benefit (versus placebo) with the risk ratio for placebo benefit (versus no treatment). We conducted several preplanned subgroup analyses: objective versus subjective outcomes, conditions tested in three or more trials, and trials with varying degrees of bias. Results In trials with continuous outcomes (n = 115) we found no difference between treatment and placebo effects (MD = −0.29, 95% CI −0.62 to 0.05, P = 0.10). In trials with binary outcomes (n = 37) treatments were significantly more effective than placebos (RRR = 0.72, 95%CI = 0.61 to 0.86, P = 0.0003). Treatment and placebo effects were not different in 22 out of 28 predefined subgroup analyses. Of the six subgroups with differences treatments were more effective than placebos in five. However when all criteria for reducing bias were ruled out (continuous outcomes) placebos were more effective than treatments (MD = 1.59, 95% CI = 0.40 to 2.77, P = 0.009). Conclusions and Implications Placebos and treatments often have similar effect sizes. Placebos with comparatively powerful effects can benefit patients either alone or as part of a therapeutic regime, and trials involving such placebos must be adequately blinded.

Primary care clinicians’ attitudes towards point-of-care blood testing: a systematic review of qualitative studies

BackgroundPoint-of-care blood tests are becoming increasingly available and could replace current venipuncture and laboratory testing for many commonly used tests. However, at present very few have been implemented in most primary care settings. Understanding the attitudes of primary care clinicians towards these tests may help to identify the barriers and facilitators to their wider adoption. We aimed to systematically review qualitative studies of primary care clinicians’ attitudes to point-of-care blood tests.MethodsWe systematically searched Medline, Embase, ISI Web of Knowledge, PsycINFO and CINAHL for qualitative studies of primary care clinicians’ attitudes towards point-of-care blood tests in high income countries. We conducted a thematic synthesis of included studies.ResultsOur search identified seven studies, including around two hundred participants from Europe and Australia. The synthesis generated three main themes: the impact of point-of-care testing on decision-making, diagnosis and treatment; impact on clinical practice more broadly; and impact on patient-clinician relationships and perceived patient experience. Primary care clinicians believed point-of-care testing improved diagnostic certainty, targeting of treatment, self-management of chronic conditions, and clinician-patient communication and relationships. There were concerns about test accuracy, over-reliance on tests, undermining of clinical skills, cost, and limited usefulness.ConclusionsWe identified several perceived benefits and barriers regarding point-of-care tests in primary care. These imply that if point-of-care tests are to become more widely adopted, primary care clinicians require evidence of their accuracy, rigorous testing of the impact of introduction on patient pathways and clinical practice, and consideration of test funding.

Evidence-based mechanistic reasoning

Systematic reviews of high quality randomized trials generally count as the ‘best evidence’. However, well-conducted randomized trials are sometimes unavailable, unfeasible, unethical or unnecessary. In such cases other forms of evidence must be considered. Many EBM proponents accept mechanistic reasoning (‘pathophysiologic rationale’) for generalizability, hypothesis generation, ruling out implausible hypotheses, and for supporting efficacy in the absence of other ‘stronger’ forms of evidence. Yet because mechanistic reasoning has often led us astray, most EBM proponents are justifiably sceptical about using mechanistic reasoning as evidence for efficacy. We suggest that the scepticism about the value of mechanistic reasoning should not extend to high quality mechanistic reasoning. Just as poor quality randomized trials (that are unblinded, underpowered or biased, that employ unconcealed allocation, or otherwise biased) will not provide high quality evidence for efficacy, so poor quality mechanistic reasoning will be unreliable. In this theoretical exploration we suggest that mechanistic reasoning involving a not incomplete inferential chain and that takes potential complexity into account can and should be used as evidence of efficacy. We support our rules for mechanistic evidence with three examples.