David C. Slawson

Teaching evidence-based medicine: should we be teaching information management instead?

To encourage high-quality patient care guided by the best evidence, many medical schools and residencies are teaching techniques for critically evaluating the medical literature. While a large step forward, these skills of evidence-based medicine are necessary but not sufficient for the practice of contemporary medicine. Incorporating the best evidence into the real world of busy clinical practice requires the applied science of information management. Clinicians must learn the techniques and skills to focus on finding, evaluating, and using information at the point of care. This information must be both relevant to themselves and their patients as well as being valid. The authors discuss the need to teach the applied science of information management along with, or perhaps even instead of, teaching the basic science of evidence-based medicine. All students, residents, and practicing physicians need three skills to practice the best medicine: the ability to select foraging—“keeping up”—tools that filter information for relevance and validity, the skill to select and use a hunting—“just in time”—information tool that presents prefiltered information easily and in a quickly accessible form at the point of care, and the ability to make decisions by combining the best patient-oriented evidence with patient-centered care, placing the evidence in perspective with the needs and desires of the patient. This teaching of information management skills will prepare students and residents for a practice of medicine that requires lifelong learning.

Obtaining useful information from expert based sources

Abstract Clinicians rely heavily on expert based systems– consultation with colleagues, journal reviews and textbooks, and continuing education activities–to obtain new information. The usefulness of sources such as these depends on the relevance and validity of the information and the work it takes to obtain it. Useful information can be distinguished from the useless by asking three questions: Does the information focus on an outcome that my patients care about? Is the issue common to my practice, and is the intervention feasible? If the information is true, will it require me to change my practice? If the answer to all three questions is yes, then the information is a common POEM (patient oriented evidence that matters), capable of improving the lives of your patients and must be evaluated for validity. Conclusions based on results of well designed clinical trials are more likely to be valid than those drawn from observations based on experience in clinical practice. Both members of the team, clinicians and experts, must take responsibility for their respective roles.

What happened to the valid POEMs? A survey of review articles on the treatment of type 2 diabetes

Objective To evaluate systematically the review literature on type 2 diabetes to assess transmission of the findings of the United Kingdom prospective diabetes study (UKPDS), an important source of recent valid patient oriented evidence that matters (POEMs). Design Inception cohort analysis of the recent medical literature. Studies reviewed Thirty five reviews on treatment of type 2 diabetes. Main outcome measures Presentation of three types of information from UKPDS in review articles: recommendations based on patient oriented outcomes of study; recommendations contradicted by patient oriented outcomes of study; and recommendations based on disease oriented outcomes for which no patient oriented evidence exists. Results Only six of the reviews included the POEM that tight blood glucose control had no effect on diabetes related or overall mortality. Just seven mentioned that metformin treatment was associated with decreased mortality. Most (30) of the reviews did not report that diabetic patients with hypertension benefit more from good blood pressure control than good blood glucose control. No review pointed out that treatment of overweight patients with type 2 diabetes with insulin or sulphonylurea drugs had no effect on microvascular or macrovascular outcomes. Thirteen reviews recommended drugs as first line treatment for which we do not have patient oriented outcomes data. The average validity assessment score was 1.3 out of a possible score of 15 (95% confidence interval 0.9 to 1.8). Conclusions Review articles on the treatment of type 2 diabetes have not accurately transmitted the valid POEM results of the UKPDS to clinicians. Clinicians relying on review articles written by experts as a source of valid POEMs may be misled.

Updating clinical knowledge: an evaluation of current information alerting services.

PURPOSE Clinicians are overwhelmed by the sheer magnitude of new clinical information that is available on a daily basis. Despite the availability of information tools for finding this information and for updating clinical knowledge, no study has examined the quality of current information alerting services. METHODS We developed a 7-item checklist based on the principles of evidence-based medicine and assessed content validity with experts and face validity with practicing clinicians and clinician researchers. A list of clinical information updating tools (push tools) was generated in a systematic fashion and the checklist was used to rate the quality of these tools by two independent raters. Prior to rating all instruments, the raters were trained to achieve good agreement (>80%) by applying the checklist to two sets of three randomly selected tools. Descriptive statistics were used to describe the quality of the identified tools and inter-rater reliability was assessed using Intraclass Correlation (ICC). RESULTS Eighteen tools were identified using our systematic search. The average quality of these tools was 2.72 (range 0-7). Only two tools met all suggested criteria for quality. Inter-rater reliability for the 7-item checklist was .82 (ICC). CONCLUSIONS We developed a checklist that can be used to reliably assess the quality of clinical information updating tools. We found many shortcomings in currently available clinical knowledge updating tools. Ideally, these tools will evolve in the direction of applying basic evidence-based medicine principles to new medical information in order to increase their usefulness to clinicians.

Patient-Oriented Evidence that Matters (POEMs)™ Suggest Potential Clinical Topics for the Choosing Wisely™ Campaign

Objective: We propose a method of identifying clinical topics for campaigns like Choosing Wisely. Methods: In the context of an ongoing continuing medication education program, we analyzed ratings on every patient-oriented evidence that matters (POEM) synopsis delivered in 2012 and 2013. Given the objective of the Choosing Wisely campaign, we focused this analysis on 1 specific item in the validated questionnaire used by physicians to rate POEMs. This questionnaire item is about “avoiding an unnecessary diagnostic test or treatment.” For each POEM, we calculated frequencies and proportions for this item, then we identified the 20 POEMs that were most commonly associated with this item in 2012 and 2013. Finally, we determined whether the clinical topic of each of these POEMs was mentioned in the Choosing Wisely master list. Results: In 2012 and 2013 we received 506,809 completed questionnaires (or ratings) linked to 530 POEMs, for an average of 956 ratings per POEM. In 59% of these POEMs (n = 312), the most commonly expected type of health benefit was “avoiding an unnecessary diagnostic test or treatment.” We then identified the top 20 POEMs most commonly associated with this item in each year by ranking all 312 POEMs from the top down. The clinical topic addressed by 29 of these 40 POEMs was not addressed in the Choosing Wisely master list. These topics fell into 3 categories: diagnostic tests, medical interventions, and surgical interventions. Conclusion: “Big data” can identify clinical topics relevant to campaigns such as Choosing Wisely. This process represents a new way to inform the expert panel approach.

Should we be teaching information management instead of evidence-based medicine?

BackgroundTo encourage high-quality patient care guided by the best evidence, many medical schools and residencies are teaching techniques for critically evaluating the medical literature. While a large step forward in many regards, these skills of evidence-based medicine are necessary but not sufficient for the practice of contemporary medicine and surgery. Incorporating the best evidence into the real world of busy clinical practice requires the applied science of information management. Clinicians must learn the techniques and skills to focus on finding, evaluating, and using information at the point of care. This information must be both relevant to themselves and their patients and be valid.Where are we now?Today, orthopaedic surgery is in the post-Flexner era of passive didactic learning combined with the practical experience of surgery as taught by supervising experts. The medical student and house officer fill their memory with mountains of facts and classic references ‘just in case’ that information is needed. With libraries and now internet repositories of orthopaedic information, all orthopaedic knowledge can be readily accessed without having to store much in one’s memory. Evidence is often trumped by the opinion of a teacher or expert in the field.Where do we need to go?To improve the quality of orthopaedic surgery there should be application of the best evidence, changing practice where needed when evidence is available. To apply evidence, the evidence has to find a way into practice without the long pipeline of change that now exists. Evidence should trump opinion and unfounded practices.How do we get there?To create a curriculum and learning space for information management requires effort on the part of medical schools, residency programs and health systems. Internet sources need to be created that have the readily available evidence-based answers to patient issues so surgeons do not need to spend all the time necessary to research the questions on their own. Information management is built on a platform created by EBM but saves the surgeon time and improves accuracy by having experts validate the evidence and make it easily available.