Paul A. Fontelo

Utilization of the PICO framework to improve searching PubMed for clinical questions

BackgroundSupporting 21st century health care and the practice of evidence-based medicine (EBM) requires ubiquitous access to clinical information and to knowledge-based resources to answer clinical questions. Many questions go unanswered, however, due to lack of skills in formulating questions, crafting effective search strategies, and accessing databases to identify best levels of evidence.MethodsThis randomized trial was designed as a pilot study to measure the relevancy of search results using three different interfaces for the PubMed search system. Two of the search interfaces utilized a specific framework called PICO, which was designed to focus clinical questions and to prompt for publication type or type of question asked. The third interface was the standard PubMed interface readily available on the Web. Study subjects were recruited from interns and residents on an inpatient general medicine rotation at an academic medical center in the US. Thirty-one subjects were randomized to one of the three interfaces, given 3 clinical questions, and asked to search PubMed for a set of relevant articles that would provide an answer for each question. The success of the search results was determined by a precision score, which compared the number of relevant or gold standard articles retrieved in a result set to the total number of articles retrieved in that set.ResultsParticipants using the PICO templates (Protocol A or Protocol B) had higher precision scores for each question than the participants who used Protocol C, the standard PubMed Web interface. (Question 1: A = 35%, B = 28%, C = 20%; Question 2: A = 5%, B = 6%, C = 4%; Question 3: A = 1%, B = 0%, C = 0%) 95% confidence intervals were calculated for the precision for each question using a lower boundary of zero. However, the 95% confidence limits were overlapping, suggesting no statistical difference between the groups.ConclusionDue to the small number of searches for each arm, this pilot study could not demonstrate a statistically significant difference between the search protocols. However there was a trend towards higher precision that needs to be investigated in a larger study to determine if PICO can improve the relevancy of search results.

Evidence-based medicine among internal medicine residents in a community hospital program using smart phones.

BackgroundThis study implemented and evaluated a point-of-care, wireless Internet access using smart phones for information retrieval during daily clinical rounds and academic activities of internal medicine residents in a community hospital. We did the project to assess the feasibility of using smart phones as an alternative to reach online medical resources because we were unable to find previous studies of this type. In addition, we wanted to learn what Web-based information resources internal medicine residents were using and whether providing bedside, real-time access to medical information would be perceived useful for patient care and academic activities.MethodsWe equipped the medical teams in the hospital wards with smart phones (mobile phone/PDA hybrid devices) to provide immediate access to evidence-based resources developed at the National Library of Medicine as well as to other medical Websites. The emphasis of this project was to measure the convenience and feasibility of real-time access to current medical literature using smart phones.ResultsThe smart phones provided real-time mobile access to medical literature during daily rounds and clinical activities in the hospital. Physicians found these devices easy to use. A post-study survey showed that the information retrieved was perceived to be useful for patient care and academic activities.ConclusionIn community hospitals and ambulatory clinics without wireless networks where the majority of physicians work, real-time access to current medical literature may be achieved through smart phones. Immediate availability of reliable and updated information obtained from authoritative sources on the Web makes evidence-based practice in a community hospital a reality.

The PDA as a portal to knowledge sources in a wireless setting.

Two recent parallel developments, the widespread deployment of wireless networks and increased use of handheld devices like the personal digital assistant (PDA), have contributed to the development of mobile access to the Internet. Recent surveys show that approximately 25% or more of physicians use PDAs. Although used mainly for personal information management and static medical applications, PDAs have capabilities to connect to the Internet. We studied the use of handheld devices to access MEDLINE and other knowledge sources in a wireless setting.

Mobile devices in health education: current use and practice

The increasing amount of new scientific information made available by computers and the Internet is demonstrated by the growing number of available health sciences journals. Medical students, nursing students, those in other health science disciplines, and clinicians need to make information more manageable and accessible, especially at the point of care. Technological advances are available to assist them in keeping up with more information than what was accessible to their mentors. In this report, we examine technological advancements and Internet technologies, particularly mobile devices that are currently available to educators and students. We review these advances with respect to how they (a) deliver content, (b) serve different purposes in the classroom and beyond, (c) play different roles in a health sciences student’s learning experience, and (d) make available a variety of resources to teachers. These innovations are broadly categorized as mobile technology, Web 2.0 innovations, Learning Management Systems/e-Learning, and medical simulations. Educators and learning institutions must be equipped for the future and maintain proficiency in the ever-expanding world of human medicine by adopting effective strategies that will infuse their lessons with the appropriate technology and allow their students to achieve their maximum potential. Given all these, we believe that the future of the health student and professional will be in mobile computing.

A UMLS-based spell checker for natural language processing in vaccine safety

BackgroundThe Institute of Medicine has identified patient safety as a key goal for health care in the United States. Detecting vaccine adverse events is an important public health activity that contributes to patient safety. Reports about adverse events following immunization (AEFI) from surveillance systems contain free-text components that can be analyzed using natural language processing. To extract Unified Medical Language System (UMLS) concepts from free text and classify AEFI reports based on concepts they contain, we first needed to clean the text by expanding abbreviations and shortcuts and correcting spelling errors. Our objective in this paper was to create a UMLS-based spelling error correction tool as a first step in the natural language processing (NLP) pipeline for AEFI reports.MethodsWe developed spell checking algorithms using open source tools. We used de-identified AEFI surveillance reports to create free-text data sets for analysis. After expansion of abbreviated clinical terms and shortcuts, we performed spelling correction in four steps: (1) error detection, (2) word list generation, (3) word list disambiguation and (4) error correction. We then measured the performance of the resulting spell checker by comparing it to manual correction.ResultsWe used 12,056 words to train the spell checker and tested its performance on 8,131 words. During testing, sensitivity, specificity, and positive predictive value (PPV) for the spell checker were 74% (95% CI: 74–75), 100% (95% CI: 100–100), and 47% (95% CI: 46%–48%), respectively.ConclusionWe created a prototype spell checker that can be used to process AEFI reports. We used the UMLS Specialist Lexicon as the primary source of dictionary terms and the WordNet lexicon as a secondary source. We used the UMLS as a domain-specific source of dictionary terms to compare potentially misspelled words in the corpus. The prototype sensitivity was comparable to currently available tools, but the specificity was much superior. The slow processing speed may be improved by trimming it down to the most useful component algorithms. Other investigators may find the methods we developed useful for cleaning text using lexicons specific to their area of interest.

Effect of image compression on telepathology. A randomized clinical trial.

CONTEXT For practitioners deploying store-and-forward telepathology systems, optimization methods such as image compression need to be studied. OBJECTIVE To determine if Joint Photographic Expert Group (JPG or JPEG) compression, a glossy image compression algorithm, negatively affects the accuracy of diagnosis in telepathology. DESIGN Double-blind, randomized, controlled trial. SETTING University-based pathology departments. PARTICIPANTS Resident and staff pathologists at the University of Illinois, Chicago, and University of Cincinnati, Cincinnati, Ohio. INTERVENTION Compression of raw images using the JPEG algorithm. MAIN OUTCOME MEASURES Image acceptability, accuracy of diagnosis, confidence level of pathologist, image quality. RESULTS There was no statistically significant difference in the diagnostic accuracy between noncompressed (bit map) and compressed (JPG) images. There were also no differences in the acceptability, confidence level, and perception of image quality. Additionally, rater experience did not significantly correlate with degree of accuracy. CONCLUSIONS For providers practicing telepathology, JPG image compression does not negatively affect the accuracy and confidence level of diagnosis. The acceptability and quality of images were also not affected.

SLIM: an alternative Web interface for MEDLINE/PubMed searches - a preliminary study

BackgroundWith the rapid growth of medical information and the pervasiveness of the Internet, online search and retrieval systems have become indispensable tools in medicine. The progress of Web technologies can provide expert searching capabilities to non-expert information seekers. The objective of the project is to create an alternative search interface for MEDLINE/PubMed searches using JavaScript slider bars. SLIM, or Slider Interface for MEDLINE/PubMed searches, was developed with PHP and JavaScript. Interactive slider bars in the search form controlled search parameters such as limits, filters and MeSH terminologies. Connections to PubMed were done using the Entrez Programming Utilities (E-Utilities). Custom scripts were created to mimic the automatic term mapping process of Entrez. Page generation times for both local and remote connections were recorded.ResultsAlpha testing by developers showed SLIM to be functionally stable. Page generation times to simulate loading times were recorded the first week of alpha and beta testing. Average page generation times for the index page, previews and searches were 2.94 milliseconds, 0.63 seconds and 3.84 seconds, respectively. Eighteen physicians from the US, Australia and the Philippines participated in the beta testing and provided feedback through an online survey. Most users found the search interface user-friendly and easy to use. Information on MeSH terms and the ability to instantly hide and display abstracts were identified as distinctive features.ConclusionSLIM can be an interactive time-saving tool for online medical literature research that improves user control and capability to instantly refine and refocus search strategies. With continued development and by integrating search limits, methodology filters, MeSH terms and levels of evidence, SLIM may be useful in the practice of evidence-based medicine.

Electronic health records improve clinical note quality

Background and objective The clinical note documents the clinicians information collection, problem assessment, clinical management, and its used for administrative purposes. Electronic health records (EHRs) are being implemented in clinical practices throughout the USA yet it is not known whether they improve the quality of clinical notes. The goal in this study was to determine if EHRs improve the quality of outpatient clinical notes. Materials and methods A five and a half year longitudinal retrospective multicenter quantitative study comparing the quality of handwritten and electronic outpatient clinical visit notes for 100 patients with type 2 diabetes at three time points: 6 months prior to the introduction of the EHR (before-EHR), 6 months after the introduction of the EHR (after-EHR), and 5 years after the introduction of the EHR (5-year-EHR). QNOTE, a validated quantitative instrument, was used to assess the quality of outpatient clinical notes. Its scores can range from a low of 0 to a high of 100. Sixteen primary care physicians with active practices used QNOTE to determine the quality of the 300 patient notes. Results The before-EHR, after-EHR, and 5-year-EHR grand mean scores (SD) were 52.0 (18.4), 61.2 (16.3), and 80.4 (8.9), respectively, and the change in scores for before-EHR to after-EHR and before-EHR to 5-year-EHR were 18% (p<0.0001) and 55% (p<0.0001), respectively. All the element and grand mean quality scores significantly improved over the 5-year time interval. Conclusions The EHR significantly improved the overall quality of the outpatient clinical note and the quality of all its elements, including the core and non-core elements. To our knowledge, this is the first study to demonstrate that the EHR significantly improves the quality of clinical notes.

Radiation Exposure and Cost Influence Physician Medical Image Decision Making A Randomized Controlled Trial

Background: It is estimated that 20%–40% of advanced medical imaging in the United States is unnecessary, resulting in patient overexposure to radiation and increasing the cost of care. Previous imaging utilization studies have focused on clinical appropriateness. An important contributor to excessive use of advanced imaging may be a physician “knowledge gap” regarding the safety and cost of the tests. Objectives: To determine whether safety and cost information will change physician medical image decision making. Research Design: Double-blinded, randomized controlled trial. Following standardized case presentation, physicians made an initial imaging choice. This was followed by the presentation of guidelines, radiation exposure and health risk, and cost information. Results: Approximately half (57 of 112, 50.9%) of participants initially selected computed tomography (CT). When presented with guideline recommendations, participants did not modify their initial imaging choice (P=0.197). A significant reduction (56.3%, P<0.001) in CT ordering occurred after presentation of radiation exposure/health risk information; ordering changed to magnetic resonance imaging or ultrasound (US). A significant reduction (48.3%, P<0.001) in CT and magnetic resonance imaging ordering occurred after presentation of Medicare reimbursement information; ordering changed to US. The majority of physicians (31 of 40, 77.5%) selecting US never modified their ordering. No significant relationship between physician demographics and decision making was observed. Conclusions: This study suggests that physician decision making can be influenced by safety and cost information and the order in which information is provided to physicians can affect their decisions.

Virtual Microscopy: Potential Applications in Medical Education and Telemedicine in Countries with Developing Economies

We evaluated the diagnostic accuracy of a virtual microscopy setup using surgical pathology specimens commonly encountered in a university hospital setting. The high quality images, Internet sharing and collaborative capability, interactivity, and ease of use suggested to us that this might have applications in countries with developing economies. We discuss the development process and its potential applications in medical education and telemedicine in countries with developing economies.