Gale G. Hannigan

New Mexico practitioners' access to and satisfaction with online clinical information resources: an interview study using qualitative data analysis software.

QUESTIONS What information resources are available to health care practitioners not affiliated with the University of New Mexico? How satisfied are they with those resources? SETTING The state is rural and medically underserved. METHODS The authors interviewed practitioners, using a nine-item guide. Interview transcripts were coded using QSR NVivo 9 software. MAIN RESULTS Fifty-one practitioners were interviewed. Most use online information resources. Many have access to a point-of-care resource within an electronic health records system. They often expressed dissatisfaction with available patient education resources. CONCLUSION New Mexico practitioners routinely use electronic information resources but indicate they need better patient information.

Computers and medical information: an elective for fourth-year medical students.

The Computers and Medical Information elective is a collaborative effort to expose students to a variety of computer applications for medical information management. The course has a modular format so that students can work with a variety of people who are enthusiastic users of computer-based information systems. The elective emphasizes learning by doing. Faculty introduce concepts and systems and serve as guides in the use of systems. Students have rated the course positively and, after four years as an elective, some of the courses content has been integrated into the required curriculum.

Personalized Information Service for Clinicians: Users Like It

A Review of: Jerome, Rebecca N., Nunzia Bettinsoli Giuse, S. Trent Rosenbloom, and Patrick G. Arbogast. “Exploring Clinician Adoption of a Novel Evidence Request Feature in an Electronic Medical Record System.” Journal of the Medical Library Association 96.1 (Jan. 2008): 34-41, with online appendices. Abstract Objective – To examine physician use of an Evidence-Based Medicine (EBM) literature request service available to clinicians through the institution’s electronic medical record system (EMR). Specifically, the authors posed the following questions: 1) Did newly implemented marketing and communication strategies increase physicians’ use of the service? 2) How did clinicians rate the relevance of the information provided? 3) How was the information provided used and shared? Design – Ten-month, prospective, observational study employing a questionnaire, statistics, a focus group, and a “before and after marketing intervention” analysis. Setting – Adult primary care outpatient clinic in an academic medical centre. Subjects – Forty-eight attending and 89 resident physicians. Methods – In 2003, a new service was introduced that allowed physicians in the Adult Primary Care Center clinic to request evidence summaries from the library regarding complex clinical questions. Contact with the library was through the secure messaging feature of the institution’s electronic medical record (EMR). From March through July 2005, the librarian employed “standard” publicity methods (e-mail, flyers, posters, demonstrations) to promote the service. A focus group in July 2005 provided feedback about the service as well as recommendations about communicating its availability and utility. New communication methods were implemented, including a monthly electronic “current awareness” newsletter, more frequent visits by the librarian during resident clinic hours, and collaborations between the librarian and residents preparing for morning report presentations. At the end of the study period, a 25-item Web-based questionnaire was sent to the 137 physicians with access to the service. Main Results – During the 10-month study period, 23 unique users submitted a total of 45 questions to the EBM Literature Request Service. More questions were from attending physicians than residents: 36 (80%) vs. 9 (20%). At least one of the 23 users asked 12 (26%) of the questions. Utilization did not significantly change after the mid-study intervention. At the end of the study, 48 physicians (35%) completed the survey (32 attending physicians and 16 residents). While 94% of the respondents indicated awareness of the service, only 40% indicated using it. The 19 who used the service, on average, agreed that the information provided was relevant and “sometimes leads to a change in my clinical practice” (p.37). Those who indicated that they shared the information (n=15) mostly did so with other attending physicians and residents, but also mentioned sharing with fellows, patients, and nurses. Information was typically shared verbally but also by distributing a printout, forwarding by e-mail, and forwarding within the EMR message system. The information was used primarily for general self-education, instruction of trainees, and confirmation of a current plan. Conclusion – The newly implemented marketing and communication strategies did not significantly increase the use of the EBM Literature Request Service. Those who used the service found it relevant and often shared the information with others. Based on a small number of respondents and survey information, the librarian-provided EBM Literature Request Service was “well-received” (39).

Users’ Awareness of Electronic Books is Limited

A review of: Levine-Clark, Michael. “Electronic Book Usage: A Survey at the University of Denver.” Portal: Libraries and the Academy 6.3 (Jul. 2006): 285-99. Abstract Objective – To determine if university library users are aware of electronic books, and how and why electronic books are used. Design – Survey. Setting – University of Denver. Subjects – Two thousand sixty-seven graduate and undergraduate students, faculty, and staff. Methods – In Spring 2005, the University of Denver faculty, and graduate and undergraduate students were invited to participate in a survey about awareness and use of electronic books. A link to the survey was also posted on the library’s home page and on the university’s Web portal. The 19-question survey consisted of 11 questions to get feedback about electronic books in general, five questions focused on netLibrary, and the remaining were demographic questions. Eligibility to win one of two university bookstore gift certificates provided incentive to complete the survey. Main results – Surveys were completed by 2,067 respondents, including undergraduate students (30.1%), graduate students (39.1%), faculty (12.5%), and staff (11.8%). Results were reported by question, broken out by status (undergraduate students, graduate students, faculty) and/or by discipline (Business, Humanities, Nontraditional, Professional, Sciences, Social Sciences), and presented in tables or in the text. In general, most respondents (59.1%) were aware that the library provides access to electronic books. The library catalog and professors were the main ways respondents learned about electronic books. Approximately half (51.3%) indicated they had used an electronic book. Of those who indicated that they used electronic books (1,061 respondents), most (72%) had used electronic books more than once. The main reasons mentioned for choosing to use an electronic book included: no print version available, working from home makes getting to the library difficult, and searching text in an electronic book is easier. When asked about typical use of electronic books, most respondents indicated they read only a part of an electronic book; only 7.1% of 1,148 respondents indicated they read the entire electronic book. In answer to a question about choosing the print or electronic version of the same book, 60.7% responded that they would always or usually use print, and 21.5% indicated they would always or usually use electronic. The amount of material to read, the need to refer to the material at a later time, and the desire to annotate or highlight text are all factors that influence whether users read electronic books on a computer or PDA, or print out the material. U.S. government publications and netLibrary were the electronic resources used the most by survey participants. Conclusion – The results of this survey suggest the need to market availability of the library’s electronic books. Problems associated with the use of electronic books are related to reading large amounts of text on a computer screen, but a reported benefit is that searching text in an electronic book is easier. Responses to the survey suggest that the use of electronic resources may not be generic, but rather depends on the type of resource (content) being used. The author notes that this finding should lead to further investigation of which items will be preferred and used in which format.

E-mailed Evidence Based Summaries Impact Physician Learning More than Practice

A Review of: Grad, Roland M., Pierre Pluye, Jay Mercer, Bernard Marlow, Marie Eve Beauchamp, Michael Shulha, Janique Johnson-Lafleur and Sharon Wood-Dauphinee. “Impact of Research-based Synopses Delivered as Daily E-mail: A Prospective Observational Study.” Journal of the American Medical Informatics Association (2008)15.2: 240-5. Objective – To determine the use and construct validity of a method to assess the cognitive impact of information derived from daily e-mail evidence based summaries (InfoPOEMs), and to describe the self-reported impact of these InfoPOEMs. Design – Prospective, observational study over a period of 150 days employing a questionnaire and rating scale. Setting – This study was conducted via the Internet between September 8, 2006 and February 4, 2007. Subjects – Canadian Medical Association (CMA) members who received InfoPOEMs via e-mail as of September 2006 were invited to participate. For inclusion in the analyses, a participant was defined as a practising family physician or general practitioner who submitted at least five ratings of InfoPOEMS during the study period (n=1,007). Methods – Volunteers completed a demographic questionnaire and provided informed consent online. Each subsequent InfoPOEM delivered included a link to a “ten-item impact assessment scale” (241). Participants checked “all that apply” of descriptive statements such as: My practice was (will be improved); I learned something new; I think this information is potentially harmful. Each combination of selections made was considered a pattern of cognitive impact. College of Family Physicians of Canada received continuing medical education (CME) credit for each InfoPOEM rated. Data were collected by the CMA and forwarded weekly to the investigators who used descriptive statistics, principal component analysis, and multilevel factor analysis to analyze the data. Main Results – 1,007 participants rated an average of 61 InfoPOEMs (ranging from five to 111). A total of 61,493 patterns of cognitive impact were submitted. Eighty-five different patterns were observed, i.e., there were 85 different combinations of the scale’s statements used. Ten patterns accounted for 89.4% of the reports. The top five patterns were: I learned something new (35.2%); No impact (17.1%); This information confirmed I did (will do) the right thing (9.6%); I learned something new AND My practice will be improved (9.4%); and, I was reassured (5.6%). I disagree with this information was checked at least once by 10.3% of the participants, and 8.0% checked I think this information is potentially harmful at least once. Conclusion – The authors applied a cognitive assessment instrument to determine the impact of InfoPOEMs e-mailed to primary care physicians in Canada and found that ten combinations of impact descriptors accounted for 89.4% of the total reports. Most suggested a positive impact on knowledge or practice. Of the total, 17.1% indicated No impact and 1.8% indicated the participant was frustrated as there was not enough information or nothing useful.

Risk Profile May Affect Search Process but Not Results

Objective – To compare the use, in terms of process and outcomes, of electronic information resources by primary care physicians with different risk profiles and comfort with uncertainty. Design – Survey, and observation using “think-aloud” method. Setting – Physicians’ offices. Subjects – Canadian and U.S. primary care physicians who report seeing patients in clinic settings. Methods – Volunteers were recruited from personal contacts and the list of physicians who rate current studies for the McMaster Online Rating of Evidence (MORE) project. Physicians completed the Pearson scale to measure attitude toward risk and the Gerrity scale to measure comfort with uncertainty, and those who scored at the extremes of each of these two scales were included in the study (n=25), resulting in four groups (risk-seeking, risk-avoiding, uncertainty-stressed, uncertainty-unstressed). One researcher observed each of these physicians in their offices for an hour during which they completed questionnaires about their computer skills and familiarity with resources, answered multiple-choice clinical questions, and indicated level of certainty with regard to those answers (scale of 0 to 100%). Physicians also chose two of the clinical questions to answer using their own resources. The think-aloud method was employed, and transcripts were coded and analyzed. Main results – The study analysis included two comparisons: risk-seeking (11 subjects) versus risk-avoiding (11 subjects) physicians, and uncertainty-stressed (11 subjects) versus uncertainty-unstressed (10 subjects) physicians. Most physicians were included in both sets of analyses. The researchers found no association of risk attitude and uncertainty stress with computer skills nor with familiarity and use of specific information resources (Internet, MEDLINE, PIER, Clinical Evidence, and UpToDate). No differences were found for the following outcomes: time spent searching, answers correct before searching, answers correct after searching, and certainty of answer if answer is right, certainty of answer if answer is wrong. There was a statistically significant association of participants’ indicating certainty for answers that were correct versus those that were not correct (p<0.03). Across all groups, the percentage of correct answers after searching was low, with the uncertainty-stressed group achieving the highest at 55.5%. There were differences observed in search process, including a trend toward a statistically significant difference in the number and proportion of search methods used across groups between the risk-seeking versus the risk-avoiding groups. Risk-seekers and those not stressed by uncertainty used more search heuristics (rules of thumb or guides that work in most circumstances) than their counterparts when searching. Those stressed by uncertainty tended to use MEDLINE less than those not stressed by uncertainty. Conclusion – Based on a small sample, it appears that primary care physicians who are risk-seeking and/or less stressed by uncertainty have the same (poor) search results as their more risk-averse and stressed-by-uncertainty colleagues although their search processes may differ.

Identifying Relevant Controlled Clinical Trials for Systematic Reviews Requires Searching Multiple Resources – and, Even Then, Comprehensiveness is Questionable

A review of: Crumley, Ellen T., Wiebe, Natasha, Cramer, Kristie, Klassen, Terry P., Hartling, Lisa. “Which resources should be used to identify RCT/CCTs for systematic reviews: a systematic review.” BMC Medical Research Methodology 5:24 (2005) doi:10.1186/1471-2288-5-24 (available from: http://www.biomedcentral.com/1471-2288/5/24 . Objective – To determine the value of searching different resources to identify relevant controlled clinical trial reports for systematic reviews. Design – Systematic review. Methods – Seven electronic databases (MEDLINE, EMBASE, CINAHL, ERIC, PsycINFO, Web of Science, Cochrane Library) were searched to April 2004;four journals (Health Information & Libraries Journal - formerly Health Libraries Review, Hypothesis, Journal of the Medical Library Association - formerly Bulletin of the Medical Library Association, Medical Reference Services Quarterly were handsearched from 1990 to 2004; all abstracts of the Cochrane Colloquia (1993-2003) were handsearched; key authors were contacted and relevant article references screened. Two reviewers independently screened results for studies that compared two or more resources to find RCTs or CCTs using defined inclusion and exclusion criteria. Two reviewers assessed studies for quality using four criteria: adequate descriptions of what the search was attempting to identify, the methods used to search, the reference standard; and, evidence that bias was avoided in selection of relevant studies. Screening and assessment differences between reviewers were resolved through discussion. Using a standard form, one investigator extracted data for each study, such as study design, results (e.g., recall, precision); a second investigator checked these data. Authors were contacted to provide missing data. Results were grouped by resources compared and these comparisons were summarized using medians and ranges. Search strategies were categorized as Complex (using a combination of types of search terms), Cochrane (the Cochrane Highly Sensitive Search Strategy or HSSS), Simple (using five or fewer search terms which may include a combination of MeSH, Publication Type, keywords), and Index (using one or two terms to check/verify if the study is in the database).1 Main results – Sixty-four studies met criteria for inclusion in the analysis. Four major comparisons were: MEDLINE vs. handsearch (n=22), MEDLINE vs. MEDLINE + handsearch (n=12), MEDLINE vs. other reference standard (n=18), and EMBASE vs. reference standard (n=13). Thirteen other comparisons had only one or two studies each. The most common comparison was between MEDLINE vs. Handsearching. Data analyzed from 23 studies and 22 unique topic comparisons showed a 58% median for search recall (range=7-97%). Data for search precision based on 12 studies and 11 unique topic comparisons indicated a median of 31% (range=0.03-78%). Data based on more than four comparisons, shows no median recall more than 75% (range=18-90%) and no median precision more than 40% (range=13-83%). Recall was higher for Trial Registries vs. Reference Standard (89%, range=84-95%) but these numbers were based on two studies and four comparisons; one study with two comparisons measured precision (range=96-97%) for Trial Registries vs Reference Standard. Subgroup analyses indicate that Complex and Cochrane searches each achieve better recall and precision compared to Simple searches. Forty-two studies reported reasons why searches miss relevant studies. The reason cited most often for electronic databases was inadequate or inappropriate indexing. Conclusion – The results of this systematic review indicate that no one resource results in particularly high recall or precision when searchers look for RCTs and CCTs.