Christiane Voisin

Strategies in assessing the need for updating evidence‐based guidelines for six clinical topics: an exploration of two search methodologies

BACKGROUND Because of the expense of updating practice guidelines, recent attention has focused on approaches that can reliably assess any updating required. Shekelle et al. (Journal of the American Medical Association 2001, 286, 1461-7) proposed using limited literature searches with expert involvement to reduce resources used in assessing whether a guideline needs updating. OBJECTIVES This study compared Shekelles method and the traditional systematic review method regarding comprehensiveness and effort. METHODS Two research teams translated critical key questions on screening test treatments and outcomes to Medical Subjects Headings (MeSH) and search strategies. They refined Shekelles method over three iterations, seeking greater efficiency. Using both methods independently, teams assessed the need to update six topics from the 1996 Guide to Clinical Preventive Services (US Preventive Services Task Force). Outcomes included completeness of study identification, importance of missed studies and effort involved. RESULTS The revised review approach produced fewer citations than the traditional approach and saved time, identifying fewer eligible studies than the traditional approach. None of the studies missed was rated important by the experts consulted. CONCLUSIONS The revised review approach provides an acceptable method for judging whether a guideline requires updating. Librarians were an integral part of the research process that streamlined the searches.

Assessing the impact of systematic reviews on future research: two case studies.

AIMS To evaluate the impact of systematic reviews on research funded by the Agency for Healthcare Research and Quality (AHRQ) through Evidence-based Practice Centers (EPCs), and to identify barriers to and facilitators for the effects of these documents on future research. METHODS & MATERIALS Two AHRQ systematic reviews were selected as case studies to evaluate their impact on future research. Key citations generated by these reports were identified through ISI Web of Science and PubMed Central and traced forward to identify effects on subsequent studies through citation analysis from updated systematic reviews on the topics. Requests for applications and program announcements from the NIH Guide for Grants and Contracts website were reviewed and dissemination data were obtained from AHRQ. Finally, interviews were conducted with 13 key informants to help identify short-, medium- and long-term impacts of the EPC reviews. RESULTS The measurable impact of the two EPC reviews is demonstrably greater on short-term outcomes (greater awareness of the issues) than on medium-term (e.g., the generation of new knowledge) or long-term outcomes (e.g., changes in patient practice or health outcomes). Factors such as the topic and the timing of the report relative to the development of the field may explain the impact of these two AHRQ reports. The degree to which the new research can be directly attributed to the AHRQ reviews remains unclear. Key informants discussed several benefits stemming from the EPC reports, including providing a foundation for the research community on which to build, heightening awareness of the gaps in knowledge, increasing the quality of research and sparking new directions of research. However, the degree to which these reports were influential hinged on several factors including marketing efforts, the very nature of the reports and other influences external to the EPC domain. CONCLUSIONS The findings outlined in this article illustrate the importance of numerous factors influencing future research: the breadth, specificity and readiness of the topic for more research, ongoing developments in the field, availability of funding and active engagement of champions. AHRQ and the EPCs may be able to improve the likelihood of impact by creating more targeted products, planning for and expanding dissemination activities, improving the readability and other attributes of the reports themselves, and actively involving funders early on and throughout the process of creating and publishing the reviews.