Haynes Rb

The Architecture of Diagnostic Research

This is the second in a series of five articles Considerable effort has been expended at the interface between clinical medicine and scientific methods to achieve the maximum validity and usefulness of diagnostic tests. This article focuses on the specific kinds of questions that arise in diagnostic research and the study architectures (the conversions of these clinical questions into appropriate research designs) used to answer them. As an example we shall take shall take assessment of the value of the plasma concentration of B-type natriuretic peptide (BNP) in the diagnosis of left ventricular dysfunction.1 Randomised controlled trials are dealt with elsewhere. As in other forms of clinical research, there are several different ways studying the potential or real diagnostic value of a physical sign or laboratory test, and each is appropriate to one kind of question and inappropriate for others. Among the possible questions about the relation between a putative diagnostic test and a target disorder (for example, the concentration of BNP and left ventricular dysfunction), four are most relevant. #### Summary points Diagnostic studies should match methods to diagnostic questions The keys to validity in diagnostic test studies are Both specificity and sensitivity may change as the same diagnostic test is applied in primary, …

Filtering Medline for a clinical discipline: diagnostic test assessment framework

Objective To develop and test a Medline filter that allows clinicians to search for articles within a clinical discipline, rather than searching the entire Medline database. Design Diagnostic test assessment framework with development and validation phases. Setting Sample of 4657 articles published in 2006 from 40 journals. Reviews Each article was manually reviewed, and 19.8% contained information relevant to the discipline of nephrology. The performance of 1 155 087 unique renal filters was compared with the manual review. Main outcome measures Sensitivity, specificity, precision, and accuracy of each filter. Results The best renal filters combined two to 14 terms or phrases and included the terms “kidney” with multiple endings (that is, truncation), “renal replacement therapy”, “renal dialysis”, “kidney function tests”, “renal”, “nephr” truncated, “glomerul” truncated, and “proteinuria”. These filters achieved peak sensitivities of 97.8% and specificities of 98.5%. Performance of filters remained excellent in the validation phase. Conclusions Medline can be filtered for the discipline of nephrology in a reliable manner. Storing these high performance renal filters in PubMed could help clinicians with their everyday searching. Filters can also be developed for other clinical disciplines by using similar methods.

Searching for medical information online: a survey of Canadian nephrologists.

BACKGROUND Physicians often search for information to improve patient care. We evaluated how nephrologists use online information sources for this purpose. METHODS In this cross-sectional study (2008 to 2010), a random sample of Canadian nephrologists completed a survey of their online search practices. We queried respondents on their searching preferences, practices and use of 9 online information sources. RESULTS Respondents (n=115; 75% response rate) comprised both academic (59%) and community-based (41%) nephrologists. Respondents were an average of 48 years old and were in practice for an average of 15 years. Nephrologists used a variety of online sources to retrieve information on patient treatment including UpToDate (92%), PubMed (89%), Google (76%) and Ovid MEDLINE (55%). Community-based nephrologists were more likely to consult UpToDate first (91%), while academic nephrologists were divided between UpToDate (58%) and PubMed (41%). When searching bibliographic resources such as PubMed, 80% of nephrologists scan a maximum of 40 citations (the equivalent of 2 search pages in PubMed). Searching practices did not differ by age, sex or years in practice. CONCLUSIONS Nephrologists routinely use a variety of online resources to search for information for patient care. These include bibliographic databases, general search engines and specialized medical resources.

Need for Quality Improvement in Renal Systematic Reviews

BACKGROUND AND OBJECTIVES Systematic reviews of clinical studies aim to compile best available evidence for various diagnosis and treatment options. This study assessed the methodologic quality of all systematic reviews relevant to the practice of nephrology published in 2005. DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS We searched electronic databases (Medline, Embase, American College of Physicians Journal Club, Cochrane) and hand searched Cochrane renal group records. Clinical practice guidelines, case reports, narrative reviews, and pooled individual patient data meta-analyses were excluded. Methodologic quality was measured using a validated questionnaire (Overview Quality Assessment Questionnaire). For reviews of randomized trials, we also evaluated adherence to recommended reporting guidelines (Quality of Reporting of Meta-Analyses). RESULTS Ninety renal systematic reviews were published in year 2005, 60 of which focused on therapy. Many systematic reviews (54%) had major methodologic flaws. The most common review flaws were failure to assess the methodologic quality of included primary studies and failure to minimize bias in study inclusion. Only 2% of reviews of randomized trials fully adhered to reporting guidelines. A minority of journals (four of 48) endorsed adherence to consensus guidelines for review reporting, and these journals published systematic reviews of higher methodologic quality (P < 0.001). CONCLUSIONS The majority of systematic reviews had major methodologic flaws. The majority of journals do not endorse consensus guidelines for review reporting in their instructions to authors; however, journals that recommended such adherence published systemic reviews of higher methodologic quality.

Impact of PubMed search filters on the retrieval of evidence by physicians

Background: Physicians face challenges when searching PubMed for research evidence, and they may miss relevant articles while retrieving too many nonrelevant articles. We investigated whether the use of search filters in PubMed improves searching by physicians. Methods: We asked a random sample of Canadian nephrologists to answer unique clinical questions derived from 100 systematic reviews of renal therapy. Physicians provided the search terms that they would type into PubMed to locate articles to answer these questions. We entered the physician-provided search terms into PubMed and applied two types of search filters alone or in combination: a methods-based filter designed to identify high-quality studies about treatment (clinical queries “therapy”) and a topic-based filter designed to identify studies with renal content. We evaluated the comprehensiveness (proportion of relevant articles found) and efficiency (ratio of relevant to nonrelevant articles) of the filtered and nonfiltered searches. Primary studies included in the systematic reviews served as the reference standard for relevant articles. Results: The average physician-provided search terms retrieved 46% of the relevant articles, while 6% of the retrieved articles were nonrelevant (the ratio of relevant to nonrelevant articles was 1:16). The use of both filters together produced a marked improvement in efficiency, resulting in a ratio of relevant to nonrelevant articles of 1:5 (16 percentage point improvement; 99% confidence interval 9% to 22%; p < 0.003) with no substantive change in comprehensiveness (44% of relevant articles found; p = 0.55). Interpretation: The use of PubMed search filters improves the efficiency of physician searches. Improved search performance may enhance the transfer of research into practice and improve patient care.