Susan F. Fesperman

Low quality of evidence for robot-assisted laparoscopic prostatectomy: results of a systematic review of the published literature.

BACKGROUND Robot-assisted laparoscopic prostatectomy (RALP) is displacing radical retropubic prostatectomy as the gold standard surgical approach for clinically localised prostate cancer in the United States and is also being increasingly used in Europe and other parts of the world. This trend has occurred despite the paucity of high-quality evidence to support its relative superiority to more established treatment modalities. OBJECTIVE We performed this study to critically assess the quality of published evidence on RALP to support this major shift in practice patterns. DESIGN, SETTING, AND PARTICIPANTS We conducted a systematic review of the published literature through Medline and Embase (1966 to December 2008). All original research publications on RALP were included. Editorials, letters to the editor, and review articles were excluded. MEASUREMENTS Two reviewers independently performed the data abstraction using a standardised form derived from the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) criteria. RESULTS AND LIMITATIONS Seventy-five original research publications met eligibility criteria. Fifty-five (73.3%) studies were published between 2005 and 2008, and 20 studies (26.7%) were published between 2001 and 2004. Approximately three-quarters of the studies were case series (74.7%), and only two (2.7%) randomised, controlled trials (RCT) were identified. Twelve authors cowrote 72% (54 of 75) of the published studies. Reporting of STROBE criteria ranged from 100.0% (scientific rationale/background explained) to 1.3% (consideration of sample size), with no improvement over time. The study was limited to published literature in the English language. CONCLUSIONS The published RALP literature is limited to observational studies of mostly low methodologic quality. Our findings draw into question to what extent valid conclusions about the relative superiority or equivalence of RALP to other surgical approaches can be drawn and whether published outcomes can be generalised to the broader community. There is an urgent need to raise the methodologic standards for clinical research on new urologic procedures and devices.

Assessment of the Methodological Quality of Systematic Reviews Published in the Urological Literature From 1998 to 2008

PURPOSE Well done systematic reviews provide the highest quality evidence for clinical questions of therapeutic effectiveness. We assessed the methodological quality of systematic reviews in the urological literature. MATERIALS AND METHODS We systematically investigated all systematic reviews published in 4 major urological journals from 1998 to 2008. Studies were identified using a predefined search strategy in PubMed and confirmed by a hand search of journal tables of contents. A validated 11-point instrument to assess the methodological quality of systematic reviews was applied by 2 independent reviewers after a pilot testing phase. Disagreements were discussed and resolved by consensus. RESULTS The systematic literature search identified 217 individual systematic reviews, of which 57 ultimately met study eligibility criteria. Ten (17.5%), 20 (35.1%) and 27 (47.4%) systematic reviews were published in 1998 to 2001, 2002 to 2005 and 2006 to 2008, respectively. Using the measurement tool to assess systematic reviews the mean +/- SD score was 4.8 +/- 2.0 points. Fewer than half of all systematic reviews performed a systematic literature search that included at least 2 databases (49.1%) or unpublished studies (31.6%), or provided a list of included and excluded studies (45.6%). Of the systematic reviews 63.2% assessed and documented the methodological quality of included studies. Systematic reviews with The Cochrane Collaboration authorship affiliation had a higher mean score than those with no such reported affiliation (6.5 +/- 1.2 vs 4.4 +/- 1.9 points, p <0.001). CONCLUSIONS Results suggest that an increasing number of systematic reviews are published in the urological literature. However, many systematic reviews fail to meet established methodological standards, raising concerns about validity. Increased efforts are indicated to promote quality standards for performing systematic reviews among the authors and readership of the urological literature.

Reporting of Harm in Randomized Controlled Trials Published in the Urological Literature

PURPOSE Evidence-based decision making seeks to balance potential benefits and harms (adverse effects) of health care interventions for an individual patient. We determined the prevalence and completeness of harm reporting in randomized controlled trials in the urological literature. MATERIALS AND METHODS We performed a systematic literature search of all randomized controlled trials of therapeutic interventions published in The Journal of Urology, Urology, European Urology and BJU International in 1996 and 2004. Each article was reviewed by 2 independent investigators for 10 harm reporting criteria recommended by the CONSORT group. Discrepancies were settled by discussion and consensus. RESULTS A total of 152 randomized controlled trials met the inclusion criteria, of which 109 (72%) reported adverse event outcomes. The median number of harm reporting criteria satisfied improved marginally from 1996 to 2004 (2.8 to 3.3, p = 0.36). A large proportion of studies failed to address harm in the abstract (55, 36%), introduction (71, 47%) and discussion (52, 34%). Few studies specified which adverse events were evaluated (21, 14%), when harm information was collected (32, 21%) or how the harm was attributed to the intervention (5, 3%). Only 48 (32%) articles provided reasons for patient withdrawal and 1 in 5 (33, 22%) reported the severity of adverse events. CONCLUSIONS Randomized controlled trials published in the urological literature contain significant deficiencies in adverse event reporting. These findings suggest the need for reporting standards for harm in urological journals. Improvements in adverse event reporting would permit a more balanced assessment of interventions and would enhance evidence-based urological practice.

How to Perform a Literature Search

PURPOSE Evidence based clinical practice seeks to integrate the current best evidence from clinical research with physician clinical expertise and patient individual preferences. We outline a stepwise approach to an effective and efficient search of electronic databases and introduce the reader to resources most relevant to the practicing urologist. MATERIALS AND METHODS The need for additional research evidence is introduced in the context of a urological clinical scenario. This information need is translated into a focused clinical question using the PICOT (population, intervention, comparison, outcome and type of study) format. This PICOT format provides key words for a literature search of pre-appraised evidence and original research studies that address the clinical scenario. RESULTS Available online resources can be broadly categorized into databases that focus on primary research studies, ie randomized, controlled trials, cohort studies, case-control or case series, such as MEDLINE and those that focus on secondary research that provides synthesis or synopsis of primary studies. Examples of such sources of pre-appraised evidence that are becoming increasingly relevant to urologists include BMJ Clinical Evidence, ACP Journal Club, The Cochrane Library and the National Guideline Clearinghouse. CONCLUSIONS The ability to search the medical literature in a time efficient manner represents an important part of an evidence based practice that is relevant to all urologists. The use of electronic databases of pre-appraised evidence can greatly expedite the search for high quality evidence, which is then integrated with urologist clinical skills and patient individual circumstances.

Evaluating the evidence : the methodological and reporting quality of comparative observational studies of surgical interventions in urological publications

To develop and apply a standardized evaluation form for assessing the methodological and reporting quality of observational studies of surgical interventions in urology.

Barriers to the Practice of Evidence-Based Urology

PURPOSE We investigated urologist perceptions of barriers to implementing evidence-based medicine in clinical practice. MATERIALS AND METHODS In April 2005 an Internet survey was conducted to assess American Urological Association member attitudes toward evidence-based medicine. This analysis presents the responses to an open-ended question about perceived barriers to implementing evidence-based clinical practice in urology. Two raters developed a coding scheme with 5 main categories of evidence concerns, system level factors, physician factors, patient factors and other barriers. Each rater independently assigned a category to each response. Discrepancies were resolved by consensus. RESULTS A total of 365 participants (72%) responded to the open-ended question, each providing up to 4 codable responses. Of the group 53% cited concerns about the evidence including the lack thereof, low quality, limited applicability and biased presentation. In addition, 37% reported system level factors such as issues of reimbursement, fear of litigation, problems with implementation, interference/bias by third parties, and expectations and attitudes of other providers as important issues. Physician factors and patient factors were further cited by 28% and 9%, respectively. In terms of the legitimacy of evidence-based medicine 9% expressed skepticism whereas 5% of respondents indicated that they saw no barriers to implementing evidence-based medicine in urology. CONCLUSIONS Efforts to promote evidence-based medicine in urology should focus not only on the generation of high quality clinical research but also on its unbiased reporting and timely dissemination. Concerted efforts should be made to reduce system level factors that hinder the implementation of evidence-based care.

Reporting quality and information consistency of randomized, controlled trials presented as abstracts at the American Urological Association annual meetings.

PURPOSE We assessed the quality of randomized, controlled trial reporting in abstracts from the annual meetings of the American Urological Association and determined whether the information provided is consistent with subsequent full text publications. MATERIALS AND METHODS All randomized, controlled trials presented in abstract form at the 2002 and 2003 American Urological Association annual meetings were identified for review. A systematic PubMed search based on authorship and key words from the study title was done to identify all subsequent full text publications. A standardized evaluation form was developed based on the published literature, pilot tested in a separate sample and applied by 2 independent reviewers. RESULTS A total of 126 randomized, controlled trials were identified for review, including 56 in 2002 and 70 in 2003. Approximately a third of the trials (43 or 34.1%) identified the study design as a randomized, controlled trial in the abstract title. The method of randomization, allocation concealment and blinding was reported in 0% (0), 0% (0) and 40.5% (51) of studies, respectively. Mean/median followup was provided in 27.0% of studies (34). Of 126 randomized, controlled trials presented in abstract form 62.7% (79) were subsequently published as full text articles. Study sample size and the number of randomized subjects differed in 24.1% and 28.9% of abstracts, respectively. From the small proportion of randomized, controlled trials (23 or 29.1%) that identified a single primary end point results differed in 9 of 23 (39.1%). CONCLUSIONS Most abstracts fail to provide the necessary information to assess methodological quality. Organizers of urological meetings should consider implementing a more structured abstract format that requires authors to provide the necessary study details, thereby allowing urologists to critically appraise study validity.

Evidence-based urology in practice: how to use PubMed effectively

You are the on-call urologist consulted by the emergency room (ER) for acute flank pain in a 44-year-old woman. The patient denies any previous history of stones and is otherwise healthy. Non-contrast enhanced abdominal and pelvic CT shows a 4-mm calculus at the vesico-ureteric junction. Urine analysis shows no infection and traces of blood. Her pain and nausea are controlled in the ER. You recommend an α -blocker to promote stone passage, which is met with some scepticism by the ER physician, who queries you whether there is more than anecdotal evidence for this recommendation, which prompts you to search the medical literature.

Evidence-based medicine training in residency: a survey of urology programme directors

To investigate whether and in what format urology residents receive formal training in evidence‐based clinical practice.

Levels of evidence ratings in the urological literature: an assessment of interobserver agreement

To determine to what extent urologists with no specific training agree upon level of evidence (LoE) ratings of studies published in the urological literature, as LoE are commonly referenced as a measure of evidence quality.