Roberta Scherer

Systematic Reviews: Identifying relevant studies for systematic reviews

Abstract Objective: To examine the sensitivity and precision of Medline searching for randomised clinical trials. Design - Comparison of results of Medline searches to a “gold standard” of known randomised clinical trials in ophthalmology published in 1988; systematic review (meta-analysis) of results of similar, but separate, studies from many fields of medicine. Populations: Randomised clinical trials published in in 1988 in journals indexed in Medline, and those not indexed in Medline and identified by hand search, comprised the gold standard. Gold standards for the other studies combined in the meta-analysis were based on: randomised clinical trials published in any journal, whether indexed in Medline or not; those published in any journal indexed in Medline; or those published in a selected group of journals indexed in Medline. Main outcome measure - Sensitivity (proportion of the total number of known randomised clinical trails identified by the search) and precision (proportion of publications retrieved by Medline that were actually randomised clinical trials) were calculated for each study and combined to obtain weighted means. Searches producing the “best” sensitivity were used for sensitivity and precision estimates when multiple searches were performed. Results: The sensitivity of searching for ophthalmology randomised clinical trials published in 1988 was 82%, when the gold standard was for any journal, 87% for any journal indexed in Medline, and 88% for selected journals indexed in Medline. Weighted means for sensitivity across all studies were 51%, 77%, and 63%, respectively. The weighted mean for precision was 8% (median 32.5%). Most searchers seemed not to use freetext subject terms and truncation of those terms. Conclusion - Although the indexing terms available for searching Medline for randomised clinical trials have improved, sensitivity still remains unsatisfactory. A mechanism is needed to “register” known trials, preferably by retrospective tagging of Medline entries, and incorporating trials published before 1966 and in journals not indexed by Medline into the system.

The fellow eye in NAION: Report from the Ischemic Optic neuropathy decompression Trial Follow-up study

PURPOSEnTo examine the prevalence and incidence of second eye nonarteritic anterior ischemic optic neuropathy (NAION) and associated patient characteristics in patients enrolled in the Ischemic Optic Neuropathy Decompression Trial (IONDT) Follow-up Study.nnnDESIGNnRandomized clinical trial with observational cohort.nnnMETHODSnPatients randomized to optic nerve sheath decompression surgery or careful follow-up had a diagnosis of acute unilateral NAION, visual acuity between 20/64 and light perception, and were aged 50 years or older. Eligible patients who declined randomization or whose visual acuity was better than 20/64 were not randomized but followed as part of an observational cohort. Follow-up examinations took place at 3, 6, 12, 18, and 24 months and annually thereafter.nnnRESULTSnFour hundred eighteen patients were enrolled; 258 randomized and 160 observed. Previous NAION or other optic neuropathy was present in the fellow eye of 21.1% (88/418) of patients at baseline. Four patients developed optic neuropathy in the fellow eye at follow up that could not be conclusively diagnosed as NAION. New NAION in the fellow eye occurred in 14.7% (48/326) of patients at risk during a median follow up of 5.1 years. Randomized patients experienced a higher incidence (35/201; 17.4%) than nonrandomized patients (13/125; 10.4%). A history of diabetes and baseline visual acuity of 20/200 or worse in the study eye, but not age, sex, aspirin use, or smoking were significantly associated with new NAION in the fellow eye. Final fellow eye visual acuity was significantly worse in those patients with new fellow eye NAION whose baseline study eye visual acuity was 20/200 or worse.nnnCONCLUSIONSnFollow-up data from the IONDT cohort provide evidence that the incidence of fellow eye NAION is lower than expected: new NAION was diagnosed in 14.7% of IONDT patients over approximately 5 years. Increased incidence is associated with poor baseline visual acuity in the study eye and diabetes, but not age, sex, smoking history, or aspirin use.

Surgical treatments outcomes project for dysfunctional uterine bleeding (STOP-DUB): design and methods

The Surgical Treatments Outcomes Project for Dysfunctional Uterine Bleeding (STOP-DUB) was a multicenter, randomized clinical trial that assessed the efficacy and effectiveness of hysterectomy versus endometrial ablation (EA) for dysfunctional uterine bleeding (DUB) in women for whom medical management has not provided relief. Resource centers included a coordinating center, a chairs office, the American College of Obstetricians and Gynecologists, the Agency for Healthcare Research and Quality Project Office and 33 clinical centers in the United States and Canada. STOP-DUB enrolled: (1) eligible patients for whom medical treatment had not been successful and who were randomized to either hysterectomy or EA and (2) an observational cohort of patients who were provisionally ineligible or who were eligible but did not wish to be randomized. Enrollment began in October 1997 and ended in June 2001. The primary outcome addressed by the randomized trial was the impact of surgery on bleeding, pain, fatigue, and the major problem (symptom) that led the woman to seek treatment for her condition, measured 1 year following surgery. Additional outcomes included the impact of surgery at time points after 1 year; changes in quality of life, activity limitation, sexual functioning, and urinary incontinence; surgical complications; additional surgery; and resource utilization. The costs and the relative cost-effectiveness of the two surgeries were calculated. The main scientific objective for the observational study was to examine changes over time in terms of treatment selected, DUB-related symptoms, and quality of life.

Surgical quality assurance in the Ischemic Optic Neuropathy Decompression Trial (IONDT).

The purpose of this article is to report the methods and results of the surgical quality assurance program associated with the Ischemic Optic Neuropathy Decompression Trial (IONDT). A surgical quality assurance committee developed and implemented a quality assurance program for a randomized clinical trial requiring surgical intervention. A surgical technique questionnaire was administered at two times during the study course, and maintenance of surgeon certification required submission and approval of a masked videotape of an optic nerve sheath decompression surgery by each study surgeon. Surgical quality was assessed through completion of surgical report forms and standardized, masked review of operative notes. Rates of compliance and intra- and interreviewer agreement were assessed for each aspect of the program. Twenty-five of 32 surgeons (81%) successfully completed and maintained certification. Item agreement varied from 21-92% among reviewers of satisfactory videotapes and 22-89% on unsatisfactory videotapes. Intrarater agreement for videotape acceptability was 11 of 13 (85%), and for specific surgical steps, 147 of 182 (81%). Operative notes were submitted for 123 of 125 (98%) patients receiving surgery. Interrater agreement on individual items ranged from 73-100%. Classification of individual items was identical on first and second review for 1285 of 1344 (95.6%) items. Overall agreement for individual reviewers was 93.8-97.8%. We conclude that use of a small peer review committee, which developed and oversaw a quality assurance program, allowed for consistent certification and monitoring of surgical performance. This in turn increased the credibility of the IONDT results, which demonstrated no difference in outcome between operated and unoperated groups of patients.

Participation in the Ischemic Optic Neuropathy Decompression Trial: sex, race, and age

BACKGROUNDnThe Ischemic Optic Neuropathy Decompression Trial (IONDT) is a randomized, single masked, multicenter trial designed to assess the safety and efficacy of optic nerve decompression surgery compared with careful follow-up in patients 50 years or older with non-arteritic anterior ischemic neuropathy (NAION).nnnOBJECTIVESnTo examine and evaluate the sex, race, and age distributions of the screened, and subsequently enrolled, IONDT population, especially the proportions of female, minority, and elderly patients, and demographic characteristics of clinical center investigators, the geographical location of IONDT Clinical Centers, and the referral patterns of local physicians.nnnSETTINGnTwenty-five U.S. clinical centers.nnnPARTICIPANTSnThere were 1,681 referrals to the Clinical Centers; an Eligibility Screening Form providing demographic information was completed for 1,152.nnnFINDINGSnForty-three percent (495/1,152) of screened cases were women. Seven percent (85/1,152) were minorities: 33 African-Americans, 34 Hispanics, 17 Asians, and 1 Native-American. The average age was 66 +/- 10 years with a range of 22-92 years of age. Of the 1,152 referred patients, 305 were eligible for randomization; 258 (85%) of these agreed to participate in the trial. The demographic makeup of the randomized IONDT patients was similar to that of the screened population.nnnCONCLUSIONSnWomen and the elderly are well represented in the IONDT. Because the number of participating minorities was low, we cannot reliably assess their level of participation. Both a low incidence of NAION in minorities and a low referral rate of minorities to clinical trials are plausible explanations for our findings.