Daniel L. Kent

The clinical efficacy of magnetic resonance imaging in neuroimaging

[Note that sections in this review are numbered so that they can be identified with cross-references as supporting evidence in the article Magnetic Resonance Imaging of the Brain and Spine: A Revised Statement, published in the Position Paper section of this issue; see pages 872-875.The Editor] 1. Magnetic resonance imaging (MRI) has revolutionized neuroimaging. Many internists now are familiar with the technology. However, strong support for a greatly expanded role of MRI for diagnosis and management of neurologic diseases is lacking because the evidence for clinical efficacy in the published literature is incomplete [1-3]. Because many more studies have appeared in the past 6 years, the American College of Physicians decided to update its 1987 report on the clinical efficacy of MRI in neuroimaging [4]. This review provides the basis for the updated assessment and accompanying suggestions for use of MRI in various clinical situations. 2. Methods 2.1 Titles and abstracts of 3125 citations from the peer-reviewed medical literature from 1987 through November 1993 were found using MEDLINE and ancillary searches. After screening out reviews, technical reports, case reports, and other publications describing fewer than 30 original case-patients, we identified 285 papers for further review. Two investigators independently read and classified each article according to the disease studied, the applicable level of clinical efficacy, and the methodologic quality of the study design. This three-part classification is summarized in Table 1 and has been described in detail previously [3]. Discrepant ratings were resolved by consensus. Table 1. Three Dimensions to Classification of Studies of a Diagnostic Test* 2.2 Because the imaging capabilities of MRI have been widely discussed in case series, we focused on the 156 articles that addressed diagnostic accuracy or diagnostic impact, therapeutic impact, or patient outcomes. For diagnostic accuracy and effect, methodologic quality was rated as A if the study had more than 35 patients with and more than 35 without the pathologic abnormality in question, drawn from a clinically relevant sample whose clinical symptoms were completely described, whose diagnoses were defined by an appropriate reference standard, and whose magnetic resonance images were technically of high quality and were evaluated independently of the reference diagnosis. Studies were rated D if they had no credible reference standard for diagnosis, or if the test result and determination of final diagnosis were not independent, or if the sample size was smaller than 35 for patients with and without the disease, or if the source of the patient cohort could not be determined or was obviously influenced by the test results (work-up bias). 2.3 Studies of therapeutic impact or changes in patient outcomes were evaluated for methodologic quality according to a similar A through D scale, adapted from Sackett [5]. For example, an A-rated study required a randomized, controlled trial using a clinically relevant, well-described sample, with adequate power to detect differences in well-characterized outcomes. The rationale and details for these ratings have been described [3, 6]. 2.4 Among 143 articles reporting on diagnostic accuracy or impact, 1 was rated A, 14 were rated B, 14 were rated C, and 113 were rated D for methodologic quality. Deficiencies included small numbers of patients, failure to describe the source of patients and their spectrum of disease, failure to maintain independence between interpretation of the MRI scan and an appropriate reference standard, or obsolescence of the magnetic resonance technology. Estimates of sensitivity and specificity were derived from the A, B, and C studies. Results from receiver-operating characteristic (ROC) curves are discussed in the text when available. Among 21 articles reporting on the therapeutic impact of MRI, none were rated A, 1 randomized trial was rated B, and 2 comparisons with computed tomography (CT) were rated C (Table 2). Five studies comparing treatment plans before and after MRI, but without comparison to an alternative test, were rated D for methodologic quality. Retrospective surveys of anecdotal experiences were not rated. No studies provided data on long-term patient outcomes. Table 2. Studies Indicating Therapeutic Impact or Changes in Patient Outcomes* 2.5 Suggestions for practice are given in the companion report from the American College of Physicians [4]. A strong strength-of-evidence rating would have been given to a suggestion supported by randomized, controlled studies of patient outcomes and by conclusive evidence about diagnostic accuracy. The one suggestion given a moderate rating was supported by studies that used comparison groups to show improved therapeutic choices and a definite impact on diagnoses. Most of the suggestions given weak ratings were supported by fair or good studies of diagnostic accuracy but had no supporting studies of therapeutic impact or patient outcomes. The suggestion given an equivocal rating was based only on prevalent expert opinion and several reports describing a predominance of clinical practice patterns. 3. Safety, Technical Issues, and Costs 3.1 Safety evaluations indicate that noncontrast MRI is safe for most patients, including pregnant women [7-9]. Pacemakers, many cerebral aneurysm clips, intraocular metal, and cochlear implants are definite contraindications to MRI. Other vascular clips, vena cava filters, and most metallic implants require individual patient consultation [10]. Incomplete or technically inadequate studies occur because of patient or equipment problems in 3% to 10% of attempted scans [11-13]. By comparison, safety concerns about CT are focused on iodinated contrast dye reactions. Some acute reaction occurs in 3% to 12% of patients, with severe reactions in 0.02% to 0.2% of patients [14, 15]. Newer nonionic contrast agents are associated with lower reaction risks. 3.2 Paramagnetic contrast agents improve confidence in diagnosis in 45% to 75% of scans and actually change the apparent diagnosis in up to 30% of patients [12, 16-22]. None of these studies could report diagnostic accuracy statistics because the comparisons were done without any independent reference standards. The larger multicenter evaluations of contrast agents probably contain case-selection biases that skew statistical analysis [21]. Paramagnetic contrast agents cross the placenta and typically are contraindicated during pregnancy [9], but all are otherwise safe [23, 24]. Minor adverse reactions to the agents occur in 2% to 3% of patients, and anaphylaxis occurs in about 1 in 100 000 patients [25]. 3.3 Diagnostic specificity for MRI has been studied only infrequently [3]. Although specificity has been used to mean that an imaging test can determine specific tissue composition, specificity as used in this paper is one minus the false-positive rate [26, 27]. Classically, a false-positive result is a positive test result when no disease is present. However, MRI often shows incidental anatomic abnormalities that are real and technically are not false-positive images (examples in Table 3). If any incidental abnormality of the cerebral white matter is called a false-positive result, the apparent diagnostic specificity can be low (0.28 to 0.58 [28-31]). The specificity is much higher if a more stringent test interpretation, such as one requiring multiple abnormalities to indicate disease, is used (0.80 to 0.90 [28]). The reliability of readings about white matter abnormalities is low [32]. No studies have reported investigations into the incidence of therapeutic misadventures caused by overinterpretation of white matter findings or by other asymptomatic abnormalities. Table 3. Diagnostic Accuracy of Magnetic Resonance Imaging for Various Conditions 3.4 Costs of MRI historically have been high compared with competing technologies. Facility and professional charges for MRI are 30% to 100% higher than for CT (Table 4). Some payers reimburse MRI only at a rate equivalent to CT (for example, Medi-Cal). If MRI obviates further tests but CT does not, using MRI as the initial test may be less expensive than using CT. Similarly, if MRI replaces an invasive test associated with inpatient facility charges (such as a myelogram, cisternogram, or cerebral arteriogram), then the MRI cost is less than the total cost of the alternative. New fast-imaging sequences, contrast agents, and studies focused only on high-yield views for particular diagnoses can shorten examination times and improve efficiency at imaging centers. Table 4. Recent Charges for Magnetic Resonance Imaging and Competing Technologies* 4. Stroke and Transient Ischemic Attacks 4.1 The clinical diagnosis of the cause of a stroke is erroneous in 10% to 20% of patients [33], so most patients have imaging. Because spatial resolution of MRI has improved, smaller infarctions have been seen, often on the first day after onset of illness [34-36]. Among several studies with small sample sizes [13, 34, 37-39], the sensitivity of MRI for the infarction ranged from 0.86 to 0.95, whereas the sensitivity of CT ranged from 0.50 to 0.68. One study of posterior circulation infarctions also reported a specificity of 0.88 [38]. 4.2 No studies rated as using grade C or better methods have compared MRI with either noncontrast or contrast-enhanced CT. Further, the impact of MRI on final diagnosis or therapeutic choices for patients with stroke has been reported only once [40]. In that study, MRI reportedly led to a change in diagnosis of the cause of stroke for 16% of patients and to changes in anticoagulation management for 19%. Treatment changes were not tabulated or evaluated for appropriateness; biasing of case selection through the influence of referral or nonreferral for imaging by treating clinicians also was not considered. No studies have investigated changes in patient outcomes attributable to

DISEASE, LEVEL OF IMPACT, AND QUALITY OF RESEARCH METHODS : THREE DIMENSIONS OF CLINICAL EFFICACY ASSESSMENT APPLIED TO MAGNETIC RESONANCE IMAGING

Assessment of the clinical efficacy of diagnostic imaging technologies frequently involves reviews of published research. Reports may be classified in three dimensions; by disease, by type of assessment, and by the quality of research methods. The disease dimension describes the condition or conditions shown by an imaging technique. The assessment dimension spans five levels: technical capacity, diagnostic accuracy, diagnostic impacts, therapeutic impacts, and patient outcome impacts. The methods quality dimension can be expressed as four levels: excellent, good, fair or poor. An important interaction exists: the level of efficacy addressed by a research project dictates which methodologic procedures are important. For example, randomization is important only when a research report addresses the levels of therapeutic and patient outcome impacts. The authors suggest that classification of studies according to the three preceding dimensions maps the breadth (across diseases), depth (across levels of clinical efficacy), and quality of the assessment of complex imaging technologies. Such a map should help participants in technology assessment define the progress they have made. The classification strategy as applied to the clinical efficacy assessment of magnetic resonance imaging (MRI) for neuroradiology is illustrated.

Magnetic Resonance Imaging of the Brain and Spine: Is Clinical Efficacy Established after the First Decade?

STUDY OBJECTIVE Evaluation of demonstrated clinical efficacy of magnetic resonance (MR) imaging in the central nervous system. DESIGN Information synthesis of studies before January 1987. SETTING Reports were classified by the level of clinical efficacy studied (technical capacity, diagnostic impacts, and therapeutic or patient outcome impacts) and were judged by the validity of their methods, especially avoidance of diagnosis review, test review, and work-up biases. MAIN RESULTS Magnetic resonance imaging probably is superior to computed tomography for detection and characterization of posterior fossa lesions and spinal cord myelopathies, imaging in multiple sclerosis, detecting lesions in patients with refractory partial seizures, and detailed display for guiding complex therapy, as for brain tumors. In other diseases, the efficacy of MR imaging is similar to that of computed tomography (cerebrovascular, radiculopathy, and infection). Magnetic resonance imaging is less invasive than intrathecal or intravenous contrast-enhanced computed tomography and costs 20% to 300% more than computed tomography, although avoidance of hospital stays may offset some costs. Generally, the quality of MR images probably exceeds that of computed tomographic (CT) scans. However, published evidence does not show that the clinical efficacy of MR imaging is generally superior to that of existing imaging modalities such as computed tomography. Only six studies avoided major methodologic biases, and lower true-positive rates for MR imaging were reported in these studies than reported in multiply biased studies. Few studies of the potential of MR imaging for false-positive diagnosis have been done. CONCLUSIONS Use of standards for quality of evidence leads to more conservative conclusions than those of reports describing the clinical potential of MR imaging. Some applications of MR imaging were confirmed by rigorous studies, whereas others were not well supported by reports free of methodologic biases. If the diagnostic alternative is invasive (for example, myelography and cisternography), MR imaging is preferred, but adequate diagnosis for many conditions (head trauma, simple stroke, and dementia) may not require the detail of an MR imaging study. In general, more rigorous clinical research studies are needed for new technologies such as MR imaging. Because the field of MR imaging is changing, review of its clinical efficacy will need to be revised frequently.

Toward a Peer Review Process for Medical Decision Analysis Models

This paper presents a framework for a peer review process for medical decision analysis models. This framework is based on the collective experience of the members of the Inter-PORT Decision Modeling work group, a team of decision analysis experts comprising members from each of the Patient Outcomes Research Teams (PORTs), sponsored by the Agency for Health Care Policy and Research. Important general principles of correct model structure include choice of model type, perspective of the analysis, choice of utility scheme and identification of strategies that should be included and events that should be modeled. In addition, a set of rules for correct decision model structure may help to identify common errors. Although not necessarily exhaustive, this scheme provides an approach by which a reviewer may judge the adequacy of a decision model presented for publication or as the basis for a health policy.

A computerized intervention to improve timing of outpatient follow-up - A multicenter randomized trial in patients treated with warfarin

Objective: To evaluate a computerized scheduling model that employs nonlinear optimization to recommend optimal follow-up intervals for patients taking warfarin.Design: Randomized trial.Setting: 5 anticoagulation clinics.Patients/participants: 620 patients expected to receive warfarin for ≥6 weeks.Interventions: Computer-generated recommendations for scheduling the next visit were presented to or withheld from practitioners.Measurements and main results: The main outcome measures were the follow-up interval scheduled by the provider, the interval at which the patient actually returned to clinic, and the quality of anticoagulation control (computed as the absolute difference between the measured and target prothrombin times [PTRs] or international normalized ratios [INRs]). Follow-up intervals scheduled for the patients whose practitioners received computer-generated recommendations were significantly longer than those for control patients (mean, 4.4 vs 3.5 weeks, p<0.001), despite the fact that the practitioners modified the suggested return interval by >1 week on 40% of the visits. The interval at which the intervention group actually returned to clinic was also longer (mean, 4.4 vs 4.1 weeks, p<0.05), even though the control patients tended to return at longer intervals than were scheduled by their practitioners. Control of anticoagulation was nearly the same among experimental and control patients. Life-threatening complications occurred in the care of three experimental patients and one control patient, while other serious complications occurred in the care of 16 experimental patients and 17 control patients.Conclusions: Recommendations based on nonlinear optimization prompted clinicians to schedule less frequent follow-up for patients taking warfarin, with no deterioration in anticoagulation control. This approach to scheduling can potentially reduce utilization while maintaining quality of care for patients who require long-term monitoring.

Designing studies of diagnostic tests for low back pain or radiculopathy

New diagnostic tests for the evaluatin of patients with low back pain are constantly emerging, but are often not completely evaluated before they become used. Many published studies have a number of biases that tend to exaggerate the estimated accuracy of a diagnostic test. Several key study design features should be considered in such studies: independent comparison of the diagnostic test results with an appropriate “gold standard”; blinded assessment of the new test and the gold standard or competing tests; the reproducibility of interpretation of the test being examined; and the sensitivity and specificity of the test for the final gold standard diagnosis. In addition, evaluations of test accuracy should include patients with a wide spectrum of illness from mild to severe, and the study setting and patient characteristics should be described in detail. Finally, the contribution of a diagnostic test to the overall validity of a full group or sequence of tests should be considered, and, ideally, the effect of the test on actual patient outcomes should be determined.

Diagnostic Accuracy and Clinical Utility of Thermography for Lumbar Radiculopathy: A Meta-analysis

The role of thermography for diagnosing lumbar radiculopathy was evaluated by literature review and meta-analysis. From 81 relevant citations, 28 studies could be analyzed for diagnostic-accuracy data (sensitivity and specificity) and method. Diagnostic-accuracy data varied significantly between studies; therefore meaningful pooled summary statistics could not be reported. Twenty-seven studies had major methodologic flaws including biased test interpretations, faulty cohort assembly, poor clinical descriptions, and small sample size. The only study of reasonably high quality found no discriminant value for liquid-crystal thermography. The role of thermography remains unclear. Rigorous clinical research is required to establish its diagnostic accuracy and clinical utility. Thermography cannot be recommended currently for routine clinical use in evaluating low-back pain.

VARIATION IN THE RATE OF CERVICAL SPINE SURGERY IN WASHINGTON STATE

Lack of consensus about the treatment of low back pain is reflected by wide regional variations in lumbar spine surgery rates. Neck pain may be as common as low back pain, but there has been no similar evaluation of regional variation for the surgical treatment of neck pain. This report examines the geographic variation and temporal trends in the rate of cervical spine surgery in Washington state from 1986 through 1989. Using diagnosis and procedure codes from the International Classification of Diseases (ICD-9 CM), the authors retrospectively identified cervical spine surgery cases from a statewide hospital discharge registry for Washington. After excluding cases associated with trauma, infection, or malignancy, 5,173 incident cervical spine surgery cases were analyzed. Cervical spine surgery was performed at approximately 25% the rate of lumbar spine surgery, and from 1986 to 1989, the age- and gender-adjusted rate increased 20%. Small area analysis demonstrated a sevenfold variation among counties in the rate of cervical spine surgery (P < 0.001), with variation of fourfold to 13-fold for specific surgical procedures. These data demonstrate that cervical spine surgery for neck pain is an increasingly common procedure with wide geographic variability. Rational treatment of neck pain requires further definition of indications for cervical spine surgery, preferably based on firm data concerning the outcomes of surgical and nonsurgical care.

A Model for Planning Optimal Follow-up for Outpatients on Warfarin Anticoagulation

Patients taking warfarin for long-term anticoagulation require frequent clinic visits to monitor the prothrombin time ratio (PTR), a measure of blood clotting. A dynamic stochastic model using nonlinear optimization was developed to select follow-up visit intervals that minimize the overall costs of patient care. Assuming that fluctuations in a patients PTR behave as a random diffusion process, future PTR fluctuations are unknown, except as revealed by past PTRs. To determine the incidence and costs of complications in relation to PTR, the authors reviewed the charts of 216 patients who had 719 patient-years of follow-up with 695 trivial, significant, life-threatening, or fatal complications. They modeled the relationship between costs of complications and deviation of the PTR from the therapeutic target as a fourth-order convex polynomial. The model is used to compute the interval to the next follow-up visit to minimize accumulated potential costs. Variables in the optimization are the cost of a mon itoring visit and the expected costs of complications. The latter are derived from the current PTR, the variability of the patients past PTR values, the number of past PTRs available, and the target PTR for the patient. No attempt is made to predict the level of the next PTR or suggest adjustments in the warfarin dose. Shorter follow-up is recommended for patients who have histories of large fluctuations in past PTRs and for patients with few prior PTR determinabons. As visits accumulate, the patients degree of variability can be estimated more accurately and visit intervals adjusted accordingly The scheduling method balances costs to the health care system of monitoring each patient against the expected costs of complications. This approach has the potential to reduce the number of monitoring visits necessary for safe management of anticoagulated patients with stable PTRs and to improve control among unstable patients. Key words: warfarin; anticoagulation, ambulatory care; follow-up studies; stochastic processes. (Med Decis Making 1992;12:132-141)

Efficient Scheduling of Cystoscopies in Monitoring for Recurrent Bladder Cancer

Proper timing for repeated evaluations is difficult to assess. The authors analyzed scheduling of cystoscopy to monitor patients for detection of recurrent bladder cancer assuming that 1) minimizing tumor detection delay helps prevent cancer morbidities; 2) only limited numbers of cystoscopies are available; 3) prediction of recurrence or progression to invasive cancer is uncertain; 4) future tumors recur according to a Poisson process. Assumptions 3 and 4 permit estimation of each patients recurrence rate. Thus, patients may be compared ac cording to their relative risks of future tumors. With these assumptions, nonlinear optimization theory was used to calculate monitoring schedules for a model practice. Given 5.4 available visits per week per 100 patients, cystoscopy was recommended in 9-11 weeks for high-risk patients and in 30-40 weeks for low-risk patients, depending on stages, grades, and numbers of previous tumors. In contrast, standard cystoscopy was recommended in 13, 26, or 52 weeks, depending only on time elapsed since last recurrence. The calculated schedule implied an average detection delay for potentially invasive tumors of eight weeks, while standard practice led to detection delays of 11 weeks (38% worse). These results suggest that inclusion of each patients tumor history in an optimization approach may improve follow-up care for patients who have superficial bladder cancers. This approach is being evaluated in a larger clinical setting.