Noninferiority Margins in Trials of Thrombectomy Devices for Acute Ischemic Stroke: Is the Bar Being Set Too Low?

Abstract

Background and Purpose- Novel endovascular thrombectomy (EVT) devices for acute ischemic stroke are often cleared by regulatory agencies on the basis of noninferiority trials. The relation between the noninferiority margins used in trials and the minimal clinically important differences (MCIDs) determined by experts have not been systematically investigated. Methods- Systematic searches were performed to identify (1) all noninferiority design or noninferiority-presented stroke-EVT trials for acute ischemic stroke, (2) all studies determining the MCIDs for the same outcomes, and (3) all noninferiority coronary revascularization trials. Stroke-EVT trial results were reanalyzed using the broad noninferiority margins originally used and narrower noninferiority margins derived from formal MCID studies. Results- We identified 7 noninferiority-designed or noninferiority-interpreted stroke-EVT controlled trials, enrolling 1766 patients, variously comparing coil retrievers, first- and second-generation stent retrievers, and aspiration devices. In 6 trials, the primary outcome was achievement of reperfusion, using noninferiority margins of 15% (3 trials), 10% (2 trials), and 8% (1 trial). In contrast, a stroke expert survey identified the MCID for reperfusion as 3.1% to 5%, and cardiac trials used noninferiority margins of 3.5% to 4.4%. In one stroke-EVT trial, the primary outcome was functional independence, using a noninferiority margin of 15%. However, 2 stroke expert survey studies identified MCIDs for functional independence as having lower values, 5% and 1% to 1.5%. For both reperfusion and functional independence outcomes, all 7 trials demonstrated noninferiority with the broadest noninferiority margin, but only 4 and 3 trials demonstrated noninferiority with actual expert-derived margins for reperfusion and functional independence, respectively. Conclusions- Noninferiority margins used in EVT device trials have regularly exceeded the MCIDs determined by stroke experts, as well as margins used for cardiac devices. New approaches, such as the use of reasonably adequate performance margins, rather than noninferiority margins, are needed to optimize stroke-EVT trial design integrity and trial performance feasibility.