Lies, damned lies, and TICI

Abstract

“Failure is success in progress,” is one of Albert Einstein’s quotes, emphasizing the importance of recognizing one’s own limitations. The devices we use in our daily neurointerventional practice are constantly undergoing changes and improvements, as well as the techniques and approaches we apply to treat a wide range of cerebrovascular pathologies. Yet, we continue to judge procedural success or failure based on the visual feedback— the approach that has remained fundamentally the same since the introduction of the digital subtraction angiography technique. The Thrombolysis in Cerebral Infarction (TICI) scoring system is the uniform currency we all use every day; it tells us when the thrombectomy procedure is completed or whether additional device passes are required, it is used by the regulatory agencies to evaluate device safety, and it allows us to compare trial results and determine if the next best thing truly achieves superior and faster reperfusion rates, as claimed by the authors. It is not surprising therefore that the TICI scale has quickly found its way in the social media arena, with endless posts announcing first pass TICI 3 results greatly surpassing the clinical trial data or with claims of new devices’ superiority . Dmytriw et al compared postintervention Twitter reported thrombectomy outcomes with the Highly Effective Reperfusion Evaluated in Multiple Endovascular Stroke (HERMES) individual patient metaanalysis and showed a striking (over 20%) difference in the reported rates of modified TICI (mTICI) score of 2c/3 (94% vs 71%, respectively; p<0.0001). Successful reperfusion, which is traditionally defined as mTICI 2b/3 recanalization, is an extremely powerful predictor of a stroke patient becoming functionally independent in the future. Given its importance, here we aimed to review the data on the reliability and reproducibility of this angiographic scale, and to discuss its limitations and possible implications in patient care. Interrater or interobserver reliability describes the accuracy of an individual’s observation of a phenomenon, which in our case will be the rater’s ability to accurately and consistently assign TICI ranges. It is a well described phenomenon that individual operators tend to overcall the degree of procedural success. Core laboratory assessments have thus become the gold standard in evaluating angiography in many interventional disciplines, including neurointerventions. How frequently this discrepancy occurs is elegantly illustrated in the MR CLEAN (Multicenter Randomized Clinical Trial of Endovascular Treatment for Acute Ischemic Stroke in the Netherlands) registry study where operator mTICI and core laboratory mTICI scores were compared. In data derived from 1130 patients with anterior circulation large vessel occlusion treated with thrombectomy, the proportion of concordance between operator and core laboratory mTICI score was 56% (95% CI 54% to 59%). Furthermore, in every 1 in 3 cases (33% of cases), mTICI was overestimated by operators (95% CI 31% to 36%). Similarly, in the ASTER (Contact Aspiration vs Stent Retriever for Successful Revascularization) trial, disagreements in mTICI scores between study sites and the core laboratory were substantial (29.7% and 25.4% of cases for intermediate and final scores, respectively). The TICI scale was originally designed for evaluating territory reperfusion in the middle cerebral artery, and is currently used for other arterial territories, including the posterior circulation. In a recent study of eight observers asked to evaluate TICI scores before and after thrombectomy of posterior circulation occlusions, Findler et al showed high variability of individual scores (k value reached 0.277) corresponding to a fair agreement among the raters. The authors specifically noted low interrater reliability of attributes such as occlusion severity, recanalization status, endangered territories, and the existence of basilar atherosclerotic disease, questioning the utility of TICI in this population of patients. What can we learn from these studies? Relying entirely on core laboratory adjudication of individual studies would help increase the accuracy of angiographic data, but it is not practical. One solution is to have an independent operator from the same institution review the results independently, rather than relying on records of dictated reports. However, neither core laboratory nor individual operator adjudication would apply to the most critical scenario, that is, when the treating neurointerventionist is in the angiography suite performing the thrombectomy procedure and needs to decide in a timely manner whether additional device passes or therapeutic interventions are needed. Individual experience, fatigue and stress, quality of image processing, and other potential biases likely all contribute to the operator’s ability to grade the TICI result accurately. Awareness of the limitations of TICI is a good start. One of the authors of this article recently performed a quality check by going back and reviewing angiographic outcomes of prior thrombectomy procedures, which demonstrated overestimation and underestimation of individual mTICI results by 20% and 8%, respectively, especially in cases requiring multiple passes and those occurring during offhours. This was an important reality check, highlighting the need for a more thorough evaluation of angiographic results in future cases. Table 1 lists potential solutions to improve the subjectivity of angiographic assessment of thrombectomy. Ongoing research is evaluating whether alternative angiographic scales, such as the expanded TICI (eTICI) score, could further improve the operator’s accuracy of determining their procedural success. Finally, technological advances, such as incorporating real time automated TICI software into stroke thrombectomy procedures, may serve as an additional safeguard in helping physicians recognize hyperperfused areas. 9 All of these are works in progress and creating awareness of this matter is the first step in the right direction. More studies similar to the MR CLEAN registry analysis are needed to confirm how prevalent the issue of TICI overestimation is depending on stroke severity, location of occlusion, neurointerventional team set up, and operator experience. As more data become available, establishing a core laboratory correction factor could be considered when comparing and interpreting the results among individual operators, institution data, or retrospective Neurosurgery, University of South Florida, Tampa, Florida, USA NeuroEndovascular Program, Massachusetts General Hospital, Boston, Massachusetts, USA Department of Neurosurgery, Stony Brook University, Stony Brook, New York, USA Neurology, University of Pittsburgh, Pittsburgh, Pennsylvania, USA