Machine Outputs Must Be Checked

Abstract

We read with interest the multicenter manuscript of Psychogios et al. in which they reported on the comparison of infarct core and tissue at risk maps generated by four different vendors as well as visual Cerebral Blood Volume—Alberta Stroke Program Early CT Score (CBVASPECTS) and visually assessed collateral scores [1]. They related the maps of 182 patients undergoing mechanical thrombectomy (MT) and receiving a TICI 2b, 2c or III reperfusion to the clinical outcome assessed with the modified Rankin score (mRS) and the functional disability defined as mRS >2. They calculated mean differences between RAPID (iSchemaView Inc, Menlo Parc, CA, USA) and other software packages and illustrated them with Bland-Altman plots. They concluded that the infarct core defined by the RAPID software correlates best with the clinical outcome whilst VEOcore (VEObrain GmbH, Freiburg, Germany) and syngo.via (Siemens Healthineers AG, Erlangen, Germany) overestimate the infarct core and Olea (OLEA medical Inc., La Ciotat, France) underestimates it [1]. The message is clear but can we trust it? In the manuscript the authors clearly state that out of 215 cases 33 cases have been excluded from the final analysis due to “... technical failure of at least 1 perfusion software”; however, if we take a look at the Bland-Altman plot of RAPID-VEOcore (only available in the Supplemental Material) there is a striking outlier in the infarct core volume difference of around –2131mL (which is distinctly larger than an entire brain). This outlier leads to a massive bias in the statistics: it can be estimated that without the outlier the true mean difference between RAPID and VEOcore is in a very good agreement range of –1.5mL instead of the –13.4mL reported in the manuscript.