Evaluation of an AI-based, automatic coronary artery calcium scoring software

Abstract

Objectives To evaluate an artificial intelligence (AI)–based, automatic coronary artery calcium (CAC) scoring software, using a semi-automatic software as a reference. Methods This observational study included 315 consecutive, non-contrast-enhanced calcium scoring computed tomography (CSCT) scans. A semi-automatic and an automatic software obtained the Agatston score (AS), the volume score (VS), the mass score (MS), and the number of calcified coronary lesions. Semi-automatic and automatic analysis time were registered, including a manual double-check of the automatic results. Statistical analyses were Spearman’s rank correlation coefficient (⍴), intra-class correlation (ICC), Bland Altman plots, weighted kappa analysis (κ), and Wilcoxon signed-rank test. Results The correlation and agreement for the AS, VS, and MS were ⍴ = 0.935, 0.932, 0.934 ( p < 0.001), and ICC = 0.996, 0.996, 0.991, respectively ( p < 0.001). The correlation and agreement for the number of calcified lesions were ⍴ = 0.903 and ICC = 0.977 ( p < 0.001), respectively. The Bland Altman mean difference and 1.96 SD upper and lower limits of agreements for the AS, VS, and MS were − 8.2 (− 115.1 to 98.2), − 7.4 (− 93.9 to 79.1), and − 3.8 (− 33.6 to 25.9), respectively. Agreement in risk category assignment was 89.5% and κ = 0.919 ( p < 0.001). The median time for the semi-automatic and automatic method was 59 s (IQR 35–100) and 36 s (IQR 29–49), respectively ( p < 0.001). Conclusions There was an excellent correlation and agreement between the automatic software and the semi-automatic software for three CAC scores and the number of calcified lesions. Risk category classification was accurate but showing an overestimation bias tendency. Also, the automatic method was less time-demanding. Key Points • Coronary artery calcium (CAC) scoring is an excellent candidate for artificial intelligence (AI) development in a clinical setting. • An AI-based, automatic software obtained CAC scores with excellent correlation and agreement compared with a conventional method but was less time-consuming.