Ultrasound multi-needle detection using deep attention U-Net with TV regularizations

Abstract

A deep-learning model based on the U-Net architecture was developed to segment multiple needles in the 3D transrectal ultrasound (TRUS) images. Attention gates were adopted in our model to improve the prediction on the small needle points. Furthermore, the spatial continuity of needles was encoded into our model with total variation (TV) regularization. The combined network was trained on 3D TRUS patches with the deep supervision strategy, where the binary needle annotation images from simulation CTs were provided as ground truth. The trained network was then used to localize and segment the HDR needles for a new patient TRUS images during high-dose-rate (HDR) prostate brachytherapy. The needle shaft and tip errors against CT-based ground truth were used to evaluate other methods and other methods as comparison. Our method detected 96% needles of 339 needles from 23 HDR prostate brachytherapy patients with 0.29±0.24 mm at shaft error and 0.442±0.831 mm at tip error. For shaft localization, our method resulted in 96% localizations with less than 0.8 mm error (needle diameter is 1.67 mm), while for tip localization, our method resulted in 75% needles with 0 mm error and 21% needles with 2 mm error (TRUS image slice thickness is 2 mm). No significant difference was observed (p = 0.83) on tip localization between our results with the ground truth. Compared with U-Net and deep supervised attention U-Net, the proposed method delivers a significant improvement on both shaft error and tip error. Besides, to our best knowledge, this is the first attempt on multi-needle localization in the prostate brachytherapy. The 3D rendering of the needles could help clinicians to evaluate the needle placements. It paves the way for the development of real-time radiation plan dose assessment tools that can further elevate the quality and outcome of prostate HDR brachytherapy.