Organ Localization in PET/CT Images using Hierarchical Conditional Faster R-CNN Method

Abstract

Localization of multiple organs from whole-body PET/CT image is a prerequisite step of automated PET/CT image analysis. However, due to the large image size and the wide body range, multi-organ detection from whole-body image suffers from high computation burden and low searching accuracy. In this paper, we propose an efficient and accurate multi-organ localization method by combining the conditional Gaussian model (CGM) and the Faster R-CNN object detection neural network. Different organs are detected subsequentially following their anatomical hierarchy. The CGM is used model the inter-organ positional correlations between different hierarchical levels. Once an organ at a coarser level is localized, its children organs at the finer level are roughly predicted using the CGM, and then the Faster R-CNN method is used to refine the locations within the predicted regions. To alleviate the computation burden, we project the volumetric image into orthogonal 2D images and perform organ localization in the projected images. The 2D localization results are then back-projected to 3D to obtain the 3D organ region. Experimental results show that our method achieved sub-centimetre detection accuracy of multiple torso organs within 10min. The accuracy and robustness of our method are both superior to the traditional Faster R-CNN method which does not use the hierarchical searching strategy. We applied our method to both the CT and PET images and found that although PET has worse pixel resolution than CT, it yielded better localization accuracy for the organs with high PET tracer contrast. This result means that intensity contrast is more important than image resolution for improving the localization accuracy.