Mingqing Chen

Automatic Liver Segmentation Using an Adversarial Image-to-Image Network

Automatic liver segmentation in 3D medical images is essential in many clinical applications, such as pathological diagnosis of hepatic diseases, surgical planning, and postoperative assessment. However, it is still a very challenging task due to the complex background, fuzzy boundary, and various appearance of liver. In this paper, we propose an automatic and efficient algorithm to segment liver from 3D CT volumes. A deep image-to-image network (DI2IN) is first deployed to generate the liver segmentation, employing a convolutional encoder-decoder architecture combined with multi-level feature concatenation and deep supervision. Then an adversarial network is utilized during training process to discriminate the output of DI2IN from ground truth, which further boosts the performance of DI2IN. The proposed method is trained on an annotated dataset of 1000 CT volumes with various different scanning protocols (e.g., contrast and non-contrast, various resolution and position) and large variations in populations (e.g., ages and pathology). Our approach outperforms the state-of-the-art solutions in terms of segmentation accuracy and computing efficiency.

Automatic extraction of 3d dynamic left ventricle model from 2d rotational angiocardiogram

In this paper, we propose an automatic method to directly extract 3D dynamic left ventricle (LV) model from sparse 2D rotational angiocardiogram (each cardiac phase contains only five projections). The extracted dynamic model provides quantitative cardiac function for analysis. The overlay of the model onto 2D real-time fluoroscopic images provides valuable visual guidance during cardiac intervention. Though containing severe cardiac motion artifacts, an ungated CT reconstruction is used in our approach to extract a rough static LV model. The initialized LV model is projected onto each 2D projection image. The silhouette of the projected mesh is deformed to match the boundary of LV blood pool. The deformation vectors of the silhouette are back-projected to 3D space and used as anchor points for thin plate spline (TPS) interpolation of other mesh points. The proposed method is validated on 12 synthesized datasets. The extracted 3D LV meshes match the ground truth quite well with a mean point-to-mesh error of 0.51 +/- 0.11 mm. The preliminary experiments on two real datasets (included a patient and a pig) show promising results too.

Automatic Vertebra Labeling in Large-Scale 3D CT Using Deep Image-to-Image Network with Message Passing and Sparsity Regularization

Automatic localization and labeling of vertebra in 3D medical images plays an important role in many clinical tasks, including pathological diagnosis, surgical planning and postoperative assessment. However, the unusual conditions of pathological cases, such as the abnormal spine curvature, bright visual imaging artifacts caused by metal implants, and the limited field of view, increase the difficulties of accurate localization. In this paper, we propose an automatic and fast algorithm to localize and label the vertebra centroids in 3D CT volumes. First, we deploy a deep image-to-image network (DI2IN) to initialize vertebra locations, employing the convolutional encoder-decoder architecture together with multi-level feature concatenation and deep supervision. Next, the centroid probability maps from DI2IN are iteratively evolved with the message passing schemes based on the mutual relation of vertebra centroids. Finally, the localization results are refined with sparsity regularization. The proposed method is evaluated on a public dataset of 302 spine CT volumes with various pathologies. Our method outperforms other state-of-the-art methods in terms of localization accuracy. The run time is around 3 seconds on average per case. To further boost the performance, we retrain the DI2IN on additional 1000+ 3D CT volumes from different patients. To the best of our knowledge, this is the first time more than 1000 3D CT volumes with expert annotation are adopted in experiments for the anatomic landmark detection tasks. Our experimental results show that training with such a large dataset significantly improves the performance and the overall identification rate, for the first time by our knowledge, reaches 90%.

Supervised Action Classifier: Approaching Landmark Detection as Image Partitioning

In medical imaging, landmarks have significant clinical and scientific importance. Clinical measurements, derived from the landmarks, are used for diagnosis, therapy planning and interventional guidance in many cases. Automatic algorithms have been studied to reduce the need for manual placement of landmarks. Traditional machine learning techniques provide reasonable results; however, they have limitation of either robustness or precision given complexities and variabilities of the medical images. Recently, deep learning technologies have been emerging to tackle the problems. Among them, a deep reinforcement learning approach (DRL) has shown to successfully detect landmark locations by implicitly learning the optimized path from a starting location; however, its learning process can only include subsets of the almost infinite paths across the image context, and may lead to major failures if not trained with adequate dataset variations. Here, we propose a new landmark detection approach inspired from DRL. Instead of learning limited action paths in an image in a greedy manner, we construct a global action map across the whole image, which divides the image into four action regions (left, right, up and bottom) depending on the relative location towards the target landmark. The action map guides how to move to reach the target landmark from any location of the input image. This effectively translates the landmark detection problem into an image partition problem which enables us to leverage a deep image-to-image network to train a supervised action classifier for detection of the landmarks. We discuss the experiment results of two ultrasound datasets (cardiac and obstetric) by applying the proposed algorithm. It shows consistent improvement over traditional machine learning based and deep learning based methods.

Automatic 3D Motion Estimation of Left Ventricle from C-arm Rotational Angiocardiography Using a Prior Motion Model and Learning Based Boundary Detector

Compared to pre-operative imaging modalities, it is more convenient to estimate the current cardiac physiological status from C-arm angiocardiography since C-arm is a widely used intra-operative imaging modality to guide many cardiac interventions. The 3D shape and motion of the left ventricle (LV) estimated from rotational angiocardiography provide important cardiac function measurements, e.g., ejection fraction and myocardium motion dyssynchrony. However, automatic estimation of the 3D LV motion is difficult since all anatomical structures overlap on the 2D X-ray projections and the nearby confounding strong image boundaries (e.g., pericardium) often cause ambiguities to LV endocardium boundary detection. In this paper, a new framework is proposed to overcome the aforementioned difficulties: (1) A new learning-based boundary detector is developed by training a boosting boundary classifier combined with the principal component analysis of a local image patch; (2) The prior LV motion model is learned from a set of dynamic cardiac computed tomography (CT) sequences to provide a good initial estimate of the 3D LV shape of different cardiac phases; (3) The 3D motion trajectory is learned for each mesh point; (4) All these components are integrated into a multi-surface graph optimization method to extract the globally coherent motion. The method is tested on seven patient scans, showing significant improvement on the ambiguous boundary cases with a detection accuracy of 2.87 +/- 1.00 mm on LV endocardium boundary delineation in the 2D projections.

Component-composition based heart isolation for 3D volume visualization of coronary arteries

Heart isolation (separating the heart from the neighboring tissues, e.g, lung, liver, and rib cage) is a prerequisite to generate a 3D volume visualization as an intuitive view for coronary disease diagnosis and treatment planning. Previously, we proposed a component-carving based heart isolation approach by removing unwanted background tissues (e.g, non-cardiac structures, left atrial appendage, and pulmonary veins/arteries) sequentially. However, the final mask usually has many small extra pieces due to the difficulty to model and segment all background tissues. In this paper, we propose a component-composition based approach, which starts from an empty mask and then adds in wanted components (coronary arteries, four chambers, and the aorta) segmented from the original image. The proposed method can generate a much cleaner mask than the component-carving approach since only the wanted structures are added and the segmentation of the heart chambers and the aorta is more robust and accurate due to their greater predictability.

Enhancement of organ of interest via background subtraction in cone beam rotational angiocardiogram

The real time X-ray angiography based on C-arm cone beam system is the workhorse imaging modality for interventional cardiac procedures. These images are two-dimensional (2D) projections of three dimensional (3D) objects along the X-ray direction. The organ of interest (OOI), such as left ventricle (LV) endocardium, in the projection image is superimposed with other anatomical structures and often has low contrast. In this study, a novel approach is proposed to isolate the OOI in projection images by subtracting with a background image, which is generated by numerical projection of 3D tomographic image with OOI masked out. Study based on one patient and one pig image is taken. About two to three-fold increase in the contrast-to-noise ratio (CNR) is achieved for LV endocardium, compared to an unprocessed image.