Guotai Wang

A homotopy-based sparse representation for fast and accurate shape prior modeling in liver surgical planning

Shape prior plays an important role in accurate and robust liver segmentation. However, liver shapes have complex variations and accurate modeling of liver shapes is challenging. Using large-scale training data can improve the accuracy but it limits the computational efficiency. In order to obtain accurate liver shape priors without sacrificing the efficiency when dealing with large-scale training data, we investigate effective and scalable shape prior modeling method that is more applicable in clinical liver surgical planning system. We employed the Sparse Shape Composition (SSC) to represent liver shapes by an optimized sparse combination of shapes in the repository, without any assumptions on parametric distributions of liver shapes. To leverage large-scale training data and improve the computational efficiency of SSC, we also introduced a homotopy-based method to quickly solve the L1-norm optimization problem in SSC. This method takes advantage of the sparsity of shape modeling, and solves the original optimization problem in SSC by continuously transforming it into a series of simplified problems whose solution is fast to compute. When new training shapes arrive gradually, the homotopy strategy updates the optimal solution on the fly and avoids re-computing it from scratch. Experiments showed that SSC had a high accuracy and efficiency in dealing with complex liver shape variations, excluding gross errors and preserving local details on the input liver shape. The homotopy-based SSC had a high computational efficiency, and its runtime increased very slowly when repositorys capacity and vertex number rose to a large degree. When repositorys capacity was 10,000, with 2000 vertices on each shape, homotopy method cost merely about 11.29 s to solve the optimization problem in SSC, nearly 2000 times faster than interior point method. The dice similarity coefficient (DSC), average symmetric surface distance (ASD), and maximum symmetric surface distance measurement was 94.31 ± 3.04%, 1.12 ± 0.69 mm and 3.65 ± 1.40 mm respectively.

On the Compactness, Efficiency, and Representation of 3D Convolutional Networks: Brain Parcellation as a Pretext Task

Deep convolutional neural networks are powerful tools for learning visual representations from images. However, designing efficient deep architectures to analyse volumetric medical images remains challenging. This work investigates efficient and flexible elements of modern convolutional networks such as dilated convolution and residual connection. With these essential building blocks, we propose a high-resolution, compact convolutional network for volumetric image segmentation. To illustrate its efficiency of learning 3D representation from large-scale image data, the proposed network is validated with the challenging task of parcellating 155 neuroanatomical structures from brain MR images. Our experiments show that the proposed network architecture compares favourably with state-of-the-art volumetric segmentation networks while being an order of magnitude more compact. We consider the brain parcellation task as a pretext task for volumetric image segmentation; our trained network potentially provides a good starting point for transfer learning. Additionally, we show the feasibility of voxel-level uncertainty estimation using a sampling approximation through dropout.

NiftyNet: a deep-learning platform for medical imaging

Highlights • An open-source platform is implemented based on TensorFlow APIs for deep learning in medical imaging domain.• A modular implementation of the typical medical imaging machine learning pipeline facilitates (1) warm starts with established pre-trained networks, (2) adapting existing neural network architectures to new problems, and (3) rapid prototyping of new solutions.• Three deep-learning applications, including segmentation, regression, image generation and representation learning, are presented as concrete examples illustrating the platform’s key features.

DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation

Accurate medical image segmentation is essential for diagnosis, surgical planning and many other applications. Convolutional Neural Networks (CNNs) have become the state-of-the-art automatic segmentation methods. However, fully automatic results may still need to be refined to become accurate and robust enough for clinical use. We propose a deep learning-based interactive segmentation method to improve the results obtained by an automatic CNN and to reduce user interactions during refinement for higher accuracy. We use one CNN to obtain an initial automatic segmentation, on which user interactions are added to indicate mis-segmentations. Another CNN takes as input the user interactions with the initial segmentation and gives a refined result. We propose to combine user interactions with CNNs through geodesic distance transforms, and propose a resolution-preserving network that gives a better dense prediction. In addition, we integrate user interactions as hard constraints into a back-propagatable Conditional Random Field. We validated the proposed framework in the context of 2D placenta segmentation from fetal MRI and 3D brain tumor segmentation from FLAIR images. Experimental results show our method achieves a large improvement from automatic CNNs, and obtains comparable and even higher accuracy with fewer user interventions and less time compared with traditional interactive methods.

Automatic Brain Tumor Segmentation Using Cascaded Anisotropic Convolutional Neural Networks

A cascade of fully convolutional neural networks is proposed to segment multi-modal Magnetic Resonance (MR) images with brain tumor into background and three hierarchical regions: whole tumor, tumor core and enhancing tumor core. The cascade is designed to decompose the multi-class segmentation problem into a sequence of three binary segmentation problems according to the subregion hierarchy. The whole tumor is segmented in the first step and the bounding box of the result is used for the tumor core segmentation in the second step. The enhancing tumor core is then segmented based on the bounding box of the tumor core segmentation result. Our networks consist of multiple layers of anisotropic and dilated convolution filters, and they are combined with multi-view fusion to reduce false positives. Residual connections and multi-scale predictions are employed in these networks to boost the segmentation performance. Experiments with BraTS 2017 validation set show that the proposed method achieved average Dice scores of 0.7859, 0.9050, 0.8378 for enhancing tumor core, whole tumor and tumor core, respectively. The corresponding values for BraTS 2017 testing set were 0.7831, 0.8739, and 0.7748, respectively.

Slic-Seg: Slice-by-Slice Segmentation Propagation of the Placenta in Fetal MRI Using One-Plane Scribbles and Online Learning

Segmentation of the placenta from fetal MRI is critical for planning of fetal surgical procedures. Unfortunately, it is made difficult by poor image quality due to sparse acquisition, inter-slice motion, and the widely varying position and orientation of the placenta between pregnant women. We propose a minimally interactive online learning-based method named Slic-Seg to obtain accurate placenta segmentations from MRI. An online random forest is first trained on data coming from scribbles provided by the user in one single selected start slice. This then forms the basis for a slice-by-slice framework that segments subsequent slices before incorporating them into the training set on the fly. The proposed method was compared with its offline counterpart that is with no retraining, and with two other widely used interactive methods. Experiments show that our method 1) has a high performance in the start slice even in cases where sparse scribbles provided by the user lead to poor results with the competitive approaches, 2) has a robust segmentation in subsequent slices, and 3) results in less variability between users.

Slic-Seg: A minimally interactive segmentation of the placenta from sparse and motion-corrupted fetal MRI in multiple views

Highlights • Minimal user interaction is needed for a good segmentation of the placenta.• Random forests with high level features improved the segmentation.• Higher accuracy than state-of-the-art interactive segmentation methods.• Co-segmentation of multiple volumes outperforms single sparse volume based method.

Myocardium segmentation combining T2 and DE MRI using Multi-Component Bivariate Gaussian mixture model

Accurately delineating the myocardium from cardiac T2 and delayed enhanced (DE) MRI is a prerequisite to identifying and quantifying the edema and infarcts. The automatic delineation is however challenging due to the heterogeneous intensity distribution of the myocardium. In this paper, we propose a fully automatic method, which combines the complementary information from the two sequences using the newly proposed Multi-Component Bivariate Gaussian (MCBG) mixture model. The expectation maximization (EM) framework is adopted to estimate the segmentation and model parameters, where a probabilistic atlas is also used. This method performs the segmentation on the two MRI sequences simultaneously, and hence improves the robustness and accuracy. The results on six clinical cases showed that the proposed method significantly improved the performance compared to the atlas-based methods: myocardium Dice scores 0.643±0.084 versus 0.576±0.103 (P=0.002) on DE MRI, and 0.623±0.129 versus 0.484±0.106 (P=0.002) on T2 MRI.

Scalable sparse shape composition and its application to liver surgical planning.

The recently proposed Sparse Shape Composition (SSC) models shape prior as a sparse linear combination of existing shapes. It is effective to represent complex shape variations, with its ability to capture gross errors and preserve local details. However, SSC has low efficiency when dealing with large-scale training data, which adversely affects its more widespread clinical use. In this paper, we investigate efficient and scalable convex optimization methods and propose a nearly real-time SSC for large dataset. The new method solves the convex optimization problem in SSC by continuously transforming it into a series of simplified problems whose solution is fast to compute, without sacrificing the accuracy. It significantly speeds up the shape modeling process. When the repositorys capacity is 10000, with 2000 vertices on each shape, the optimization can be solved by the new method in less than 10 seconds, nearly 2000 times faster than traditional method in SSC. Thus, it is more applicable in real-time clinical applications.

An Automated Localization, Segmentation and Reconstruction Framework for Fetal Brain MRI

Reconstructing a high-resolution (HR) volume from motion-corrupted and sparsely acquired stacks plays an increasing role in fetal brain Magnetic Resonance Imaging (MRI) studies. Existing reconstruction methods are time-consuming and often require user interaction to localize and extract the brain from several stacks of 2D slices. In this paper, we propose a fully automatic framework for fetal brain reconstruction that consists of three stages: (1) brain localization based on a coarse segmentation of a down-sampled input image by a Convolutional Neural Network (CNN), (2) fine segmentation by a second CNN trained with a multi-scale loss function, and (3) novel, single-parameter outlier-robust super-resolution reconstruction (SRR) for HR visualization in the standard anatomical space. We validate our framework with images from fetuses with variable degrees of ventriculomegaly associated with spina bifida. Experiments show that each step of our proposed pipeline outperforms state-of-the-art methods in both segmentation and reconstruction comparisons. Overall, we report automatic SRR reconstructions that compare favorably with those obtained by manual, labor-intensive brain segmentations. This potentially unlocks the use of automatic fetal brain reconstruction studies in clinical practice.