Pim Moeskops

Automatic Segmentation of MR Brain Images With a Convolutional Neural Network

Automatic segmentation in MR brain images is important for quantitative analysis in large-scale studies with images acquired at all ages. This paper presents a method for the automatic segmentation of MR brain images into a number of tissue classes using a convolutional neural network. To ensure that the method obtains accurate segmentation details as well as spatial consistency, the network uses multiple patch sizes and multiple convolution kernel sizes to acquire multi-scale information about each voxel. The method is not dependent on explicit features, but learns to recognise the information that is important for the classification based on training data. The method requires a single anatomical MR image only. The segmentation method is applied to five different data sets: coronal T2-weighted images of preterm infants acquired at 30 weeks postmenstrual age (PMA) and 40 weeks PMA, axial T2-weighted images of preterm infants acquired at 40 weeks PMA, axial T1-weighted images of ageing adults acquired at an average age of 70 years, and T1-weighted images of young adults acquired at an average age of 23 years. The method obtained the following average Dice coefficients over all segmented tissue classes for each data set, respectively: 0.87, 0.82, 0.84, 0.86, and 0.91. The results demonstrate that the method obtains accurate segmentations in all five sets, and hence demonstrates its robustness to differences in age and acquisition protocol.Automatic segmentation in MR brain images is important for quantitative analysis in large-scale studies with images acquired at all ages. This paper presents a method for the automatic segmentation of MR brain images into a number of tissue classes using a convolutional neural network. To ensure that the method obtains accurate segmentation details as well as spatial consistency, the network uses multiple patch sizes and multiple convolution kernel sizes to acquire multi-scale information about each voxel. The method is not dependent on explicit features, but learns to recognise the information that is important for the classification based on training data. The method requires a single anatomical MR image only. The segmentation method is applied to five different data sets: coronal T2-weighted images of preterm infants acquired at 30 weeks postmenstrual age (PMA) and 40 weeks PMA, axial T2-weighted images of preterm infants acquired at 40 weeks PMA, axial T1-weighted images of ageing adults acquired at an average age of 70 years, and T1-weighted images of young adults acquired at an average age of 23 years. The method obtained the following average Dice coefficients over all segmented tissue classes for each data set, respectively: 0.87, 0.82, 0.84, 0.86, and 0.91. The results demonstrate that the method obtains accurate segmentations in all five sets, and hence demonstrates its robustness to differences in age and acquisition protocol.

Structural and Resting State Functional Connectivity of the Subthalamic Nucleus: Identification of Motor STN Parts and the Hyperdirect Pathway

Deep brain stimulation (DBS) for Parkinson’s disease often alleviates the motor symptoms, but causes cognitive and emotional side effects in a substantial number of cases. Identification of the motor part of the subthalamic nucleus (STN) as part of the presurgical workup could minimize these adverse effects. In this study, we assessed the STN’s connectivity to motor, associative, and limbic brain areas, based on structural and functional connectivity analysis of volunteer data. For the structural connectivity, we used streamline counts derived from HARDI fiber tracking. The resulting tracks supported the existence of the so-called “hyperdirect” pathway in humans. Furthermore, we determined the connectivity of each STN voxel with the motor cortical areas. Functional connectivity was calculated based on functional MRI, as the correlation of the signal within a given brain voxel with the signal in the STN. Also, the signal per STN voxel was explained in terms of the correlation with motor or limbic brain seed ROI areas. Both right and left STN ROIs appeared to be structurally and functionally connected to brain areas that are part of the motor, associative, and limbic circuit. Furthermore, this study enabled us to assess the level of segregation of the STN motor part, which is relevant for the planning of STN DBS procedures.

Evaluation of automatic neonatal brain segmentation algorithms:the NeoBrainS12 challenge

A number of algorithms for brain segmentation in preterm born infants have been published, but a reliable comparison of their performance is lacking. The NeoBrainS12 study (http://neobrains12.isi.uu.nl), providing three different image sets of preterm born infants, was set up to provide such a comparison. These sets are (i) axial scans acquired at 40 weeks corrected age, (ii) coronal scans acquired at 30 weeks corrected age and (iii) coronal scans acquired at 40 weeks corrected age. Each of these three sets consists of three T1- and T2-weighted MR images of the brain acquired with a 3T MRI scanner. The task was to segment cortical grey matter, non-myelinated and myelinated white matter, brainstem, basal ganglia and thalami, cerebellum, and cerebrospinal fluid in the ventricles and in the extracerebral space separately. Any team could upload the results and all segmentations were evaluated in the same way. This paper presents the results of eight participating teams. The results demonstrate that the participating methods were able to segment all tissue classes well, except myelinated white matter.

Development of Cortical Morphology Evaluated with Longitudinal MR Brain Images of Preterm Infants

Introduction The cerebral cortex develops rapidly in the last trimester of pregnancy. In preterm infants, brain development is very vulnerable because of their often complicated extra-uterine conditions. The aim of this study was to quantitatively describe cortical development in a cohort of 85 preterm infants with and without brain injury imaged at 30 and 40 weeks postmenstrual age (PMA). Methods In the acquired T2-weighted MR images, unmyelinated white matter (UWM), cortical grey matter (CoGM), and cerebrospinal fluid in the extracerebral space (CSF) were automatically segmented. Based on these segmentations, cortical descriptors evaluating volume, surface area, thickness, gyrification index, and global mean curvature were computed at both time points, for the whole brain, as well as for the frontal, temporal, parietal, and occipital lobes separately. Additionally, visual scoring of brain abnormality was performed using a conventional scoring system at 40 weeks PMA. Results The evaluated descriptors showed larger change in the occipital lobes than in the other lobes. Moreover, the cortical descriptors showed an association with the abnormality scores: gyrification index and global mean curvature decreased, whereas, interestingly, median cortical thickness increased with increasing abnormality score. This was more pronounced at 40 weeks PMA than at 30 weeks PMA, suggesting that the period between 30 and 40 weeks PMA might provide a window of opportunity for intervention to prevent delay in cortical development.

Automatic segmentation of MR brain images of preterm infants using supervised classification

Preterm birth is often associated with impaired brain development. The state and expected progression of preterm brain development can be evaluated using quantitative assessment of MR images. Such measurements require accurate segmentation of different tissue types in those images. This paper presents an algorithm for the automatic segmentation of unmyelinated white matter (WM), cortical grey matter (GM), and cerebrospinal fluid in the extracerebral space (CSF). The algorithm uses supervised voxel classification in three subsequent stages. In the first stage, voxels that can easily be assigned to one of the three tissue types are labelled. In the second stage, dedicated analysis of the remaining voxels is performed. The first and the second stages both use two-class classification for each tissue type separately. Possible inconsistencies that could result from these tissue-specific segmentation stages are resolved in the third stage, which performs multi-class classification. A set of T1- and T2-weighted images was analysed, but the optimised system performs automatic segmentation using a T2-weighted image only. We have investigated the performance of the algorithm when using training data randomly selected from completely annotated images as well as when using training data from only partially annotated images. The method was evaluated on images of preterm infants acquired at 30 and 40weeks postmenstrual age (PMA). When the method was trained using random selection from the completely annotated images, the average Dice coefficients were 0.95 for WM, 0.81 for GM, and 0.89 for CSF on an independent set of images acquired at 30weeks PMA. When the method was trained using only the partially annotated images, the average Dice coefficients were 0.95 for WM, 0.78 for GM and 0.87 for CSF for the images acquired at 30weeks PMA, and 0.92 for WM, 0.80 for GM and 0.85 for CSF for the images acquired at 40weeks PMA. Even though the segmentations obtained using training data from the partially annotated images resulted in slightly lower Dice coefficients, the performance in all experiments was close to that of a second human expert (0.93 for WM, 0.79 for GM and 0.86 for CSF for the images acquired at 30weeks, and 0.94 for WM, 0.76 for GM and 0.87 for CSF for the images acquired at 40weeks). These results show that the presented method is robust to age and acquisition protocol and that it performs accurate segmentation of WM, GM, and CSF when the training data is extracted from complete annotations as well as when the training data is extracted from partial annotations only. This extends the applicability of the method by reducing the time and effort necessary to create training data in a population with different characteristics.

Deep learning for multi-task medical image segmentation in multiple modalities

Automatic segmentation of medical images is an important task for many clinical applications. In practice, a wide range of anatomical structures are visualised using different imaging modalities. In this paper, we investigate whether a single convolutional neural network (CNN) can be trained to perform different segmentation tasks.

Relation between clinical risk factors, early cortical changes, and neurodevelopmental outcome in preterm infants

Cortical folding mainly takes place in the third trimester of pregnancy and may therefore be influenced by preterm birth. The aim of this study was to evaluate the development of specific cortical structures between early age (around 30weeks postmenstrual age) and term-equivalent age (TEA, around 40weeks postmenstrual age) in 71 extremely preterm infants, and to associate this to clinical characteristics and neurodevelopmental outcome at two years of age. First, analysis showed that the central sulcus (CS), lateral fissure (LF) and insula (INS) were present at early MRI in all infants, whereas the other sulci (post-central sulcus [PCS], superior temporal sulcus [STS], superior [SFS] and inferior [IFS] frontal sulcus) were only seen in part of the infants. Relative growth from early to TEA examination was largest in the SFS. A rightward asymmetry of the surface area was seen in development between both examinations except for the LF, which showed a leftward asymmetry at both time points. Second, lower birth weight z-score, multiple pregnancy and prolonged mechanical ventilation showed negative effects on cortical folding of the CS, LF, INS, STS and PCS, mainly on the first examination, suggesting that sulci developing the earliest were the most affected by clinical factors. Finally, in this cohort, a clear association between cortical folding and neurodevelopmental outcome at two years corrected age was found, particularly for receptive language.

Adversarial training and dilated convolutions for brain MRI segmentation

Convolutional neural networks (CNNs) have been applied to various automatic image segmentation tasks in medical image analysis, including brain MRI segmentation. Generative adversarial networks have recently gained popularity because of their power in generating images that are difficult to distinguish from real images.

Automatic segmentation of the preterm neonatal brain with MRI using supervised classification

Cortical folding ensues around 13-14 weeks gestational age and a qualitative analysis of the cortex around this period is required to observe and better understand the folds arousal. A quantitative assessment of cortical folding can be based on the cortical surface area, extracted from segmentations of unmyelinated white matter (UWM), cortical grey matter (CoGM) and cerebrospinal fluid in the extracerebral space (CSF). This work presents a method for automatic segmentation of these tissue types in preterm infants. A set of T1- and T2-weighted images of ten infants scanned at 30 weeks postmenstrual age was used. The reference standard was obtained by manual expert segmentation. The method employs supervised pixel classification in three subsequent stages. The classification is performed based on the set of spatial and texture features. Segmentation results are evaluated in terms of Dice coefficient (DC), Hausdorff distance (HD), and modified Hausdorff distance (MHD) defined as 95th percentile of the HD. The method achieved average DC of 0.94 for UWM, 0.73 for CoGM and 0.86 for CSF. The average HD and MHD were 6.89 mm and 0.34 mm for UWM, 6.49 mm and 0.82 mm for CoGM, and 7.09 mm and 0.79 mm for CSF, respectively. The presented method can provide volumetric measurements of the segmented tissues, and it enables quantification of cortical characteristics. Therefore, the method provides a basis for evaluation of clinical relevance of these biomarkers in the given population.

Domain-Adversarial Neural Networks to Address the Appearance Variability of Histopathology Images

Preparing and scanning histopathology slides consists of several steps, each with a multitude of parameters. The parameters can vary between pathology labs and within the same lab over time, resulting in significant variability of the tissue appearance that hampers the generalization of automatic image analysis methods. Typically, this is addressed with ad-hoc approaches such as staining normalization that aim to reduce the appearance variability. In this paper, we propose a systematic solution based on domain-adversarial neural networks. We hypothesize that removing the domain information from the model representation leads to better generalization. We tested our hypothesis for the problem of mitosis detection in breast cancer histopathology images and made a comparative analysis with two other approaches. We show that combining color augmentation with domain-adversarial training is a better alternative than standard approaches to improve the generalization of deep learning methods.