Christian F. Baumgartner

Unsupervised Domain Adaptation in Brain Lesion Segmentation with Adversarial Networks

Significant advances have been made towards building accurate automatic segmentation systems for a variety of biomedical applications using machine learning. However, the performance of these systems often degrades when they are applied on new data that differ from the training data, for example, due to variations in imaging protocols. Manually annotating new data for each test domain is not a feasible solution. In this work we investigate unsupervised domain adaptation using adversarial neural networks to train a segmentation method which is more robust to differences in the input data, and which does not require any annotations on the test domain. Specifically, we derive domain-invariant features by learning to counter an adversarial network, which attempts to classify the domain of the input data by observing the activations of the segmentation network. Furthermore, we propose a multi-connected domain discriminator for improved adversarial training. Our system is evaluated using two MR databases of subjects with traumatic brain injuries, acquired using different scanners and imaging protocols. Using our unsupervised approach, we obtain segmentation accuracies which are close to the upper bound of supervised domain adaptation.

High-resolution dynamic MR imaging of the thorax for respiratory motion correction of PET using groupwise manifold alignment

Respiratory motion is a complicating factor in PET imaging as it leads to blurring of the reconstructed images which adversely affects disease diagnosis and staging. Existing motion correction techniques are often based on 1D navigators which cannot capture the inter- and intra-cycle variabilities that may occur in respiration. MR imaging is an attractive modality for estimating such motion more accurately, and the recent emergence of hybrid PET/MR systems allows the combination of the high molecular sensitivity of PET with the versatility of MR. However, current MR imaging techniques cannot achieve good image contrast inside the lungs in 3D. 2D slices, on the other hand, have excellent contrast properties inside the lungs due to the in-flow of previously unexcited blood, but lack the coverage of 3D volumes. In this work we propose an approach for the robust, navigator-less reconstruction of dynamic 3D volumes from 2D slice data. Our technique relies on the fact that data acquired at different slice positions have similar low-dimensional representations which can be extracted using manifold learning. By aligning these manifolds we are able to obtain accurate matchings of slices with regard to respiratory position. The approach naturally models all respiratory variabilities. We compare our method against two recently proposed MR slice stacking methods for the correction of PET data: a technique based on a 1D pencil beam navigator, and an image-based technique. On synthetic data with a known ground truth our proposed technique produces significantly better reconstructions than all other examined techniques. On real data without a known ground truth the method gives the most plausible reconstructions and high consistency of reconstruction. Lastly, we demonstrate how our method can be applied for the respiratory motion correction of simulated PET/MR data.

SonoNet: Real-Time Detection and Localisation of Fetal Standard Scan Planes in Freehand Ultrasound

Identifying and interpreting fetal standard scan planes during 2-D ultrasound mid-pregnancy examinations are highly complex tasks, which require years of training. Apart from guiding the probe to the correct location, it can be equally difficult for a non-expert to identify relevant structures within the image. Automatic image processing can provide tools to help experienced as well as inexperienced operators with these tasks. In this paper, we propose a novel method based on convolutional neural networks, which can automatically detect 13 fetal standard views in freehand 2-D ultrasound data as well as provide a localization of the fetal structures via a bounding box. An important contribution is that the network learns to localize the target anatomy using weak supervision based on image-level labels only. The network architecture is designed to operate in real-time while providing optimal output for the localization task. We present results for real-time annotation, retrospective frame retrieval from saved videos, and localization on a very large and challenging dataset consisting of images and video recordings of full clinical anomaly screenings. We found that the proposed method achieved an average F1-score of 0.798 in a realistic classification experiment modeling real-time detection, and obtained a 90.09% accuracy for retrospective frame retrieval. Moreover, an accuracy of 77.8% was achieved on the localization task.

Real-Time Standard Scan Plane Detection and Localisation in Fetal Ultrasound Using Fully Convolutional Neural Networks

Fetal mid-pregnancy scans are typically carried out according to fixed protocols. Accurate detection of abnormalities and correct biometric measurements hinge on the correct acquisition of clearly defined standard scan planes. Locating these standard planes requires a high level of expertise. However, there is a worldwide shortage of expert sonographers. In this paper, we consider a fully automated system based on convolutional neural networks which can detect twelve standard scan planes as defined by the UK fetal abnormality screening programme. The network design allows real-time inference and can be naturally extended to provide an approximate localisation of the fetal anatomy in the image. Such a framework can be used to automate or assist with scan plane selection, or for the retrospective retrieval of scan planes from recorded videos. The method is evaluated on a large database of 1003 volunteer mid-pregnancy scans. We show that standard planes acquired in a clinical scenario are robustly detected with a precision and recall of 69 % and 80 %, which is superior to the current state-of-the-art. Furthermore, we show that it can retrospectively retrieve correct scan planes with an accuracy of 71 % for cardiac views and 81 % for non-cardiac views.

Information Processing in Medical Imaging (IPMI)

Respiratory motion is a complicating factor for many applications in medical imaging and there is significant interest in dynamic imaging that can be used to estimate such motion. Magnetic resonance imaging (MRI) is an attractive modality for motion estimation but current techniques cannot achieve good image contrast inside the lungs. Manifold learning is a powerful tool to discover the underlying structure of high-dimensional data. Aligning the manifolds of multiple datasets can be useful to establish relationships between different types of data. However, the current state-of-the-art in manifold alignment is not robust to the wide variations in manifold structure that may occur in clinical datasets. In this work we propose a novel, fully automatic technique for the simultaneous alignment of large numbers of manifolds with varying manifold structure. We apply the technique to reconstruct high-resolution and high-contrast dynamic 3D MRI images from multiple 2D datasets for the purpose of respiratory motion estimation. The proposed method is validated on synthetic data with known ground truth and real data. We demonstrate that our approach can be applied to reconstruct significantly more accurate and consistent dynamic images of the lungs compared to the current state-of-the-art in manifold alignment.

An Exploration of 2D and 3D Deep Learning Techniques for Cardiac MR Image Segmentation

Accurate segmentation of the heart is an important step towards evaluating cardiac function. In this paper, we present a fully automated framework for segmentation of the left (LV) and right (RV) ventricular cavities and the myocardium (Myo) on short-axis cardiac MR images. We investigate various 2D and 3D convolutional neural network architectures for this task. We investigate the suitability of various state-of-the art 2D and 3D convolutional neural network architectures, as well as slight modifications thereof, for this task. Experiments were performed on the ACDC 2017 challenge training dataset comprising cardiac MR images of 100 patients, where manual reference segmentations were made available for end-diastolic (ED) and end-systolic (ES) frames. We find that processing the images in a slice-by-slice fashion using 2D networks is beneficial due to a relatively large slice thickness. However, the exact network architecture only plays a minor role. We report mean Dice coefficients of

Groupwise simultaneous manifold alignment for high-resolution dynamic MR imaging of respiratory motion

0.950

Multi-Atlas Segmentation Using Partially Annotated Data: Methods and Annotation Strategies

(LV),

Autoadaptive motion modelling for MR-based respiratory motion estimation

0.893

A Lifelong Learning Approach to Brain MR Segmentation Across Scanners and Protocols

(RV), and