Mohsen Ghafoorian

A survey on deep learning in medical image analysis

Deep learning algorithms, in particular convolutional networks, have rapidly become a methodology of choice for analyzing medical images. This paper reviews the major deep learning concepts pertinent to medical image analysis and summarizes over 300 contributions to the field, most of which appeared in the last year. We survey the use of deep learning for image classification, object detection, segmentation, registration, and other tasks. Concise overviews are provided of studies per application area: neuro, retinal, pulmonary, digital pathology, breast, cardiac, abdominal, musculoskeletal. We end with a summary of the current state-of-the-art, a critical discussion of open challenges and directions for future research.

Location Sensitive Deep Convolutional Neural Networks for Segmentation of White Matter Hyperintensities

The anatomical location of imaging features is of crucial importance for accurate diagnosis in many medical tasks. Convolutional neural networks (CNN) have had huge successes in computer vision, but they lack the natural ability to incorporate the anatomical location in their decision making process, hindering success in some medical image analysis tasks. In this paper, to integrate the anatomical location information into the network, we propose several deep CNN architectures that consider multi-scale patches or take explicit location features while training. We apply and compare the proposed architectures for segmentation of white matter hyperintensities in brain MR images on a large dataset. As a result, we observe that the CNNs that incorporate location information substantially outperform a conventional segmentation method with handcrafted features as well as CNNs that do not integrate location information. On a test set of 50 scans, the best configuration of our networks obtained a Dice score of 0.792, compared to 0.805 for an independent human observer. Performance levels of the machine and the independent human observer were not statistically significantly different (p-value = 0.06).

Deep multi-scale location-aware 3D convolutional neural networks for automated detection of lacunes of presumed vascular origin

Lacunes of presumed vascular origin (lacunes) are associated with an increased risk of stroke, gait impairment, and dementia and are a primary imaging feature of the small vessel disease. Quantification of lacunes may be of great importance to elucidate the mechanisms behind neuro-degenerative disorders and is recommended as part of study standards for small vessel disease research. However, due to the different appearance of lacunes in various brain regions and the existence of other similar-looking structures, such as perivascular spaces, manual annotation is a difficult, elaborative and subjective task, which can potentially be greatly improved by reliable and consistent computer-aided detection (CAD) routines. In this paper, we propose an automated two-stage method using deep convolutional neural networks (CNN). We show that this method has good performance and can considerably benefit readers. We first use a fully convolutional neural network to detect initial candidates. In the second step, we employ a 3D CNN as a false positive reduction tool. As the location information is important to the analysis of candidate structures, we further equip the network with contextual information using multi-scale analysis and integration of explicit location features. We trained, validated and tested our networks on a large dataset of 1075 cases obtained from two different studies. Subsequently, we conducted an observer study with four trained observers and compared our method with them using a free-response operating characteristic analysis. Shown on a test set of 111 cases, the resulting CAD system exhibits performance similar to the trained human observers and achieves a sensitivity of 0.974 with 0.13 false positives per slice. A feasibility study also showed that a trained human observer would considerably benefit once aided by the CAD system.

Longitudinal multiple sclerosis lesion segmentation: Resource and challenge

Abstract In conjunction with the ISBI 2015 conference, we organized a longitudinal lesion segmentation challenge providing training and test data to registered participants. The training data consisted of five subjects with a mean of 4.4 time‐points, and test data of fourteen subjects with a mean of 4.4 time‐points. All 82 data sets had the white matter lesions associated with multiple sclerosis delineated by two human expert raters. Eleven teams submitted results using state‐of‐the‐art lesion segmentation algorithms to the challenge, with ten teams presenting their results at the conference. We present a quantitative evaluation comparing the consistency of the two raters as well as exploring the performance of the eleven submitted results in addition to three other lesion segmentation algorithms. The challenge presented three unique opportunities: (1) the sharing of a rich data set; (2) collaboration and comparison of the various avenues of research being pursued in the community; and (3) a review and refinement of the evaluation metrics currently in use. We report on the performance of the challenge participants, as well as the construction and evaluation of a consensus delineation. The image data and manual delineations will continue to be available for download, through an evaluation website2 as a resource for future researchers in the area. This data resource provides a platform to compare existing methods in a fair and consistent manner to each other and multiple manual raters. HighlightsPublic lesion data base of 21 training data sets and 61 testing data sets.Fully automated evaluation website.Comparison between 14 state‐of‐the‐art algorithms and 2 manual delineators.

Automated detection of white matter hyperintensities of all sizes in cerebral small vessel disease

PURPOSE White matter hyperintensities (WMH) are seen on FLAIR-MRI in several neurological disorders, including multiple sclerosis, dementia, Parkinsonism, stroke and cerebral small vessel disease (SVD). WMHs are often used as biomarkers for prognosis or disease progression in these diseases, and additionally longitudinal quantification of WMHs is used to evaluate therapeutic strategies. Human readers show considerable disagreement and inconsistency on detection of small lesions. A multitude of automated detection algorithms for WMHs exists, but since most of the current automated approaches are tuned to optimize segmentation performance according to Jaccard or Dice scores, smaller WMHs often go undetected in these approaches. In this paper, the authors propose a method to accurately detect all WMHs, large as well as small. METHODS A two-stage learning approach was used to discriminate WMHs from normal brain tissue. Since small and larger WMHs have quite a different appearance, the authors have trained two probabilistic classifiers: one for the small WMHs (⩽3 mm effective diameter) and one for the larger WMHs (>3 mm in-plane effective diameter). For each size-specific classifier, an Adaboost is trained for five iterations, with random forests as the basic classifier. The feature sets consist of 22 features including intensities, location information, blob detectors, and second order derivatives. The outcomes of the two first-stage classifiers were combined into a single WMH likelihood by a second-stage classifier. Their method was trained and evaluated on a dataset with MRI scans of 362 SVD patients (312 subjects for training and validation annotated by one and 50 for testing annotated by two trained raters). To analyze performance on the separate test set, the authors performed a free-response receiving operating characteristic (FROC) analysis, instead of using segmentation based methods that tend to ignore the contribution of small WMHs. RESULTS Experimental results based on FROC analysis demonstrated a close performance of the proposed computer aided detection (CAD) system to human readers. While an independent reader had 0.78 sensitivity with 28 false positives per volume on average, their proposed CAD system reaches a sensitivity of 0.73 with the same number of false positives. CONCLUSIONS The authors have developed a CAD system with all its ingredients being optimized for a better detection of WMHs of all size, which shows performance close to an independent reader.

Non-uniform patch sampling with deep convolutional neural networks for white matter hyperintensity segmentation

Convolutional neural networks (CNN) have been widely used for visual recognition tasks including semantic segmentation of images. While the existing methods consider uniformly sampled single-or multi-scale patches from the neighborhood of each voxel, this approach might be sub-optimal as it captures and processes unnecessary details far away from the center of the patch. We instead propose to train CNNs with non-uniformly sampled patches that allow a wider extent for the sampled patches. This results in more captured contextual information, which is in particular of interest for biomedical image analysis, where the anatomical location of imaging features are often crucial. We evaluate and compare this strategy for white matter hyperintensity segmentation on a test set of 46 MRI scans. We show that the proposed method not only outperforms identical CNNs with uniform patches of the same size (0.780 Dice coefficient compared to 0.736), but also gets very close to the performance of an independent human expert (0.796 Dice coefficient).

Nonlinear temporal dynamics of cerebral small vessel disease: The RUN DMC study.

Objective: To investigate the temporal dynamics of cerebral small vessel disease (SVD) by 3 consecutive assessments over a period of 9 years, distinguishing progression from regression. Methods: Changes in SVD markers of 276 participants of the Radboud University Nijmegen Diffusion Tensor and Magnetic Resonance Imaging Cohort (RUN DMC) cohort were assessed at 3 time points over 9 years. We assessed white matter hyperintensities (WMH) volume by semiautomatic segmentation and rated lacunes and microbleeds manually. We categorized baseline WMH severity as mild, moderate, or severe according to the modified Fazekas scale. We performed mixed-effects regression analysis including a quadratic term for increasing age. Results: Mean WMH progression over 9 years was 4.7 mL (0.54 mL/y; interquartile range 0.95–5.5 mL), 20.3% of patients had incident lacunes (2.3%/y), and 18.9% had incident microbleeds (2.2%/y). WMH volume declined in 9.4% of the participants during the first follow-up interval, but only for 1 participant (0.4%) throughout the whole follow-up. Lacunes disappeared in 3.6% and microbleeds in 5.7% of the participants. WMH progression accelerated over time: including a quadratic term for increasing age during follow-up significantly improved the model (p < 0.001). SVD progression was predominantly seen in participants with moderate to severe WMH at baseline compared to those with mild WMH (odds ratio [OR] 35.5, 95% confidence interval [CI] 15.8–80.0, p < 0.001 for WMH progression; OR 5.7, 95% CI 2.8–11.2, p < 0.001 for incident lacunes; and OR 2.9, 95% CI 1.4–5.9, p = 0.003 for incident microbleeds). Conclusions: SVD progression is nonlinear, accelerating over time, and a highly dynamic process, with progression interrupted by reduction in some, in a population that on average shows progression.

Transfer Learning for Domain Adaptation in MRI: Application in Brain Lesion Segmentation

Magnetic Resonance Imaging (MRI) is widely used in routine clinical diagnosis and treatment. However, variations in MRI acquisition protocols result in different appearances of normal and diseased tissue in the images. Convolutional neural networks (CNNs), which have shown to be successful in many medical image analysis tasks, are typically sensitive to the variations in imaging protocols. Therefore, in many cases, networks trained on data acquired with one MRI protocol, do not perform satisfactorily on data acquired with different protocols. This limits the use of models trained with large annotated legacy datasets on a new dataset with a different domain which is often a recurring situation in clinical settings. In this study, we aim to answer the following central questions regarding domain adaptation in medical image analysis: Given a fitted legacy model, (1) How much data from the new domain is required for a decent adaptation of the original network?; and, (2) What portion of the pre-trained model parameters should be retrained given a certain number of the new domain training samples? To address these questions, we conducted extensive experiments in white matter hyperintensity segmentation task. We trained a CNN on legacy MR images of brain and evaluated the performance of the domain-adapted network on the same task with images from a different domain. We then compared the performance of the model to the surrogate scenarios where either the same trained network is used or a new network is trained from scratch on the new dataset. The domain-adapted network tuned only by two training examples achieved a Dice score of 0.63 substantially outperforming a similar network trained on the same set of examples from scratch.

Accelerated development of cerebral small vessel disease in young stroke patients

Objective: To study the long-term prevalence of small vessel disease after young stroke and to compare this to healthy controls. Methods: This prospective cohort study comprises 337 patients with an ischemic stroke or TIA, aged 18–50 years, without a history of TIA or stroke. In addition, 90 age- and sex-matched controls were included. At follow-up, lacunes, microbleeds, and white matter hyperintensity (WMH) volume were assessed using MRI. To investigate the relation between risk factors and small vessel disease, logistic and linear regression were used. Results: After mean follow-up of 9.9 (SD 8.1) years, 337 patients were included (227 with an ischemic stroke and 110 with a TIA). Mean age of patients was 49.8 years (SD 10.3) and 45.4% were men; for controls, mean age was 49.4 years (SD 11.9) and 45.6% were men. Compared with controls, patients more often had at least 1 lacune (24.0% vs 4.5%, p < 0.0001). In addition, they had a higher WMH volume (median 1.5 mL [interquartile range (IQR) 0.5–3.7] vs 0.4 mL [IQR 0.0–1.0], p < 0.001). Compared with controls, patients had the same volume WMHs on average 10–20 years earlier. In the patient group, age at stroke (β = 0.03, 95% confidence interval [CI] 0.02–0.04) hypertension (β = 0.22, 95% CI 0.04–0.39), and smoking (β = 0.18, 95% CI 0.01–0.34) at baseline were associated with WMH volume. Conclusions: Patients with a young stroke have a higher burden of small vessel disease than controls adjusted for confounders. Cerebral aging seems accelerated by 10–20 years in these patients, which may suggest an increased vulnerability to vascular risk factors.

Small white matter lesion detection in cerebral small vessel disease

Cerebral small vessel disease (SVD) is a common finding on magnetic resonance images of elderly people. White matter lesions (WML) are important markers for not only the small vessel disease, but also neuro-degenerative diseases including multiple sclerosis, Alzheimer’s disease and vascular dementia. Volumetric measurements such as the “total lesion load”, have been studied and related to these diseases. With respect to SVD we conjecture that small lesions are important, as they have been observed to grow over time and they form the majority of lesions in number. To study these small lesions they need to be annotated, which is a complex and time-consuming task. Existing (semi) automatic methods have been aimed at volumetric measurements and large lesions, and are not suitable for the detection of small lesions. In this research we established a supervised voxel classification CAD system, optimized and trained to exclusively detect small WMLs. To achieve this, several preprocessing steps were taken, which included a robust standardization of subject intensities to reduce inter-subject intensity variability as much as possible. A number of features that were found to be well identifying small lesions were calculated including multimodal intensities, tissue probabilities, several features for accurate location description, a number of second order derivative features as well as multi-scale annular filter for blobness detection. Only small lesions were used to learn the target concept via Adaboost using random forests as its basic classifiers. Finally the results were evaluated using Free-response receiver operating characteristic.