Thijs Kooi

A survey on deep learning in medical image analysis

Deep learning algorithms, in particular convolutional networks, have rapidly become a methodology of choice for analyzing medical images. This paper reviews the major deep learning concepts pertinent to medical image analysis and summarizes over 300 contributions to the field, most of which appeared in the last year. We survey the use of deep learning for image classification, object detection, segmentation, registration, and other tasks. Concise overviews are provided of studies per application area: neuro, retinal, pulmonary, digital pathology, breast, cardiac, abdominal, musculoskeletal. We end with a summary of the current state-of-the-art, a critical discussion of open challenges and directions for future research.

Large scale deep learning for computer aided detection of mammographic lesions

&NA; Recent advances in machine learning yielded new techniques to train deep neural networks, which resulted in highly successful applications in many pattern recognition tasks such as object detection and speech recognition. In this paper we provide a head‐to‐head comparison between a state‐of‐the art in mammography CAD system, relying on a manually designed feature set and a Convolutional Neural Network (CNN), aiming for a system that can ultimately read mammograms independently. Both systems are trained on a large data set of around 45,000 images and results show the CNN outperforms the traditional CAD system at low sensitivity and performs comparable at high sensitivity. We subsequently investigate to what extent features such as location and patient information and commonly used manual features can still complement the network and see improvements at high specificity over the CNN especially with location and context features, which contain information not available to the CNN. Additionally, a reader study was performed, where the network was compared to certified screening radiologists on a patch level and we found no significant difference between the network and the readers. HighlightsA system based on deep learning is shown to outperform a state‐of‐the art CAD system.Adding complementary handcrafted features to the CNN is shown to increase performance.The system based on deep learning is shown to perform at the level of a radiologist. Graphical abstract Figure. No caption available.

Discriminating solitary cysts from soft tissue lesions in mammography using a pretrained deep convolutional neural network

Purpose: It is estimated that 7% of women in the western world will develop palpable breast cysts in their lifetime. Even though cysts have been correlated with risk of developing breast cancer, many of them are benign and do not require follow‐up. We develop a method to discriminate benign solitary cysts from malignant masses in digital mammography. We think a system like this can have merit in the clinic as a decision aid or complementary to specialized modalities. Methods: We employ a deep convolutional neural network (CNN) to classify cyst and mass patches. Deep CNNs have been shown to be powerful classifiers, but need a large amount of training data for which medical problems are often difficult to come by. The key contribution of this paper is that we show good performance can be obtained on a small dataset by pretraining the network on a large dataset of a related task. We subsequently investigate the following: (a) when a mammographic exam is performed, two different views of the same breast are recorded. We investigate the merit of combining the output of the classifier from these two views. (b) We evaluate the importance of the resolution of the patches fed to the network. (c) A method dubbed tissue augmentation is subsequently employed, where we extract normal tissue from normal patches and superimpose this onto the actual samples aiming for a classifier invariant to occluding tissue. (d) We combine the representation extracted using the deep CNN with our previously developed features. Results: We show that using the proposed deep learning method, an area under the ROC curve (AUC) value of 0.80 can be obtained on a set of benign solitary cysts and malignant mass findings recalled in screening. We find that it works significantly better than our previously developed approach by comparing the AUC of the ROC using bootstrapping. By combining views, the results can be further improved, though this difference was not found to be significant. We find no significant difference between using a resolution of 100 versus 200 micron. The proposed tissue augmentations give a small improvement in performance, but this improvement was also not found to be significant. The final system obtained an AUC of 0.80 with 95% confidence interval [0.78, 0.83], calculated using bootstrapping. The system works best for lesions larger than 27 mm where it obtains an AUC value of 0.87. Conclusion: We have presented a computer‐aided diagnosis (CADx) method to discriminate cysts from solid lesion in mammography using features from a deep CNN trained on a large set of mass candidates, obtaining an AUC of 0.80 on a set of diagnostic exams recalled from screening. We believe the system shows great potential and comes close to the performance of recently developed spectral mammography. We think the system can be further improved when more data and computational power becomes available.

A Comparison Between a Deep Convolutional Neural Network and Radiologists for Classifying Regions of Interest in Mammography

In this paper, we employ a deep Convolutional Neural Network CNN for the classification of regions of interest of malignant soft tissue lesions in mammography and show that it performs on par to experienced radiologists. The CNN was applied to 398 regions of 5

Invariant Features for Discriminating Cysts from Solid Lesions in Mammography

\,\times \,

Deep learning of symmetrical discrepancies for computer-aided detection of mammographic masses

5i?źcm, half of which contained a malignant lesion and the other half depicted suspicious regions in normal mammograms detected by a traditional CAD system. Four radiologists participated in the study. ROC analysis was used for evaluating results. The AUC of CNN was 0.87, which was higher than the mean AUC of the radiologists 0.84, though the difference was not significant.

Classifying symmetrical differences and temporal change for the detection of malignant masses in mammography using deep neural networks

Convolutional neural networks CNNs have shown to be powerful for classification of image data and are increasingly used in medical image analysis. Therefore, CNNs might be very suitable to detect microcalcifications in mammograms. In this study, we have configured a deep learning approach to fulfill this task. To overcome the large class imbalance between pixels belonging to microcalcifications and other breast tissue, we applied a hard negative mining strategy where two CNNs are used. The deep learning approach was compared to a current state-of-the-art method for the detection of microcalcifications: the cascade classifier. Both methods were trained on a large training set including 11,711 positive and 27 million negative samples. For testing, an independent test set was configured containing 5,298 positive and 18 million negative samples. The mammograms included in this study were acquired on mammography systems from three manufactures: Hologic, GE, and Siemens. Receiver operating characteristics analysis was carried out. Over the whole specificity range, the CNN approach yielded a higher sensitivity compared to the cascade classifier. Significantly higher mean sensitivities were obtained with the CNN on the mammograms of each individual manufacturer compared to the cascade classifier in the specificity range of 0 to 0.1. To our knowledge, this was the first study to use a deep learning strategy for the detection of microcalcifications in mammograms.

Conditional random field modelling of interactions between findings in mammography

Feature extraction is an integral of all Computer Aided Diagnosis (CAD) systems. Due to the presence of fibroglandular tissue however, measurements are perturbed by unwanted influences and therefore, the same descriptor will yield different values for different amounts of occluding structures. To aid the statistical learning used for classification, we need to design features that are invariant to unwanted influences. In this paper, we propose a simple model of the tumour and its surrounding tissue and show how this model can be used to derive descriptors that are invariant to obscuring tissue, rather than heuristically defining a set of descriptors, which is common practice in many CAD papers. We tailor the descriptors to optimally discriminate between tumours and cysts, by assuming a parametric form of the lesions. Results show a significant discriminative improvement over simple, more commonly used contrast features and we obtained an AUC of 0.77 using both CC and MLO images.