Idit Diamant

Deep learning with non-medical training used for chest pathology identification

In this work, we examine the strength of deep learning approaches for pathology detection in chest radiograph data. Convolutional neural networks (CNN) deep architecture classification approaches have gained popularity due to their ability to learn mid and high level image representations. We explore the ability of a CNN to identify different types of pathologies in chest x-ray images. Moreover, since very large training sets are generally not available in the medical domain, we explore the feasibility of using a deep learning approach based on non-medical learning. We tested our algorithm on a dataset of 93 images. We use a CNN that was trained with ImageNet, a well-known large scale nonmedical image database. The best performance was achieved using a combination of features extracted from the CNN and a set of low-level features. We obtained an area under curve (AUC) of 0.93 for Right Pleural Effusion detection, 0.89 for Enlarged heart detection and 0.79 for classification between healthy and abnormal chest x-ray, where all pathologies are combined into one large class. This is a first-of-its-kind experiment that shows that deep learning with large scale non-medical image databases may be sufficient for general medical image recognition tasks.

Chest pathology detection using deep learning with non-medical training

In this work, we examine the strength of deep learning approaches for pathology detection in chest radiographs. Convolutional neural networks (CNN) deep architecture classification approaches have gained popularity due to their ability to learn mid and high level image representations. We explore the ability of CNN learned from a non-medical dataset to identify different types of pathologies in chest x-rays. We tested our algorithm on a 433 image dataset. The best performance was achieved using CNN and GIST features. We obtained an area under curve (AUC) of 0.87-0.94 for the different pathologies. The results demonstrate the feasibility of detecting pathology in chest x-rays using deep learning approaches based on non-medical learning. This is a first-of-its-kind experiment that shows that Deep learning with ImageNet, a large scale non-medical image database may be a good substitute to domain specific representations, which are yet to be available, for general medical image recognition tasks.

Fully Convolutional Network for Liver Segmentation and Lesions Detection

In this work we explore a fully convolutional network (FCN) for the task of liver segmentation and liver metastases detection in computed tomography (CT) examinations. FCN has proven to be a very powerful tool for semantic segmentation. We explore the FCN performance on a relatively small dataset and compare it to patch based CNN and sparsity based classification schemes. Our data contains CT examinations from 20 patients with overall 68 lesions and 43 livers marked in one slice and 20 different patients with a full 3D liver segmentation. We ran 3-fold cross-validation and results indicate superiority of the FCN over all other methods tested. Using our fully automatic algorithm we achieved true positive rate of 0.86 and 0.6 false positive per case which are very promising and clinically relevant results.

How to select the elastic modulus for cancellous bone in patient-specific continuum models of the spine.

Patient-specific finite element (FE) modelling is a promising technology that is expected to support clinical assessment of the spine in the near future. To allow rapid, robust and economic patient-specific modelling of the whole spine or of large spine segments, it is practicable to consider vertebral cancellous bone in the spine as a continuum material, but the elastic modulus of that continuum material must reflect the quality of the individual vertebral bone. A numerical parametric model of lattice trabecular architecture has been developed for determining the apparent elastic modulus of cancellous bone Ecb in vertebrae. The model inputs were apparent morphological parameters (trabecular thickness TbTh and trabecular separation TbSp) and the bone mineral density (BMD), which can all be measuredin vivo, using the spatial resolution of current clinical quantitative computed tomography (QCT) commercial whole-body scanners. The model predicted that Ecb values between 30 and 110 MPa represent normal morphology and BMD of human spinal cancellous bone. The present Ecb to TbTh, TbSp and BMD relationships pave the way for automatic generation of patientspecific continuum FE spine models that consider the individuals osteoporotic or other degenerative condition of cancellous bone.

Chest pathology identification using deep feature selection with non-medical training

We demonstrate the feasibility of detecting pathology in chest X-rays using deep learning approaches based on non-medical learning. Convolutional neural networks (CNN) learn higher level image representations. In this work, we explore the features extracted from layers of the CNN along with a set of classical features, including GIST and bag-of-words. We show results of classification using each feature set as well as fusing among the features. Finally, we perform feature selection on the collection of features to show the most informative feature set for the task. Results of 0.78–0.95 AUC for various pathologies are shown on a data-set of more than 600 radiographs. This study shows the strength and robustness of the CNN features. We conclude that deep learning with large-scale non- medical image databases may be a good substitute, or addition to domain-specific representations which are yet to be available for general medical image recognition tasks.

Improved Patch-Based Automated Liver Lesion Classification by Separate Analysis of the Interior and Boundary Regions

The bag-of-visual-words (BoVW) method with construction of a single dictionary of visual words has been used previously for a variety of classification tasks in medical imaging, including the diagnosis of liver lesions. In this paper, we describe a novel method for automated diagnosis of liver lesions in portal-phase computed tomography (CT) images that improves over single-dictionary BoVW methods by using an image patch representation of the interior and boundary regions of the lesions. Our approach captures characteristics of the lesion margin and of the lesion interior by creating two separate dictionaries for the margin and the interior regions of lesions (“dual dictionaries” of visual words). Based on these dictionaries, visual word histograms are generated for each region of interest within the lesion and its margin. For validation of our approach, we used two datasets from two different institutions, containing CT images of 194 liver lesions (61 cysts, 80 metastasis, and 53 hemangiomas). The final diagnosis of each lesion was established by radiologists. The classification accuracy for the images from the two institutions was 99% and 88%, respectively, and 93% for a combined dataset. Our new BoVW approach that uses dual dictionaries shows promising results. We believe the benefits of our approach may generalize to other application domains within radiology.

Multi-phase liver lesions classification using relevant visual words based on mutual information

We present a novel method for automated diagnosis of liver lesions in multi-phase CT images. Our approach is a variant of the Bag-of-Visual-Words (BoVW) method. It improves the BoVW model by selecting the most relevant words to be used for the input representation using a mutual information based criterion. Additionally, we generate relevance maps to visualize and localize the decision of the automatic classification algorithm. We validated our algorithm on 85 multi-phase CT images of 4 categories: hemangiomas, Focal Nodular Hyper-plasia (FNH), Hepatic Cellular Carcinoma (HCC) and cholangiocarcinoma. The new algorithm suggested in this paper improves the classical BoVW method sensitivity by 7% and specificity by 3%. The shift from single-phase liver data to a multi-phase representation is shown to substantially improve classification results. Overall, the system presented reaches state-of-the-art classification results of 82.4% sensitivity and 92.7% specificity on the 4 category lesion data, a challenging clinical diagnosis task.

A network-based method for predicting gene-nutrient interactions and its application to yeast amino-acid metabolism.

Cellular metabolism is highly dependent on environmental factors, such as nutrients, toxins and drugs, genetic factors, and interactions between the two. Previous experimental and computational studies of how environmental factors affect cellular metabolism were limited to the analysis of only a small set of growth media. In this study, we present a new computational method for predicting metabolic gene-nutrient interactions (GNI) that uncovers the dependence of gene essentiality on the presence or absence of nutrients in the growth medium. The method is based on constraint-based modeling, permitting the systematic exploration of a large putative growth media space. Applying this method to predict GNIs in the amino-acid metabolism system of yeast reveals complex interdependencies between amino-acid biosynthesis pathways. The predicted GNIs also enable the reverse-prediction of growth media composition, based on gene essentiality data. These results suggest that our approach may be applied to learn about the host environment in which a microorganism is embedded given data pertaining to gene lethality, providing a means for the identification of a species natural habitat.

Lung Texture Classification Using Bag of Visual Words

Interstitial lung diseases (ILD) refer to a group of more than 150 parenchymal lung disorders. High-Resolution Computed Tomography (HRCT) is the most essential imaging modality of ILD diagnosis. Nonetheless, classification of various lung tissue patterns caused by ILD is still regarded as a challenging task. The current study focuses on the classification of five most common categories of lung tissues of ILD in HRCT images: normal, emphysema, ground glass, fibrosis and micronodules. The objective of the research is to classify an expert-given annotated region of interest (AROI) using a bag of visual words (BoVW) framework. The images are divided into small patches and a collection of representative patches are defined as visual words. This procedure, termed dictionary construction, is performed for each individual lung texture category. The assumption is that different lung textures are represented by a different visual word distribution. The classification is performed using an SVM classifier with histogram intersection kernel. In the experiments, we use a dataset of 1018 AROIs from 95 patients. Classification using a leave-one-patient-out cross validation (LOPO CV) is used. Current classification accuracy obtained is close to 80%.

Automatic Detection and Segmentation of Liver Metastatic Lesions on Serial CT Examinations

In this paper we present a fully automated method for detection and segmentation of liver metastases on serial CT examinations (portal phase) given a 2D baseline segmentation mask. Our database contains 27 CT scans, baselines and follow-ups, of 12 patients and includes 22 test cases. Our method is based on the information given in the baseline CT scan which contains the lesions segmentation mask marked manually by a radiologist. We use the 2D baseline segmentation mask to identify the lesion location in the follow-up CT scan using non-rigid image registration. The baseline CT scan is also used to locate regions of tissues surrounding the lesion and to map them onto the follow-up CT scan, in order to reduce the search area on the follow-up CT scan. Adaptive region-growing and mean-shift segmentation are used to obtain the final lesion segmentation. The segmentation results are compared to those obtained by a human radiologist. Compared to the reference standard our method made a correct RECIST 1.1 assessment for 21 out of 22 test cases. The average Dice index was 0.83 ± 0.07, average Hausdorff distance was 7.85± 4.84 mm, average sensitivity was 0.87 ± 0.11 and positive predictive value was 0.81 ± 0.10. The segmentation performance and the RECIST assessment results look promising. We are pursuing the methodology further with expansion to 3D segmentation while increasing the dataset we are collecting from the CT abdomen unit at Sheba medical center.