Zahra Karimaghaloo

Self-similarity weighted mutual information: a new nonrigid image registration metric.

Mutual information (MI) has been widely used as a similarity measure for rigid registration of multi-modal and uni-modal medical images. However, robust application of MI to deformable registration is challenging mainly because rich structural information, which are critical cues for successful deformable registration, are not incorporated into MI. We propose a self-similarity weighted graph-based implementation of α-mutual information (α-MI) for nonrigid image registration. We use a self-similarity measure that uses local structural information and is invariant to rotation and to local affine intensity distortions, and therefore the new Self Similarity α-MI (SeSaMI) metric inherits these properties and is robust against signal nonstationarity and intensity distortions. We have used SeSaMI as the similarity measure in a regularized cost function with B-spline deformation field to achieve nonrigid registration. Since the gradient of SeSaMI can be derived analytically, the cost function can be efficiently optimized using stochastic gradient descent methods. We show that SeSaMI produces a robust and smooth cost function and outperforms the state of the art statistical based similarity metrics in simulation and using data from image-guided neurosurgery.

Nonrigid Registration of Ultrasound and MRI Using Contextual Conditioned Mutual Information

Mutual information (MI) quantifies the information that is shared between two random variables and has been widely used as a similarity metric for multi-modal and uni-modal image registration. A drawback of MI is that it only takes into account the intensity values of corresponding pixels and not of neighborhoods. Therefore, it treats images as “bag of words” and the contextual information is lost. In this work, we present Contextual Conditioned Mutual Information (CoCoMI), which conditions MI estimation on similar structures. Our rationale is that it is more likely for similar structures to undergo similar intensity transformations. The contextual analysis is performed on one of the images offline. Therefore, CoCoMI does not significantly change the registration time. We use CoCoMI as the similarity measure in a regularized cost function with a B-spline deformation field and efficiently optimize the cost function using a stochastic gradient descent method. We show that compared to the state of the art local MI based similarity metrics, CoCoMI does not distort images to enforce erroneous identical intensity transformations for different image structures. We further present the results on nonrigid registration of ultrasound (US) and magnetic resonance (MR) patient data from image-guided neurosurgery trials performed in our institute and publicly available in the BITE dataset. We show that CoCoMI performs significantly better than the state of the art similarity metrics in US to MR registration. It reduces the average mTRE over 13 patients from 4.12 mm to 2.35 mm, and the maximum mTRE from 9.38 mm to 3.22 mm.

Automatic Detection of Gadolinium-Enhancing Multiple Sclerosis Lesions in Brain MRI Using Conditional Random Fields

Gadolinium-enhancing lesions in brain magnetic resonance imaging of multiple sclerosis (MS) patients are of great interest since they are markers of disease activity. Identification of gadolinium-enhancing lesions is particularly challenging because the vast majority of enhancing voxels are associated with normal structures, particularly blood vessels. Furthermore, these lesions are typically small and in close proximity to vessels. In this paper, we present an automatic, probabilistic framework for segmentation of gadolinium-enhancing lesions in MS using conditional random fields. Our approach, through the integration of different components, encodes different information such as correspondence between the intensities and tissue labels, patterns in the labels, or patterns in the intensities. The proposed algorithm is evaluated on 80 multimodal clinical datasets acquired from relapsing-remitting MS patients in the context of multicenter clinical trials. The experimental results exhibit a sensitivity of 98% with a low false positive lesion count. The performance of the proposed algorithm is also compared to a logistic regression classifier, a support vector machine and a Markov random field approach. The results demonstrate superior performance of the proposed algorithm at successfully detecting all of the gadolinium-enhancing lesions while maintaining a low false positive lesion count.

Evaluation of state-of-the-art segmentation algorithms for left ventricle infarct from late Gadolinium enhancement MR images

Studies have demonstrated the feasibility of late Gadolinium enhancement (LGE) cardiovascular magnetic resonance (CMR) imaging for guiding the management of patients with sequelae to myocardial infarction, such as ventricular tachycardia and heart failure. Clinical implementation of these developments necessitates a reproducible and reliable segmentation of the infarcted regions. It is challenging to compare new algorithms for infarct segmentation in the left ventricle (LV) with existing algorithms. Benchmarking datasets with evaluation strategies are much needed to facilitate comparison. This manuscript presents a benchmarking evaluation framework for future algorithms that segment infarct from LGE CMR of the LV. The image database consists of 30 LGE CMR images of both humans and pigs that were acquired from two separate imaging centres. A consensus ground truth was obtained for all data using maximum likelihood estimation. Six widely-used fixed-thresholding methods and five recently developed algorithms are tested on the benchmarking framework. Results demonstrate that the algorithms have better overlap with the consensus ground truth than most of the n-SD fixed-thresholding methods, with the exception of the Full-Width-at-Half-Maximum (FWHM) fixed-thresholding method. Some of the pitfalls of fixed thresholding methods are demonstrated in this work. The benchmarking evaluation framework, which is a contribution of this work, can be used to test and benchmark future algorithms that detect and quantify infarct in LGE CMR images of the LV. The datasets, ground truth and evaluation code have been made publicly available through the website: https://www.cardiacatlas.org/web/guest/challenges.

Detection of gad-enhancing lesions in multiple sclerosis using conditional random fields

Identification of Gad-enhancing lesions is of great interest in Multiple Sclerosis (MS) disease since they are associated with disease activity. Current techniques for detecting Gad-enhancing lesions use a contrast agent (Gadolinium) which is administered intravenously to highlight Gad-enhancing lesions. However, the contrast agent not only highlights these lesions, but also causes other tissues (e.g., blood vessels) or noise in the Magnetic Resonance Image (MRI) to appear hyperintense. Discrimination of enhanced lesions from other enhanced structures is particularly challenging as these lesions are typically small and can be found in close proximity to vessels. We present a new approach to address the segmentation of Gad-enhancing MS lesions using Conditional Random Fields (CRF). CRF performs the classification by simultaneously incorporating the spatial dependencies of data and labels. The performance of the CRF classifier on 20 clinical data sets shows promising results in successfully capturing all Gad-enhancing lesions. Furthermore, the quantitative results of the CRF classifier indicate a reduction in the False Positive (FP) rate by an average factor of 5.8 when comparing to Linear Discriminant Analysis (LDA) and 1.6 comparing to a Markov Random Field (MRF) classifier.

Adaptive Voxel, Texture and Temporal Conditional Random Fields for Detection of Gad-Enhancing Multiple Sclerosis Lesions in Brain MRI

The detection of Gad-enhancing lesions in brain MRI of Multiple Sclerosis (MS) patients is of great clinical interest since they are important markers of disease activity. However, many of the enhancing voxels are associated with normal structures (i.e. blood vessels) or noise in the MRI, making the detection of Gad-enhancing lesions a challenging task. Furthermore, these lesions are typically small and in close proximity to vessels. In this paper, we present a probabilistic Adaptive Multi-level Conditional Random Field (AMCRF) framework, capable of leveraging spatial and temporal information, for detection of MS Gad-enhancing lesions. In the first level, a voxel based CRF with cliques of up to size three, is used to identify candidate lesions. In the second level, higher order potentials are incorporated leveraging robust textural features which are invariant to rotation and local intensity distortions. Furthermore, we show how to exploit temporal and longitudinal images, should they be available, into the AMCRF model. The proposed algorithm is tested on 120 multimodal clinical datasets acquired from Relapsing-Remitting MS patients during multi-center clinical trials. Results show a sensitivity of 93%, a positive predictive value of 70% and average False Positive (FP) counts of 0.77. Moreover, the temporal AMCRF results show the same sensitivity as the AMCRF model while decreasing the FP counts by 22%.

Hierarchical conditional random fields for detection of gad-enhancing lesions in multiple sclerosis

The detection of gad-enhancing lesions in brain MRI of Multiple Sclerosis (MS) patients is of great interest since they are important markers of disease activity. However, many of the enhancing voxels are associated with normal structures (i.e. blood vessels) or noise in the MRI, making the detection of gad-enhancing lesions a challenging task. Furthermore, these lesions are typically small and in close proximity to vessels. In this paper, we present an automatic, probabilistic Hierarchical Conditional Random Field (HCRF) framework for detection of gad-enhancing lesions in brain images of patients with MS. In the first level, a CRF with unary and pairwise potentials is used to identify candidate lesion voxel. In the second level, these lesion candidates are grouped based on anatomical and spatial features, and feature-specific lesion based CRF models are designed for each group. This lesion level CRF incorporates higher order potentials which account for shape, group intensities and symmetries. The proposed algorithm is trained on 92 multimodal clinical datasets acquired from Relapsing-Remitting MS patients during multicenter clinical trials and is evaluated on 30 independent cases. The experimental results show a sensitivity of 98%, a positive predictive value of 66% and an average false positive count of 1.55, outperforming the CRF and MRF frameworks proposed in.

Temporal Hierarchical Adaptive Texture CRF for Automatic Detection of Gadolinium-Enhancing Multiple Sclerosis Lesions in Brain MRI

We propose a conditional random field (CRF) based classifier for segmentation of small enhanced pathologies. Specifically, we develop a temporal hierarchical adaptive texture CRF (THAT-CRF) and apply it to the challenging problem of gad enhancing lesion segmentation in brain MRI of patients with multiple sclerosis. In this context, the presence of many nonlesion enhancements (such as blood vessels) renders the problem more difficult. In addition to voxel-wise features, the framework exploits multiple higher order textures to discriminate the true lesional enhancements from the pool of other enhancements. Since lesional enhancements show more variation over time as compared to the nonlesional ones, we incorporate temporal texture analysis in order to study the textures of enhanced candidates over time. The parameters of the THAT-CRF model are learned based on 2380 scans from a multi-center clinical trial. The effect of different components of the model is extensively evaluated on 120 scans from a separate multi-center clinical trial. The incorporation of the temporal textures results in a general decrease of the false discovery rate. Specifically, THAT-CRF achieves overall sensitivity of 95% along with false discovery rate of 20% and average false positive count of 0.5 lesions per scan. The sensitivity of the temporal method to the trained time interval is further investigated on five different intervals of 69 patients. Moreover, superior performance is achieved by the reviewed labelings of our model compared to the fully manual labeling when applied to the context of separating different treatment arms in a real clinical trial.

Adaptive multi-level conditional random fields for detection and segmentation of small enhanced pathology in medical images.

Detection and segmentation of large structures in an image or within a region of interest have received great attention in the medical image processing domains. However, the problem of small pathology detection and segmentation still remains an unresolved challenge due to the small size of these pathologies, their low contrast and variable position, shape and texture. In many contexts, early detection of these pathologies is critical in diagnosis and assessing the outcome of treatment. In this paper, we propose a probabilistic Adaptive Multi-level Conditional Random Fields (AMCRF) with the incorporation of higher order cliques for detecting and segmenting such pathologies. In the first level of our graphical model, a voxel-based CRF is used to identify candidate lesions. In the second level, in order to further remove falsely detected regions, a new CRF is developed that incorporates higher order textural features, which are invariant to rotation and local intensity distortions. At this level, higher order textures are considered together with the voxel-wise cliques to refine boundaries and is therefore adaptive. The proposed algorithm is tested in the context of detecting enhancing Multiple Sclerosis (MS) lesions in brain MRI, where the problem is further complicated as many of the enhancing voxels are associated with normal structures (i.e. blood vessels) or noise in the MRI. The algorithm is trained and tested on large multi-center clinical trials from Relapsing-Remitting MS patients. The effect of several different parameter learning and inference techniques is further investigated. When tested on 120 cases, the proposed method reaches a lesion detection rate of 90%, with very few false positive lesion counts on average, ranging from 0.17 for very small (3-5 voxels) to 0 for very large (50+ voxels) regions. The proposed model is further tested on a very large clinical trial containing 2770 scans where a high sensitivity of 91% with an average false positive count of 0.5 is achieved. Incorporation of contextual information at different scales is also explored. Finally, superior performance is shown upon comparing with Support Vector Machine (SVM), Random Forest and variant of an MRF.

Lesion Detection, Segmentation and Prediction in Multiple Sclerosis Clinical Trials.

A variety of automatic segmentation techniques have been successfully applied to the delineation of larger T2 lesions in patient MRI in the context of Multiple Sclerosis (MS), assisting in the estimation of lesion volume, a common clinical measure of disease activity and stage. In the context of clinical trials, however, a wider number of metrics are required to determine the “burden of disease” and activity in order to measure treatment efficacy. These include: (1) the number and volume of T2 lesions in MRI, (2) the number of new and enlarging T2 volumes in longitudinal MRI, and (3) the number of gadolinium enhancing lesions in T1 MRI, the portion of lesions that enhance in T1w MRI after injection with a contrast agent, often associated with active inflammations. In this context, accurate lesion detection must ensure that even small lesions (e.g. 3 to 10 voxels) are detected as they are prevalent in trials. Manual or semi-manual approaches are too time-consuming, inconsistent and expensive to be practical in large clinical trials. To this end, we present a series of fully-automatic, probabilistic machine learning frameworks to detect and segment all lesions in patient MRI, and show their accuracy and robustness in large multi-center, multi-scanner, clinical trial datasets. Several of these algorithms have been placed into a commercial software analysis pipeline, where they have assisted in improving the efficiency and precision of the development of most new MS treatments worldwide. Recent work has shown how a new Bag-of-Lesions brain representation can be used in the context of clinical trials to automatically predict the probability of future disease activity and potential treatment responders, leading to the possibility of personalized medicine.