Shannon Agner

Automated gland and nuclei segmentation for grading of prostate and breast cancer histopathology

Automated detection and segmentation of nuclear and glandular structures is critical for classification and grading of prostate and breast cancer histopathology. In this paper, we present a methodology for automated detection and segmentation of structures of interest in digitized histopathology images. The scheme integrates image information from across three different scales: (1) low- level information based on pixel values, (2) high-level information based on relationships between pixels for object detection, and (3) domain-specific information based on relationships between histological structures. Low-level information is utilized by a Bayesian classifier to generate a likelihood that each pixel belongs to an object of interest. High-level information is extracted in two ways: (i) by a level-set algorithm, where a contour is evolved in the likelihood scenes generated by the Bayesian classifier to identify object boundaries, and (ii) by a template matching algorithm, where shape models are used to identify glands and nuclei from the low-level likelihood scenes. Structural constraints are imposed via domain- specific knowledge in order to verify whether the detected objects do indeed belong to structures of interest. In this paper we demonstrate the utility of our glandular and nuclear segmentation algorithm in accurate extraction of various morphological and nuclear features for automated grading of (a) prostate cancer, (b) breast cancer, and (c) distinguishing between cancerous and benign breast histology specimens. The efficacy of our segmentation algorithm is evaluated by comparing breast and prostate cancer grading and benign vs. cancer discrimination accuracies with corresponding accuracies obtained via manual detection and segmentation of glands and nuclei.

Automated grading of breast cancer histopathology using spectral clustering with textural and architectural image features

In this paper we present a novel image analysis methodology for automatically distinguishing low and high grades of breast cancer from digitized histopathology. A set of over 3,400 image features, including textural and nuclear architecture based features, are extracted from a database of 48 breast biopsy tissue studies (30 cancerous and 18 benign images). Spectral clustering is used to reduce the dimensionality of the feature set. A support vector machine (SVM) classifier is used (1) to distinguish between cancerous and non-cancerous images, and (2) to distinguish between images containing low and high grades of cancer. Classification is repeated using different subsets of features to compare their performance. The system achieves a 95.8% accuracy in distinguishing cancer from non-cancer using texture-based characteristics (Gabor filter features), and 93.3% accuracy in distinguishing high from low grades of cancer using architectural features. In addition, we investigate the underlying manifold structure on which the different grades of breast cancer lie as revealed through spectral clustering. The manifold shows a smooth spatial transition from low to high grade breast cancer.

Computerized Image-Based Detection and Grading of Lymphocytic Infiltration in HER2+ Breast Cancer Histopathology

The identification of phenotypic changes in breast cancer (BC) histopathology on account of corresponding molecular changes is of significant clinical importance in predicting disease outcome. One such example is the presence of lymphocytic infiltration (LI) in histopathology, which has been correlated with nodal metastasis and distant recurrence in HER2+ BC patients. In this paper, we present a computer-aided diagnosis (CADx) scheme to automatically detect and grade the extent of LI in digitized HER2+ BC histopathology. Lymphocytes are first automatically detected by a combination of region growing and Markov random field algorithms. Using the centers of individual detected lymphocytes as vertices, three graphs (Voronoi diagram, Delaunay triangulation, and minimum spanning tree) are constructed and a total of 50 image-derived features describing the arrangement of the lymphocytes are extracted from each sample. A nonlinear dimensionality reduction scheme, graph embedding (GE), is then used to project the high-dimensional feature vector into a reduced 3-D embedding space. A support vector machine classifier is used to discriminate samples with high and low LI in the reduced dimensional embedding space. A total of 41 HER2+ hematoxylin-and-eosin-stained images obtained from 12 patients were considered in this study. For more than 100 three-fold cross-validation trials, the architectural feature set successfully distinguished samples of high and low LI levels with a classification accuracy greater than 90%. The popular unsupervised Varma-Zisserman texton-based classification scheme was used for comparison and yielded a classification accuracy of only 60%. Additionally, the projection of the 50 image-derived features for all 41 tissue samples into a reduced dimensional space via GE allowed for the visualization of a smooth manifold that revealed a continuum between low, intermediate, and high levels of LI. Since it is known that extent of LI in BC biopsy specimens is a prognostic indicator, our CADx scheme will potentially help clinicians determine disease outcome and allow them to make better therapy recommendations for patients with HER2+ BC.

Textural Kinetics: A Novel Dynamic Contrast-Enhanced (DCE)-MRI Feature for Breast Lesion Classification

Dynamic contrast-enhanced (DCE)-magnetic resonance imaging (MRI) of the breast has emerged as an adjunct imaging tool to conventional X-ray mammography due to its high detection sensitivity. Despite the increasing use of breast DCE-MRI, specificity in distinguishing malignant from benign breast lesions is low, and interobserver variability in lesion classification is high. The novel contribution of this paper is in the definition of a new DCE-MRI descriptor that we call textural kinetics, which attempts to capture spatiotemporal changes in breast lesion texture in order to distinguish malignant from benign lesions. We qualitatively and quantitatively demonstrated on 41 breast DCE-MRI studies that textural kinetic features outperform signal intensity kinetics and lesion morphology features in distinguishing benign from malignant lesions. A probabilistic boosting tree (PBT) classifier in conjunction with textural kinetic descriptors yielded an accuracy of 90%, sensitivity of 95%, specificity of 82%, and an area under the curve (AUC) of 0.92. Graph embedding, used for qualitative visualization of a low-dimensional representation of the data, showed the best separation between benign and malignant lesions when using textural kinetic features. The PBT classifier results and trends were also corroborated via a support vector machine classifier which showed that textural kinetic features outperformed the morphological, static texture, and signal intensity kinetics descriptors. When textural kinetic attributes were combined with morphologic descriptors, the resulting PBT classifier yielded 89% accuracy, 99% sensitivity, 76% specificity, and an AUC of 0.91.

Computer-aided prognosis: Predicting patient and disease outcome via quantitative fusion of multi-scale, multi-modal data

Computer-aided prognosis (CAP) is a new and exciting complement to the field of computer-aided diagnosis (CAD) and involves developing and applying computerized image analysis and multi-modal data fusion algorithms to digitized patient data (e.g. imaging, tissue, genomic) for helping physicians predict disease outcome and patient survival. While a number of data channels, ranging from the macro (e.g. MRI) to the nano-scales (proteins, genes) are now being routinely acquired for disease characterization, one of the challenges in predicting patient outcome and treatment response has been in our inability to quantitatively fuse these disparate, heterogeneous data sources. At the Laboratory for Computational Imaging and Bioinformatics (LCIB)(1) at Rutgers University, our team has been developing computerized algorithms for high dimensional data and image analysis for predicting disease outcome from multiple modalities including MRI, digital pathology, and protein expression. Additionally, we have been developing novel data fusion algorithms based on non-linear dimensionality reduction methods (such as Graph Embedding) to quantitatively integrate information from multiple data sources and modalities with the overarching goal of optimizing meta-classifiers for making prognostic predictions. In this paper, we briefly describe 4 representative and ongoing CAP projects at LCIB. These projects include (1) an Image-based Risk Score (IbRiS) algorithm for predicting outcome of Estrogen receptor positive breast cancer patients based on quantitative image analysis of digitized breast cancer biopsy specimens alone, (2) segmenting and determining extent of lymphocytic infiltration (identified as a possible prognostic marker for outcome in human epidermal growth factor amplified breast cancers) from digitized histopathology, (3) distinguishing patients with different Gleason grades of prostate cancer (grade being known to be correlated to outcome) from digitized needle biopsy specimens, and (4) integrating protein expression measurements obtained from mass spectrometry with quantitative image features derived from digitized histopathology for distinguishing between prostate cancer patients at low and high risk of disease recurrence following radical prostatectomy.

Computerized Image Analysis for Identifying Triple-Negative Breast Cancers and Differentiating Them from Other Molecular Subtypes of Breast Cancer on Dynamic Contrast-enhanced MR Images: A Feasibility Study

PURPOSE To determine the feasibility of using a computer-aided diagnosis (CAD) system to differentiate among triple-negative breast cancer, estrogen receptor (ER)-positive cancer, human epidermal growth factor receptor type 2 (HER2)-positive cancer, and benign fibroadenoma lesions on dynamic contrast material-enhanced (DCE) magnetic resonance (MR) images. MATERIALS AND METHODS This is a retrospective study of prospectively acquired breast MR imaging data collected from an institutional review board-approved, HIPAA-compliant study between 2002 and 2007. Written informed consent was obtained from all patients. The authors collected DCE MR images from 65 women with 76 breast lesions who had been recruited into a larger study of breast MR imaging. The women had triple-negative (n = 21), ER-positive (n = 25), HER2-positive (n = 18), or fibroadenoma (n = 12) lesions. All lesions were classified as Breast Imaging Reporting and Data System category 4 or higher on the basis of previous imaging. Images were subject to quantitative feature extraction, feed-forward feature selection by means of linear discriminant analysis, and lesion classification by using a support vector machine classifier. The area under the receiver operating characteristic curve (Az) was calculated for each of five lesion classification tasks involving triple-negative breast cancers. RESULTS For each pair-wise lesion type comparison, linear discriminant analysis helped identify the most discriminatory features, which in conjunction with a support vector machine classifier yielded an Az of 0.73 (95% confidence interval [CI]: 0.59, 0.87) for triple-negative cancer versus all non-triple-negative lesions, 0.74 (95% CI: 0.60, 0.88) for triple-negative cancer versus ER- and HER2-positive cancer, 0.77 (95% CI: 0.63, 0.91) for triple-negative versus ER-positive cancer, 0.74 (95% CI: 0.58, 0.89) for triple-negative versus HER2-positive cancer, and 0.97 (95% CI: 0.91, 1.00) for triple-negative cancer versus fibroadenoma. CONCLUSION Triple-negative cancers possess certain characteristic features on DCE MR images that can be captured and quantified with CAD, enabling good discrimination of triple-negative cancers from non-triple-negative cancers, as well as between triple-negative cancers and benign fibroadenomas. Such CAD algorithms may provide added diagnostic benefit in identifying the highly aggressive triple-negative cancer phenotype with DCE MR imaging in high-risk women.

Spectral embedding based active contour (SEAC) for lesion segmentation on breast dynamic contrast enhanced magnetic resonance imaging

PURPOSE Segmentation of breast lesions on dynamic contrast enhanced (DCE) magnetic resonance imaging (MRI) is the first step in lesion diagnosis in a computer-aided diagnosis framework. Because manual segmentation of such lesions is both time consuming and highly susceptible to human error and issues of reproducibility, an automated lesion segmentation method is highly desirable. Traditional automated image segmentation methods such as boundary-based active contour (AC) models require a strong gradient at the lesion boundary. Even when region-based terms are introduced to an AC model, grayscale image intensities often do not allow for clear definition of foreground and background region statistics. Thus, there is a need to find alternative image representations that might provide (1) strong gradients at the margin of the object of interest (OOI); and (2) larger separation between intensity distributions and region statistics for the foreground and background, which are necessary to halt evolution of the AC model upon reaching the border of the OOI. METHODS In this paper, the authors introduce a spectral embedding (SE) based AC (SEAC) for lesion segmentation on breast DCE-MRI. SE, a nonlinear dimensionality reduction scheme, is applied to the DCE time series in a voxelwise fashion to reduce several time point images to a single parametric image where every voxel is characterized by the three dominant eigenvectors. This parametric eigenvector image (PrEIm) representation allows for better capture of image region statistics and stronger gradients for use with a hybrid AC model, which is driven by both boundary and region information. They compare SEAC to ACs that employ fuzzy c-means (FCM) and principal component analysis (PCA) as alternative image representations. Segmentation performance was evaluated by boundary and region metrics as well as comparing lesion classification using morphological features from SEAC, PCA+AC, and FCM+AC. RESULTS On a cohort of 50 breast DCE-MRI studies, PrEIm yielded overall better region and boundary-based statistics compared to the original DCE-MR image, FCM, and PCA based image representations. Additionally, SEAC outperformed a hybrid AC applied to both PCA and FCM image representations. Mean dice similarity coefficient (DSC) for SEAC was significantly better (DSC = 0.74 ± 0.21) than FCM+AC (DSC = 0.50 ± 0.32) and similar to PCA+AC (DSC = 0.73 ± 0.22). Boundary-based metrics of mean absolute difference and Hausdorff distance followed the same trends. Of the automated segmentation methods, breast lesion classification based on morphologic features derived from SEAC segmentation using a support vector machine classifier also performed better (AUC = 0.67 ± 0.05; p < 0.05) than FCM+AC (AUC = 0.50 ± 0.07), and PCA+AC (AUC = 0.49 ± 0.07). CONCLUSIONS In this work, we presented SEAC, an accurate, general purpose AC segmentation tool that could be applied to any imaging domain that employs time series data. SE allows for projection of time series data into a PrEIm representation so that every voxel is characterized by the dominant eigenvectors, capturing the global and local time-intensity curve similarities in the data. This PrEIm allows for the calculation of strong tensor gradients and better region statistics than the original image intensities or alternative image representations such as PCA and FCM. The PrEIm also allows for building a more accurate hybrid AC scheme.

Segmentation and classification of triple negative breast cancers using DCE-MRI

Triple-negative (TN) breast cancer has gained much interest recently due to its lack of response to receptor-targeted therapies and its aggressive clinical nature. In this study, we evaluate the ability of a computer-aided diagnosis (CAD) system to not only distinguish benign from malignant lesions on dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI), but also to quantitatively distinguish triple negative breast cancers from other molecular subtypes of breast cancer. 41 breast lesions (24 malignant, 17 benign) as imaged on DCE-MRI were included in the dataset. Of the 24 malignant cases, 13 were of the TN phenotype. Using the dynamic signal intensity information from the DCE-MRIs, an Expectation Maximization-driven active contours scheme is used to automatically segment the breast lesions. Following quantitative morphological, textural, and kinetic feature extraction, a support vector machine classifier was employed to distinguish (a) benign from malignant lesions and (b) TN from non-TN cancers. In the former case, the classifier yielded an accuracy of 83%, sensitivity of 79%, and specificity of 88%. In distinguishing TN from non-TN cases, the classifier had an accuracy of 92%, sensitivity of 92%, and specificity of 91%. The results suggest that the TN phenotype has distinct and quantifiable signatures on DCE-MRI that will be instrumental in the early detection of this aggressive breast cancer subtype.

Computer-aided prognosis: Predicting patient and disease outcome via multi-modal image analysis

Computer-aided prognosis (CAP) is a new and exciting complement to the field of computer-aided diagnosis (CAD) and involves developing computerized image analysis and multi-modal data fusion algorithms for helping physicians predict disease outcome and patient survival. At the Laboratory for Computational Imaging and Bioinformatics (LCIB)1 at Rutgers University we have been developing computerized algorithms for high dimensional data and image analysis for predicting disease outcome from multiple modalities includng MRI, digital pathology, and protein expression. Additionally, we have been developing novel data fusion algorithms based on nonlinear dimensionality reduction methods (such as Graph Embedding) to quantitatively integrate prognostic information from multiple data sources and modalities. In this paper, we briefly describe 5 representative and ongoing CAP projects at LCIB. These projects include (1) an Image-based Risk Score (IbRiS) algorithm for predicting outcome of ER+ breast cancer patients based on quantitative image analysis of digitized breast cancer biopsy specimens alone, (2) segmenting and determining extent of lymphocytic infiltration (identified as a possible prognostic marker for outcome in Her2+ breast cancers) from digitized histopathology, (3) segmenting and diagnosing highly agressive triple-negative breast cancers on dynamic contrast enhanced (DCE) MRI, (4) distinguishing patients with different Gleason grades of prostate cancer (grade being known to be correlated to outcome) from digitzed needle biopsy specimens, and (5) integrating protein expression measurements obtained from mass spectrometry with quantitative image features derived from digitized histopathology for distinguishing between prostate cancer patients at low and high risk of disease recurrence.

Spectral Embedding based Active Contour (SEAC): Application to Breast Lesion Segmentation on DCE-MRI

Spectral embedding (SE), a graph-based manifold learning method, has previously been shown to be useful in high dimensional data classification. In this work, we present a novel SE based active contour (SEAC) segmentation scheme and demonstrate its applications in lesion segmentation on breast dynamic contrast enhance magnetic resonance imaging (DCE-MRI). In this work, we employ SE on DCE-MRI on a per voxel basis to embed the high dimensional time series intensity vector into a reduced dimensional space, where the reduced embedding space is characterized by the principal eigenvectors. The orthogonal eigenvector-based data representation allows for computation of strong tensor gradients in the spectrally embedded space and also yields improved region statistics that serve as optimal stopping criteria for SEAC. We demonstrate both analytically and empirically that the tensor gradients in the spectrally embedded space are stronger than the corresponding gradients in the original grayscale intensity space. On a total of 50 breast DCE-MRI studies, SEAC yielded a mean absolute difference (MAD) of 3.2±2.1 pixels and mean Dice similarity coefficient (DSC) of 0.74±0.13 compared to manual ground truth segmentation. An active contour in conjunction with fuzzy c-means (FCM+AC), a commonly used segmentation method for breast DCE-MRI, produced a corresponding MAD of 7.2±7.4 pixels and mean DSC of 0.58±0.32. In conjunction with a set of 6 quantitative morphological features automatically extracted from the SEAC derived lesion boundary, a support vector machine (SVM) classifier yielded an area under the curve (AUC) of 0.73, for discriminating between 10 benign and 30 malignant lesions; the corresponding SVM classifier with the FCM+AC derived morphological features yielded an AUC of 0.65.