Ajay Basavanhally

Computerized Image-Based Detection and Grading of Lymphocytic Infiltration in HER2+ Breast Cancer Histopathology

The identification of phenotypic changes in breast cancer (BC) histopathology on account of corresponding molecular changes is of significant clinical importance in predicting disease outcome. One such example is the presence of lymphocytic infiltration (LI) in histopathology, which has been correlated with nodal metastasis and distant recurrence in HER2+ BC patients. In this paper, we present a computer-aided diagnosis (CADx) scheme to automatically detect and grade the extent of LI in digitized HER2+ BC histopathology. Lymphocytes are first automatically detected by a combination of region growing and Markov random field algorithms. Using the centers of individual detected lymphocytes as vertices, three graphs (Voronoi diagram, Delaunay triangulation, and minimum spanning tree) are constructed and a total of 50 image-derived features describing the arrangement of the lymphocytes are extracted from each sample. A nonlinear dimensionality reduction scheme, graph embedding (GE), is then used to project the high-dimensional feature vector into a reduced 3-D embedding space. A support vector machine classifier is used to discriminate samples with high and low LI in the reduced dimensional embedding space. A total of 41 HER2+ hematoxylin-and-eosin-stained images obtained from 12 patients were considered in this study. For more than 100 three-fold cross-validation trials, the architectural feature set successfully distinguished samples of high and low LI levels with a classification accuracy greater than 90%. The popular unsupervised Varma-Zisserman texton-based classification scheme was used for comparison and yielded a classification accuracy of only 60%. Additionally, the projection of the 50 image-derived features for all 41 tissue samples into a reduced dimensional space via GE allowed for the visualization of a smooth manifold that revealed a continuum between low, intermediate, and high levels of LI. Since it is known that extent of LI in BC biopsy specimens is a prognostic indicator, our CADx scheme will potentially help clinicians determine disease outcome and allow them to make better therapy recommendations for patients with HER2+ BC.

Expectation–Maximization-Driven Geodesic Active Contour With Overlap Resolution (EMaGACOR): Application to Lymphocyte Segmentation on Breast Cancer Histopathology

The presence of lymphocytic infiltration (LI) has been correlated with nodal metastasis and tumor recurrence in HER2+ breast cancer (BC). The ability to automatically detect and quantify extent of LI on histopathology imagery could potentially result in the development of an image based prognostic tool for human epidermal growth factor receptor-2 (HER2+) BC patients. Lymphocyte segmentation in hematoxylin and eosin (H&E) stained BC histopathology images is complicated by the similarity in appearance between lymphocyte nuclei and other structures (e.g., cancer nuclei) in the image. Additional challenges include biological variability, histological artifacts, and high prevalence of overlapping objects. Although active contours are widely employed in image segmentation, they are limited in their ability to segment overlapping objects and are sensitive to initialization. In this paper, we present a new segmentation scheme, expectation-maximization (EM) driven geodesic active contour with overlap resolution (EMaGACOR), which we apply to automatically detecting and segmenting lymphocytes on HER2+ BC histopathology images. EMaGACOR utilizes the expectation-maximization algorithm for automatically initializing a geodesic active contour (GAC) and includes a novel scheme based on heuristic splitting of contours via identification of high concavity points for resolving overlapping structures. EMaGACOR was evaluated on a total of 100 HER2+ breast biopsy histology images and was found to have a detection sensitivity of over 86% and a positive predictive value of over 64%. By comparison, the EMaGAC model (without overlap resolution) and GAC model yielded corresponding detection sensitivities of 42% and 19%, respectively. Furthermore, EMaGACOR was able to correctly resolve over 90% of overlaps between intersecting lymphocytes. Hausdorff distance (HD) and mean absolute distance (MAD) for EMaGACOR were found to be 2.1 and 0.9 pixels, respectively, and significantly better compared to the corresponding performance of the EMaGAC and GAC models. EMaGACOR is an efficient, robust, reproducible, and accurate segmentation technique that could potentially be applied to other biomedical image analysis problems.

Computer-aided prognosis: Predicting patient and disease outcome via quantitative fusion of multi-scale, multi-modal data

Computer-aided prognosis (CAP) is a new and exciting complement to the field of computer-aided diagnosis (CAD) and involves developing and applying computerized image analysis and multi-modal data fusion algorithms to digitized patient data (e.g. imaging, tissue, genomic) for helping physicians predict disease outcome and patient survival. While a number of data channels, ranging from the macro (e.g. MRI) to the nano-scales (proteins, genes) are now being routinely acquired for disease characterization, one of the challenges in predicting patient outcome and treatment response has been in our inability to quantitatively fuse these disparate, heterogeneous data sources. At the Laboratory for Computational Imaging and Bioinformatics (LCIB)(1) at Rutgers University, our team has been developing computerized algorithms for high dimensional data and image analysis for predicting disease outcome from multiple modalities including MRI, digital pathology, and protein expression. Additionally, we have been developing novel data fusion algorithms based on non-linear dimensionality reduction methods (such as Graph Embedding) to quantitatively integrate information from multiple data sources and modalities with the overarching goal of optimizing meta-classifiers for making prognostic predictions. In this paper, we briefly describe 4 representative and ongoing CAP projects at LCIB. These projects include (1) an Image-based Risk Score (IbRiS) algorithm for predicting outcome of Estrogen receptor positive breast cancer patients based on quantitative image analysis of digitized breast cancer biopsy specimens alone, (2) segmenting and determining extent of lymphocytic infiltration (identified as a possible prognostic marker for outcome in human epidermal growth factor amplified breast cancers) from digitized histopathology, (3) distinguishing patients with different Gleason grades of prostate cancer (grade being known to be correlated to outcome) from digitized needle biopsy specimens, and (4) integrating protein expression measurements obtained from mass spectrometry with quantitative image features derived from digitized histopathology for distinguishing between prostate cancer patients at low and high risk of disease recurrence following radical prostatectomy.

Multi-Field-of-View Framework for Distinguishing Tumor Grade in ER+ Breast Cancer From Entire Histopathology Slides

Modified Bloom-Richardson (mBR) grading is known to have prognostic value in breast cancer (BCa), yet its use in clinical practice has been limited by intra- and interobserver variability. The development of a computerized system to distinguish mBR grade from entire estrogen receptor-positive (ER+) BCa histopathology slides will help clinicians identify grading discrepancies and improve overall confidence in the diagnostic result. In this paper, we isolate salient image features characterizing tumor morphology and texture to differentiate entire hematoxylin and eosin (H and E) stained histopathology slides based on mBR grade. The features are used in conjunction with a novel multifield-of-view (multi-FOV) classifier-a whole-slide classifier that extracts features from a multitude of FOVs of varying sizes-to identify important image features at different FOV sizes. Image features utilized include those related to the spatial arrangement of cancer nuclei (i.e., nuclear architecture) and the textural patterns within nuclei (i.e., nuclear texture). Using slides from 126 ER+ patients (46 low, 60 intermediate, and 20 high mBR grade), our grading system was able to distinguish low versus high, low versus intermediate, and intermediate versus high grade patients with area under curve values of 0.93, 0.72, and 0.74, respectively. Our results suggest that the multi-FOV classifier is able to 1) successfully discriminate low, medium, and high mBR grade and 2) identify specific image features at different FOV sizes that are important for distinguishing mBR grade in Hand E stained ER+ BCa histology slides.

Multi-field-of-view strategy for image-based outcome prediction of multi-parametric estrogen receptor-positive breast cancer histopathology: Comparison to Oncotype DX

In this paper, we attempt to quantify the prognostic information embedded in multi-parametric histologic biopsy images to predict disease aggressiveness in estrogen receptor-positive (ER+) breast cancers (BCa). The novel methodological contribution is in the use of a multi-field-of-view (multi-FOV) framework for integrating image-based information from differently stained histopathology slides. The multi-FOV approach involves a fixed image resolution while simultaneously integrating image descriptors from many FOVs corresponding to different sizes. For each study, the corresponding risk score (high scores reflecting aggressive disease and vice versa), predicted by a molecular assay (Oncotype DX), is available and serves as the surrogate ground truth for long-term patient outcome. Using the risk scores, a trained classifier is used to identify disease aggressiveness for each FOV size. The predictions for each FOV are then combined to yield the final prediction of disease aggressiveness (good, intermediate, or poor outcome). Independent multi-FOV classifiers are constructed for (1) 50 image features describing the spatial arrangement of cancer nuclei (via Voronoi diagram, Delaunay triangulation, and minimum spanning tree graphs) in H and E stained histopathology and (2) one image feature describing the vascular density in CD34 IHC stained histopathology. In a cohort of 29 patients, the multi-FOV classifiers obtained by combining information from the H and E and CD34 IHC stained channels were able to distinguish low- and high-risk patients with an accuracy of 0.91 ± 0.02 and a positive predictive value of 0.94 ± 0.10, suggesting that a purely image-based assay could potentially replace more expensive molecular assays for making disease prognostic predictions.

Computer-aided prognosis of ER+ breast cancer histopathology and correlating survival outcome with Oncotype DX assay

The current gold standard for predicting disease survival and outcome for lymph node-negative, estrogen receptor-positive breast cancer (LN-, ER+ BC) patients is via the gene-expression based assay, Oncotype DX. In this paper, we present a novel computer-aided prognosis (CAP) scheme that employs quantitatively derived image information to predict patient outcome analogous to the Oncotype DX Recurrence Score (RS), with high RS implying poor outcome and vice versa. While digital pathology has made tissue specimens amenable to computer-aided diagnosis (CAD) for disease detection, our CAP scheme is the first of its kind for predicting disease outcome and patient survival. Since cancer grade is known to be correlated to disease outcome, low grade implying good outcome and vice versa, our CAP scheme captures quantitative image features that are reflective of BC grade. Our scheme involves first semi-automatically detecting BC nuclei via an Expectation Maximization driven algorithm. Using the nuclear centroids, two graphs (Delaunay Triangulation and Minimum Spanning Tree) are constructed and a total of 12 features are extracted from each image. A non-linear dimensionality reduction scheme, Graph Embedding, projects the image-derived features into a low-dimensional space, and a Support Vector Machine classifies the BC images in the reduced dimensional space. On a cohort of 37 samples, and for 100 trials of 3-fold randomized cross-validation, the SVM yielded a mean accuracy of 84.15% in distinguishing samples with low and high RS and 84.12% in distinguishing low and high grade BC. The projection of the high-dimensional image feature data to a 1D line for all BC samples via GE shows a clear separation between, low, intermediate, and high BC grades, which in turn shows high correlation with low, medium, and high RS. The results suggest that our image-based CAP scheme might provide a cheaper alternative to Oncotype DX in predicting BC outcome.

Integrated diagnostics: a conceptual framework with examples

Abstract With the advent of digital pathology, imaging scientists have begun to develop computerized image analysis algorithms for making diagnostic (disease presence), prognostic (outcome prediction), and theragnostic (choice of therapy) predictions from high resolution images of digitized histopathology. One of the caveats to developing image analysis algorithms for digitized histopathology is the ability to deal with highly dense, information rich datasets; datasets that would overwhelm most computer vision and image processing algorithms. Over the last decade, manifold learning and non-linear dimensionality reduction schemes have emerged as popular and powerful machine learning tools for pattern recognition problems. However, these techniques have thus far been applied primarily to classification and analysis of computer vision problems (e.g., face detection). In this paper, we discuss recent work by a few groups in the application of manifold learning methods to problems in computer aided diagnosis, prognosis, and theragnosis of digitized histopathology. In addition, we discuss some exciting recent developments in the application of these methods for multi-modal data fusion and classification; specifically the building of meta-classifiers by fusion of histological image and proteomic signatures for prostate cancer outcome prediction. Clin Chem Lab Med 2010;48:989–98.

A boosted distance metric: application to content based image retrieval and classification of digitized histopathology

Distance metrics are often used as a way to compare the similarity of two objects, each represented by a set of features in high-dimensional space. The Euclidean metric is a popular distance metric, employed for a variety of applications. Non-Euclidean distance metrics have also been proposed, and the choice of distance metric for any specific application or domain is a non-trivial task. Furthermore, most distance metrics treat each dimension or object feature as having the same relative importance in determining object similarity. In many applications, such as in Content-Based Image Retrieval (CBIR), where images are quantified and then compared according to their image content, it may be beneficial to utilize a similarity metric where features are weighted according to their ability to distinguish between object classes. In the CBIR paradigm, every image is represented as a vector of quantitative feature values derived from the image content, and a similarity measure is applied to determine which of the database images is most similar to the query. In this work, we present a boosted distance metric (BDM), where individual features are weighted according to their discriminatory power, and compare the performance of this metric to 9 other traditional distance metrics in a CBIR system for digital histopathology. We apply our system to three different breast tissue histology cohorts - (1) 54 breast histology studies corresponding to benign and cancerous images, (2) 36 breast cancer studies corresponding to low and high Bloom-Richardson (BR) grades, and (3) 41 breast cancer studies with high and low levels of lymphocytic infiltration. Over all 3 data cohorts, the BDM performs better compared to 9 traditional metrics, with a greater area under the precision-recall curve. In addition, we performed SVM classification using the BDM along with the traditional metrics, and found that the boosted metric achieves a higher classification accuracy (over 96%) in distinguishing between the tissue classes in each of 3 data cohorts considered. The 10 different similarity metrics were also used to generate similarity matrices between all samples in each of the 3 cohorts. For each cohort, each of the 10 similarity matrices were subjected to normalized cuts, resulting in a reduced dimensional representation of the data samples. The BDM resulted in the best discrimination between tissue classes in the reduced embedding space.

Towards improved cancer diagnosis and prognosis using analysis of gene expression data and computer aided imaging

With the increasing cost effectiveness of whole slide digital scanners, gene expression microarray and SNP technologies, tissue specimens can now be analyzed using sophisticated computer aided image and data analysis techniques for accurate diagnoses and identification of prognostic markers and potential targets for therapeutic intervention. Microarray analysis is routinely able to identify biomarkers correlated with survival and reveal pathways underlying pathogenesis and invasion. In this paper we describe how microarray profiling of tumor samples combined with simple but powerful methods of analysis can identify biologically distinct disease subclasses of breast cancer with distinct molecular signatures, differential recurrence rates and potentially, very different response to therapy. Image analysis methods are also rapidly finding application in the clinic, complementing the pathologist in quantitative, reproducible, detection, staging, and grading of disease. We will describe novel computerized image analysis techniques and machine learning tools for automated cancer detection from digitized histopathology and how they can be employed for disease diagnosis and prognosis for prostate and breast cancer.

EM-based segmentation-driven color standardization of digitized histopathology

The development of tools for the processing of color images is often complicated by nonstandardness – the notion that different image regions corresponding to the same tissue will occupy different ranges in the color spectrum. In digital pathology (DP), these issues are often caused by variations in slide thickness, staining, scanning parameters, and illumination. Nonstandardness can be addressed via standardization, a pre-processing step that aims to improve color constancy by realigning color distributions of images to match that of a predefined template image. Unlike color normalization methods, which aim to scale (usually linearly or assuming that the transfer function of the system is known) the intensity of individual images, standardization is employed to align distributions in broad tissue classes (e.g. epithelium, stroma) across different DP images irrespective of institution, protocol, or scanner. Intensity standardization has previously been used for addressing the issue of intensity drift in MRI images, where similar tissue regions have different image intensities across scanners and patients. However, this approach is a global standardization (GS) method that aligns histograms of entire images at once. By contrast, histopathological imagery is complicated by the (a) additional information present in color images and (b) heterogeneity of tissue composition. In this paper, we present a novel color Expectation Maximization (EM) based standardization (EMS) scheme to decompose histological images into independent tissue classes (e.g. nuclei, epithelium, stroma, lumen) via the EM algorithm and align the color distributions for each class independently. Experiments are performed on prostate and oropharyngeal histopathology tissues from 19 and 26 patients, respectively. Evaluation methods include (a) a segmentation-based assessment of color consistency in which normalized median intensity (NMI) is calculated from segmented regions across a dataset and (b) a quantitative measure of histogram alignment via mean landmark distance. EMS produces lower NMI standard deviations (i.e. greater consistency) of 0.0054 and 0.0034 for prostate and oropharyngeal cohorts, respectively, than unstandardized (0.034 and 0.026) and GS (0.031 and 0.017) approaches. Similarly, we see decreased mean landmark distance for EMS (2.25 and 4.20) compared to both unstandardized (54.8 and 27.3) and GS (27.1 and 8.8) images. These results suggest that the separation of broad tissue classes in EMS is vital to the standardization of DP imagery and subsequent development of computerized image analysis tools.