Sahirzeeshan Ali

An Integrated Region-, Boundary-, Shape-Based Active Contour for Multiple Object Overlap Resolution in Histological Imagery

Active contours and active shape models (ASM) have been widely employed in image segmentation. A major limitation of active contours, however, is in their 1) inability to resolve boundaries of intersecting objects and to 2) handle occlusion. Multiple overlapping objects are typically segmented out as a single object. On the other hand, ASMs are limited by point correspondence issues since object landmarks need to be identified across multiple objects for initial object alignment. ASMs are also are constrained in that they can usually only segment a single object in an image. In this paper, we present a novel synergistic boundary and region-based active contour model that incorporates shape priors in a level set formulation with automated initialization based on watershed. We demonstrate an application of these synergistic active contour models using multiple level sets to segment nuclear and glandular structures on digitized histopathology images of breast and prostate biopsy specimens. Unlike previous related approaches, our model is able to resolve object overlap and separate occluded boundaries of multiple objects simultaneously. The energy functional of the active contour is comprised of three terms. The first term is the prior shape term, modeled on the object of interest, thereby constraining the deformation achievable by the active contour. The second term, a boundary-based term detects object boundaries from image gradients. The third term drives the shape prior and the contour towards the object boundary based on region statistics. The results of qualitative and quantitative evaluation on 100 prostate and 14 breast cancer histology images for the task of detecting and segmenting nuclei and lymphocytes reveals that the model easily outperforms two state of the art segmentation schemes (geodesic active contour and Rousson shape-based model) and on average is able to resolve up to 91% of overlapping/occluded structures in the images.

A quantitative histomorphometric classifier (QuHbIC) identifies aggressive versus indolent p16-positive oropharyngeal squamous cell carcinoma.

Human papillomavirus–related (p16-positive) oropharyngeal squamous cell carcinoma patients develop recurrent disease, mostly distant metastasis, in approximately 10% of cases, and the remaining patients, despite cure, can have major morbidity from treatment. Identifying patients with aggressive versus indolent tumors is critical. Hematoxylin and eosin-stained slides of a microarray cohort of p16-positive oropharyngeal squamous cell carcinoma cases were digitally scanned. A novel cluster cell graph was constructed using the nuclei as vertices to characterize and measure spatial distribution and cell clustering. A series of topological features defined on each node of the subgraph were analyzed, and a random forest decision tree classifier was developed. The classifier (QuHbIC) was validated over 25 runs of 3-fold cross-validation using case subsets for independent training and testing. Nineteen (11.9%) of the 160 patients on the array developed recurrence. QuHbIC correctly predicted outcomes in 140 patients (87.5% accuracy). There were 23 positive patients, of whom 11 developed recurrence (47.8% positive predictive value), and 137 negative patients, of whom only 8 developed recurrence (94.2% negative predictive value). The best other predictive features were stage T4 (18 patients; 83.1% accuracy) and N3 nodal disease (10 patients; 88.6% accuracy). QuHbIC-positive patients had poorer overall, disease-free, and disease-specific survival (P<0.001 for each). In multivariate analysis, QuHbIC-positive patients still showed significantly poorer disease-free and disease-specific survival, independent of all other variables. In summary, using just tiny hematoxylin and eosin punches, a computer-aided histomorphometric classifier (QuHbIC) can strongly predict recurrence risk. With prospective validation, this testing may be useful to stratify patients into different treatment groups.

Co-Occurring Gland Angularity in Localized Subgraphs: Predicting Biochemical Recurrence in Intermediate-Risk Prostate Cancer Patients

Quantitative histomorphometry (QH) refers to the application of advanced computational image analysis to reproducibly describe disease appearance on digitized histopathology images. QH thus could serve as an important complementary tool for pathologists in interrogating and interpreting cancer morphology and malignancy. In the US, annually, over 60,000 prostate cancer patients undergo radical prostatectomy treatment. Around 10,000 of these men experience biochemical recurrence within 5 years of surgery, a marker for local or distant disease recurrence. The ability to predict the risk of biochemical recurrence soon after surgery could allow for adjuvant therapies to be prescribed as necessary to improve long term treatment outcomes. The underlying hypothesis with our approach, co-occurring gland angularity (CGA), is that in benign or less aggressive prostate cancer, gland orientations within local neighborhoods are similar to each other but are more chaotically arranged in aggressive disease. By modeling the extent of the disorder, we can differentiate surgically removed prostate tissue sections from (a) benign and malignant regions and (b) more and less aggressive prostate cancer. For a cohort of 40 intermediate-risk (mostly Gleason sum 7) surgically cured prostate cancer patients where half suffered biochemical recurrence, the CGA features were able to predict biochemical recurrence with 73% accuracy. Additionally, for 80 regions of interest chosen from the 40 studies, corresponding to both normal and cancerous cases, the CGA features yielded a 99% accuracy. CGAs were shown to be statistically signicantly () better at predicting BCR compared to state-of-the-art QH methods and postoperative prostate cancer nomograms.

Cell Orientation Entropy (COrE): Predicting Biochemical Recurrence from Prostate Cancer Tissue Microarrays

We introduce a novel feature descriptor to describe cancer cells called Cell Orientation Entropy (COrE). The main objective of this work is to employ COrE to quantitatively model disorder of cell/nuclear orientation within local neighborhoods and evaluate whether these measurements of directional disorder are correlated with biochemical recurrence (BCR) in prostate cancer (CaP) patients. COrE has a number of novel attributes that are unique to digital pathology image analysis. Firstly, it is the first rigorous attempt to quantitatively model cell/nuclear orientation. Secondly, it provides for modeling of local cell networks via construction of subgraphs. Thirdly, it allows for quantifying the disorder in local cell orientation via second order statistical features. We evaluated the ability of 39 COrE features to capture the characteristics of cell orientation in CaP tissue microarray (TMA) images in order to predict 10 year BCR in men with CaP following radical prostatectomy. Randomized 3-fold cross-validation via a random forest classifier evaluated on a combination of COrE and other nuclear features achieved an accuracy of 82.7 +/- 3.1% on a dataset of 19 BCR and 20 non-recurrence patients. Our results suggest that COrE features could be extended to characterize disease states in other histological cancer images in addition to prostate cancer.

Adaptive energy selective active contour with shape priors for nuclear segmentation and gleason grading of prostate cancer

Shape based active contours have emerged as a natural solution to overlap resolution. However, most of these shape-based methods are computationally expensive. There are instances in an image where no overlapping objects are present and applying these schemes results in significant computational overhead without any accompanying, additional benefit. In this paper we present a novel adaptive active contour scheme (AdACM) that combines boundary and region based energy terms with a shape prior in a multi level set formulation. To reduce the computational overhead, the shape prior term in the variational formulation is only invoked for those instances in the image where overlaps between objects are identified; these overlaps being identified via a contour concavity detection scheme. By not having to invoke all 3 terms (shape, boundary, region) for segmenting every object in the scene, the computational expense of the integrated active contour model is dramatically reduced, a particularly relevant consideration when multiple objects have to be segmented on very large histopathological images. The AdACM was employed for the task of segmenting nuclei on 80 prostate cancer tissue microarray images. Morphological features extracted from these segmentations were found to able to discriminate different Gleason grade patterns with a classification accuracy of 84% via a Support Vector Machine classifier. On average the AdACM model provided 100% savings in computational times compared to a non-optimized hybrid AC model involving a shape prior.

Selective invocation of shape priors for deformable segmentation and morphologic classification of prostate cancer tissue microarrays

Shape based active contours have emerged as a natural solution to overlap resolution. However, most of these shape-based methods are computationally expensive. There are instances in an image where no overlapping objects are present and applying these schemes results in significant computational overhead without any accompanying, additional benefit. In this paper we present a novel adaptive active contour scheme (AdACM) that combines boundary and region based energy terms with a shape prior in a multi level set formulation. To reduce the computational overhead, the shape prior term in the variational formulation is only invoked for those instances in the image where overlaps between objects are identified; these overlaps being identified via a contour concavity detection scheme. By not having to invoke all three terms (shape, boundary, region) for segmenting every object in the scene, the computational expense of the integrated active contour model is dramatically reduced, a particularly relevant consideration when multiple objects have to be segmented on very large histopathological images. The AdACM was employed for the task of segmenting nuclei on 80 prostate cancer tissue microarray images from 40 patient studies. Nuclear shape based, architectural and textural features extracted from these segmentations were extracted and found to able to discriminate different Gleason grade patterns with a classification accuracy of 86% via a quadratic discriminant analysis (QDA) classifier. On average the AdACM model provided 60% savings in computational times compared to a non-optimized hybrid active contour model involving a shape prior.

Cell cluster graph for prediction of biochemical recurrence in prostate cancer patients from tissue microarrays

Prostate cancer (CaP) is evidenced by profound changes in the spatial distribution of cells. Spatial arrangement and architectural organization of nuclei, especially clustering of the cells, within CaP histopathology is known to be predictive of disease aggressiveness and potentially patient outcome. Quantitative histomorphometry is a relatively new field which attempt to develop and apply novel advanced computerized image analysis and feature extraction methods for the quantitative characterization of tumor morphology on digitized pathology slides. Recently, graph theory has been used to characterize the spatial arrangement of these cells by constructing a graph with cell/nuclei as the node. One disadvantage of several extant graph based algorithms (Voronoi, Delaunay, Minimum Spanning Tree) is that they do not allow for extraction of local spatial attributes from complex networks such as those that emerges from large histopathology images with potentially thousands of nuclei. In this paper, we define a cluster of cells as a node and construct a novel graph called Cell Cluster Graph (CCG) to characterize local spatial architecture. CCG is constructed by first identifying the cell clusters to use as nodes for the construction of the graph. Pairwise spatial relationship between nodes is translated into edges of the CCG, each of which are assigned certain probability, i.e. each edge between any pair of a nodes has a certain probability to exist. Spatial constraints are employed to deconstruct the entire graph into subgraphs and we then extract global and local graph based features from the CCG. We evaluated the ability of the CCG to predict 5 year biochemical failures in men with CaP and who had previously undergone radical prostatectomy. Extracted features from CCG constructed using nuclei as nodal centers on tissue microarray (TMA) images obtained from the surgical specimens of 80 patients allowed us to train a support vector machine classifier via a 3 fold randomized cross validation procedure which yielded a classification accuracy of 83:1±1:2%. By contrast the Voronoi, Delaunay, and Minimum spanning tree based graph classifiers yielded corresponding classification accuracies of 67:1±1:8% and 60:7±0:9% respectively.

Active Contour for Overlap Resolution using Watershed BASED initialization (ACOReW): Applications to histopathology

In recent years, shape based active contours have emerged as a natural solution to overlap resolution. However, most of these shape-based methods are limited to finding and resolving one object overlap per scene and require user intervention for model initialization. In this paper, we present a novel synergistic segmentation scheme called Active Contour for Overlap Resolution using Watershed (ACOReW). ACOReW combines shape priors with boundary and region-based active contours in a level set formulation with a watershed scheme for model initialization for identifying and resolving multiple object overlaps in an image scene. The energy functional for the variational active contour model is composed of three complimentary terms (a) a shape model which constrains the active contour to a pre-defined shape, (b) boundary based term which directs the active contour model to the image gradient, and (c) a third term driving the shape prior and the active contour towards a homogeneous intensity region. In this paper we show an application of ACOReW in the context of segmenting nuclear and glandular structures on prostate and breast cancer histopathology. The results of qualitative and quantitative evaluation on 100 prostate and 14 breast cancer histology images reveals that ACOReW outperforms two state of the art segmentation schemes (Geodesic Active Contour (GAC) and Roussons shape based model) and resolves up to 92% of overlapping/occluded lymphocytes and nuclei on prostate and breast cancer histology images.

Feature Importance in Nonlinear Embeddings (FINE): Applications in Digital Pathology

Quantitative histomorphometry (QH) refers to the process of computationally modeling disease appearance on digital pathology images by extracting hundreds of image features and using them to predict disease presence or outcome. Since constructing a robust and interpretable classifier is challenging in a high dimensional feature space, dimensionality reduction (DR) is often implemented prior to classifier construction. However, when DR is performed it can be challenging to quantify the contribution of each of the original features to the final classification result. We have previously presented a method for scoring features based on their importance for classification on an embedding derived via principal components analysis (PCA). However, nonlinear DR involves the eigen-decomposition of a kernel matrix rather than the data itself, compounding the issue of classifier interpretability. In this paper we present feature importance in nonlinear embeddings (FINE), an extension of our PCA-based feature scoring method to kernel PCA (KPCA), as well as several NLDR algorithms that can be cast as variants of KPCA. FINE is applied to four digital pathology datasets to identify key QH features for predicting the risk of breast and prostate cancer recurrence. Measures of nuclear and glandular architecture and clusteredness were found to play an important role in predicting the likelihood of recurrence of both breast and prostate cancers. Compared to the t-test, Fisher score, and Gini index, FINE was able to identify a stable set of features that provide good classification accuracy on four publicly available datasets from the NIPS 2003 Feature Selection Challenge.

Spatially Aware Cell Cluster(SpACCl) Graphs: Predicting Outcome in Oropharyngeal p16+ Tumors

Quantitative measurements of spatial arrangement of nuclei in histopathology images for different cancers has been shown to have prognostic value. Traditionally, graph algorithms (with cell/nuclei as node) have been used to characterize the spatial arrangement of these cells. However, these graphs inherently extract only global features of cell or nuclear architecture and, therefore, important information at the local level may be left unexploited. Additionally, since the graph construction does not draw a distinction between nuclei in the stroma or epithelium, the graph edges often traverse the stromal and epithelial regions. In this paper, we present a new spatially aware cell cluster (SpACC1) graph that can efficiently and accurately model local nuclear interactions, separately within the stromal and epithelial regions alone. SpACC1 is built locally on nodes that are defined on groups/clusters of nuclei rather than individual nuclei. Local nodes are connected with edges which have a certain probability of connectedness. The SpACC1 graph allows for exploration of (a) contribution of nuclear arrangement within the stromal and epithelial regions separately and (b) combined contribution of stromal and epithelial nuclear architecture in predicting disease aggressiveness and patient outcome. In a cohort of 160 p16+ oropharyngeal tumors (141 non-progressors and 19 progressors), a support vector machine (SVM) classifier in conjunction with 7 graph features extracted from the SpACC1 graph yielded a mean accuracy of over 90% with PPV of 89.4% in distinguishing between progressors and non-progressors. Our results suggest that (a) stromal nuclear architecture has a role to play in predicting disease aggressiveness and that (b) combining nuclear architectural contributions from the stromal and epithelial regions yields superior prognostic accuracy compared to individual contributions from stroma and epithelium alone.