Harshita Sharma

Determining similarity in histological images using graph-theoretic description and matching methods for content-based image retrieval in medical diagnostics

BackgroundComputer-based analysis of digitalized histological images has been gaining increasing attention, due to their extensive use in research and routine practice. The article aims to contribute towards the description and retrieval of histological images by employing a structural method using graphs. Due to their expressive ability, graphs are considered as a powerful and versatile representation formalism and have obtained a growing consideration especially by the image processing and computer vision community.MethodsThe article describes a novel method for determining similarity between histological images through graph-theoretic description and matching, for the purpose of content-based retrieval. A higher order (region-based) graph-based representation of breast biopsy images has been attained and a tree-search based inexact graph matching technique has been employed that facilitates the automatic retrieval of images structurally similar to a given image from large databases.ResultsThe results obtained and evaluation performed demonstrate the effectiveness and superiority of graph-based image retrieval over a common histogram-based technique. The employed graph matching complexity has been reduced compared to the state-of-the-art optimal inexact matching methods by applying a pre-requisite criterion for matching of nodes and a sophisticated design of the estimation function, especially the prognosis function.ConclusionThe proposed method is suitable for the retrieval of similar histological images, as suggested by the experimental and evaluation results obtained in the study. It is intended for the use in Content Based Image Retrieval (CBIR)-requiring applications in the areas of medical diagnostics and research, and can also be generalized for retrieval of different types of complex images.Virtual SlidesThe virtual slide(s) for this article can be found here: http://www.diagnosticpathology.diagnomx.eu/vs/1224798882787923.

A Multi-resolution Approach for Combining Visual Information using Nuclei Segmentation and Classification in Histopathological Images

This paper describes a multi-resolution technique to combine diagnostically important visual information at different magnifications in H&E whole slide images (WSI) of gastric cancer. The primary goal is to improve the results of nuclei segmentation method for heterogeneous histopathological datasets with variations in stain intensity and malignancy levels. A minimum-model nuclei segmentation method is first applied to tissue images at multiple resolutions, and a comparative evaluation is performed. A comprehensive set of 31 nuclei features based on color, texture and morphology are derived from the nuclei segments. AdaBoost classification method is used to classify these segments into a set of pre-defined classes. Two classification approaches are evaluated for this purpose. A relevance score is assigned to each class and a combined segmentation result is obtained consisting of objects with higher visual significance from individual magnifications, thereby preserving both coarse and fine details in the image. Quantitative and visual assessment of combination results shows that they contain comprehensive and diagnostically more relevant information than in constituent

Deep convolutional neural networks for automatic classification of gastric carcinoma using whole slide images in digital histopathology.

Deep learning using convolutional neural networks is an actively emerging field in histological image analysis. This study explores deep learning methods for computer-aided classification in H&E stained histopathological whole slide images of gastric carcinoma. An introductory convolutional neural network architecture is proposed for two computerized applications, namely, cancer classification based on immunohistochemical response and necrosis detection based on the existence of tumor necrosis in the tissue. Classification performance of the developed deep learning approach is quantitatively compared with traditional image analysis methods in digital histopathology requiring prior computation of handcrafted features, such as statistical measures using gray level co-occurrence matrix, Gabor filter-bank responses, LBP histograms, gray histograms, HSV histograms and RGB histograms, followed by random forest machine learning. Additionally, the widely known AlexNet deep convolutional framework is comparatively analyzed for the corresponding classification problems. The proposed convolutional neural network architecture reports favorable results, with an overall classification accuracy of 0.6990 for cancer classification and 0.8144 for necrosis detection.

Cell nuclei attributed relational graphs for efficient representation and classification of gastric cancer in digital histopathology

This paper describes a novel graph-based method for efficient representation and subsequent classification in histological whole slide images of gastric cancer. Her2/neu immunohistochemically stained and haematoxylin and eosin stained histological sections of gastric carcinoma are digitized. Immunohistochemical staining is used in practice by pathologists to determine extent of malignancy, however, it is laborious to visually discriminate the corresponding malignancy levels in the more commonly used haematoxylin and eosin stain, and this study attempts to solve this problem using a computer-based method. Cell nuclei are first isolated at high magnification using an automatic cell nuclei segmentation strategy, followed by construction of cell nuclei attributed relational graphs of the tissue regions. These graphs represent tissue architecture comprehensively, as they contain information about cell nuclei morphology as vertex attributes, along with knowledge of neighborhood in the form of edge linking and edge attributes. Global graph characteristics are derived and ensemble learning is used to discriminate between three types of malignancy levels, namely, non-tumor, Her2/neu positive tumor and Her2/neu negative tumor. Performance is compared with state of the art methods including four texture feature groups (Haralick, Gabor, Local Binary Patterns and Varma Zisserman features), color and intensity features, and Voronoi diagram and Delaunay triangulation. Texture, color and intensity information is also combined with graph-based knowledge, followed by correlation analysis. Quantitative assessment is performed using two cross validation strategies. On investigating the experimental results, it can be concluded that the proposed method provides a promising way for computer-based analysis of histopathological images of gastric cancer.

Appearance-based necrosis detection using textural features and SVM with discriminative thresholding in histopathological whole slide images

Automatic detection of necrosis in histological images is an interesting problem of digital pathology that needs to be addressed. Determination of presence and extent of necrosis can provide useful information for disease diagnosis and prognosis, and the detected necrotic regions can also be excluded before analyzing the remaining living tissue. This paper describes a novel appearance-based method to detect tumor necrosis in histopathogical whole slide images. Studies are performed on heterogeneous microscopic images of gastric cancer containing tissue regions with variation in malignancy level and stain intensity. Textural image features are extracted from image patches to efficiently represent necrotic appearance in the tissue and machine learning is performed using support vector machines followed by discriminative thresholding for our complex datasets. The classification results are quantitatively evaluated for different image patch sizes using two cross validation approaches namely three-fold and leave one out cross validation, and the best average cross validation rate of 85.31% is achieved for the most suitable patch size. Therefore, the proposed method is a promising tool to detect necrosis in heterogeneous whole slide images, showing its robustness to varying visual appearances.

Multi-task SonoEyeNet: Detection of Fetal Standardized Planes Assisted by Generated Sonographer Attention Maps

We present a novel multi-task convolutional neural network called Multi-task SonoEyeNet (M-SEN) that learns to generate clinically relevant visual attention maps using sonographer gaze tracking data on input ultrasound (US) video frames so as to assist standardized abdominal circumference (AC) plane detection. Our architecture consists of a generator and a discriminator, which are trained in an adversarial scheme. The generator learns sonographer attention on a given US video frame to predict the frame label (standardized AC plane / background). The discriminator further fine-tunes the predicted attention map by encouraging it to mimick the ground-truth sonographer attention map. The novel model expands the potential clinical usefulness of a previous model by eliminating the requirement of input gaze tracking data during inference without compromising its plane detection performance (Precision: 96.8, Recall: 96.2, F-1 score: 96.5).

A Comparative Study of Cell Nuclei Attributed Relational Graphs for Knowledge Description and Categorization in Histopathological Gastric Cancer Whole Slide Images

In this paper, cell nuclei attributed relational graphs are extensively studied and comparatively analyzed for effective knowledge description and classification in H&E stained whole slide images of gastric cancer. This includes design and implementation of multiple graph variations with diverse tissue component characteristics and architectural properties to obtain enhanced image representations, followed by hierarchical ensemble learning and classification. A detailed comparative analysis of the proposed graph-based methods, also with the established low-level, object-level and high-level image descriptions is performed, that further leads to a hybrid approach combining salient visual information. Quantitative evaluation of investigated methods suggests the suitability of particular graph variants for automatic classification using H&E stained histopathological gastric cancer whole slide images based on HER2 immunohistochemistry.