Antoine Veillard

Methods for Nuclei Detection, Segmentation, and Classification in Digital Histopathology: A Review—Current Status and Future Potential

Digital pathology represents one of the major evolutions in modern medicine. Pathological examinations constitute the gold standard in many medical protocols, and also play a critical and legal role in the diagnosis process. In the conventional cancer diagnosis, pathologists analyze biopsies to make diagnostic and prognostic assessments, mainly based on the cell morphology and architecture distribution. Recently, computerized methods have been rapidly evolving in the area of digital pathology, with growing applications related to nuclei detection, segmentation, and classification. In cancer research, these approaches have played, and will continue to play a key (often bottleneck) role in minimizing human intervention, consolidating pertinent second opinions, and providing traceable clinical information. Pathological studies have been conducted for numerous cancer detection and grading applications, including brain, breast, cervix, lung, and prostate cancer grading. Our study presents, discusses, and extracts the major trends from an exhaustive overview of various nuclei detection, segmentation, feature computation, and classification techniques used in histopathology imagery, specifically in hematoxylin-eosin and immunohistochemical staining protocols. This study also enables us to measure the challenges that remain, in order to reach robust analysis of whole slide images, essential high content imaging with diagnostic biomarkers and prognosis support in digital pathology.

Time-efficient sparse analysis of histopathological whole slide images

Histopathological examination is a powerful standard for the prognosis of critical diseases. But, despite significant advances in high-speed and high-resolution scanning devices or in virtual exploration capabilities, the clinical analysis of whole slide images (WSI) largely remains the work of human experts. We propose an innovative platform in which multi-scale computer vision algorithms perform fast analysis of a histopathological WSI. It relies on application-driven for high-resolution and generic for low-resolution image analysis algorithms embedded in a multi-scale framework to rapidly identify the high power fields of interest used by the pathologist to assess a global grading. GPU technologies as well speed up the global time-efficiency of the system. Sparse coding and dynamic sampling constitute the keystone of our approach. These methods are implemented within a computer-aided breast biopsy analysis application based on histopathology images and designed in collaboration with a pathology department. The current ground truth slides correspond to about 36,000 high magnification (40×) high power fields. The processing time to achieve automatic WSI analysis is on a par with the pathologists performance (about ten minutes a WSI), which constitutes by itself a major contribution of the proposed methodology.

A practical guide to CNNs and Fisher Vectors for image instance retrieval

With deep learning becoming the dominant approach in computer vision, the use of representations extracted from Convolutional Neural Nets (CNNs) is quickly gaining ground on Fisher Vectors (FVs) as favoured state-of-the-art global image descriptors for image instance retrieval. While the good performance of CNNs for image classification are unambiguously recognised, which of the two has the upper hand in the image retrieval context is not entirely clear yet.We propose a comprehensive study that systematically evaluates FVs and CNNs for image instance retrieval. The first part compares the performances of FVs and CNNs on multiple publicly available data sets and for multiple criteria. We show that no descriptor is systematically better than the other and that performance gains can usually be obtained by using both types together. The second part of the study focuses on the impact of geometrical transformations. We show that performance of CNNs can quickly degrade in the presence of certain transformations and propose a number of ways to incorporate the required invariances in the CNN pipeline.Our findings are organised as a reference guide offering practically useful and simply implementable guidelines to anyone looking for state-of-the-art global descriptors best suited to their specific image instance retrieval problem. HighlightsCNNs exhibit very limited invariance to rotation changes compared to FVDoG.CNNs are more robust to scale changes than any variants of FV.Max-pooling across rotated/scaled database images gains rotation/scale invariance.Combining FV with CNN can improve retrieval accuracy by a significant margin.

A cognitive virtual microscopic framework for knowlege-based exploration of large microscopic images in breast cancer histopathology

Histopathological examination is a powerful method for prognosis of major diseases such as breast cancer. Analysis of medical images largely remains the work of human experts. Current virtual microscope systems are mainly an emulation of real microscopes with annotation and some image analysis capabilities. However, the lack of effective knowledge management prevents such systems from being computer-aided prognosis platforms. The cognitive virtual microscopic framework, through an extended modeling and use of medical knowledge, has the capacity to analyse histopathological images and to perform grading of breast cancer, providing pathologists with a robust and traceable second opinion.

Tiny Descriptors for Image Retrieval with Unsupervised Triplet Hashing

A typical image retrieval pipeline starts with the comparison of global descriptors from a large database to find a short list of candidate matches. A good image descriptor is key to the retrieval pipeline and should reconcile two contradictory requirements: providing recall rates as high as possible and being as compact as possible for fast matching. Following the recent successes of Deep Convolutional Neural Networks (DCNN) for large scale image classification, descriptors extracted from DCNNs are increasingly used in place of the traditional hand crafted descriptors such as Fisher Vectors (FV) with better retrieval performances. Nevertheless, the dimensionality of a typical DCNN descriptor-extracted either from the visual feature pyramid or the fully-connected layers-remains quite high at several thousands of scalar values. In this paper, we propose Unsupervised Triplet Hashing (UTH), a fully unsupervised method to compute extremely compact binary hashes-in the 32-256 bits range-from high-dimensional global descriptors. UTH consists of two successive deep learning steps. First, Stacked Restricted Boltzmann Machines (SRBM), a type of unsupervised deep neural nets, are used to learn binary embedding functions able to bring the descriptor size down to the desired bitrate. SRBMs are typically able to ensure a very high compression rate at the expense of loosing some desirable metric properties of the original DCNN descriptor space. Then, triplet networks, a rank learning scheme based on weight sharing nets is used to fine-tune the binary embedding functions to retain as much as possible of the useful metric properties of the original space. A thorough empirical evaluation conducted on multiple publicly available dataset using DCNN descriptors shows that our method is able to significantly outperform state-of-the-art unsupervised schemes in the target bit range.

Nuclei extraction from histopathological images using a marked point process approach

Morphology of cell nuclei is a central aspect in many histopathological studies, in particular in breast cancer grading. Therefore, the automatic detection and extraction of cell nuclei from microscopic images obtained from cancer tissue slides is one of the most important problems in digital histopathology. We propose to tackle the problem using a model based on marked point processes (MPP), a methodology for extraction of multiple objects from images. The advantage of MPP based models is their ability to take into account the geometry of objects; and the information about their spatial repartition in the image. Previously, the MPP models have been applied for the extraction of objects of simple geometrical shapes. For histological grading, a morphological criterion known as nuclear pleomorphism corresponding to fine morphological differences between the nuclei is assessed by pathologists. Therefore, the accurate delineation of nuclei became an issue of even greater importance than optimal nuclei detection. Recently, the MPP framework has been defined on the space of arbitrarily-shaped objects allowing more accurate extraction of complex-shaped objects. The nuclei often appear joint or even overlap in histopathological images. The model still allows to extract them as individual joint or overlapping objects without discarding the overlapping parts and therefore without significant loss in delineation precision. We aim to compare the MPP model with two state-of-the-art methods selected from a comprehensive review of the available methods. The experiments are performed using a database of H&E stained breast cancer images covering a wide range of histological grades.

Compact Global Descriptors for Visual Search

The first step in an image retrieval pipeline consists of comparing global descriptors from a large database to find a short list of candidate matching images. The more compact the global descriptor, the faster the descriptors can be compared for matching. State-of-the-art global descriptors based on Fisher Vectors are represented with tens of thousands of floating point numbers. While there is significant work on compression of local descriptors, there is relatively little work on compression of high dimensional Fisher Vectors. We study the problem of global descriptor compression in the context of image retrieval, focusing on extremely compact binary representations: 64-1024 bits. Motivated by the remarkable success of deep neural networks in recent literature, we propose a compression scheme based on deeply stacked Restricted Boltzmann Machines (SRBM), which learn lower dimensional non-linear subspaces on which the data lie. We provide a thorough evaluation of several state-of-the-art compression schemes based on PCA, Locality Sensitive Hashing, Product Quantization and greedy bit selection, and show that the proposed compression scheme outperforms all existing schemes.

Cell nuclei extraction from breast cancer histopathologyimages using colour, texture, scale and shape information

Cell nuclei extraction from Haematoxylin and Eosin (H&E) stained breast cancer slide images is a challenging task due to the high content complexity of images: nuclei have heterogeneous appearance and overlap while the background is complex and non-homogeneous. This causes standard extraction methods to perform poorly.

An Exploration Scheme for Large Images: Application to Breast Cancer Grading

Most research works focus on pattern recognition within a small sample images but strategies for running efficiently these algorithms over large images are rarely if ever specifically considered. In particular, the new generation of satellite and microscopic images are acquired at a very high resolution and a very high daily rate. We propose an efficient, generic strategy to explore large images by combining computational geometry tools with a local signal measure of relevance in a dynamic sampling framework. An application to breast cancer grading from huge histopathological images illustrates the benefit of such a general strategy for new major applications in the field of microscopy.

DeepHash for Image Instance Retrieval: Getting Regularization, Depth and Fine-Tuning Right

This work focuses on representing very high-dimensional global image descriptors using very compact 64-1024 bit binary hashes for instance retrieval. We propose DeepHash: a hashing scheme based on deep networks. Key to making DeepHash work at extremely low bitrates are three important considerations -- regularization, depth and fine-tuning -- each requiring solutions specific to the hashing problem. In-depth evaluation shows that our scheme outperforms state-of-the-art methods over several benchmark datasets for both Fisher Vectors and Deep Convolutional Neural Network features, by up to 8.5% over other schemes. The retrieval performance with 256-bit hashes is close to that of the uncompressed floating point features -- a remarkable 512x compression.