Soheil Kolouri

Transport-based single frame super resolution of very low resolution face images

Extracting high-resolution information from highly degraded facial images is an important problem with several applications in science and technology. Here we describe a single frame super resolution technique that uses a transport-based formulation of the problem. The method consists of a training and a testing phase. In the training phase, a nonlinear Lagrangian model of high-resolution facial appearance is constructed fully automatically. In the testing phase, the resolution of a degraded image is enhanced by finding the model parameters that best fit the given low resolution data. We test the approach on two face datasets, namely the extended Yale Face Database B and the AR face datasets, and compare it to state of the art methods. The proposed method outperforms existing solutions in problems related to enhancing images of very low resolution.

Detecting and visualizing cell phenotype differences from microscopy images using transport-based morphometry

Significance Much of what is currently known about how cells work has been derived through visual interpretation of microscopy images. Computational methods for image analysis have emerged as quantitative alternatives to visual interpretation. We describe an analysis pipeline for cell image databases that combines statistical pattern recognition with the mathematics of optimal mass transport. The approach is fully automated and does not require the use of ad hoc numerical features. It enables the identification of discriminant phenotypic variations, or biomarkers, between sets of cells (e.g., normal vs. diseased) while at the same time allowing for the visualization of meaningful differences. The approach can be used for fully automated high content screening with a variety of microscopic image modalities. Modern microscopic imaging devices are able to extract more information regarding the subcellular organization of different molecules and proteins than can be obtained by visual inspection. Predetermined numerical features (descriptors) often used to quantify cells extracted from these images have long been shown useful for discriminating cell populations (e.g., normal vs. diseased). Direct visual or biological interpretation of results obtained, however, is often not a trivial task. We describe an approach for detecting and visualizing phenotypic differences between classes of cells based on the theory of optimal mass transport. The method is completely automated, does not require the use of predefined numerical features, and at the same time allows for easily interpretable visualizations of the most significant differences. Using this method, we demonstrate that the distribution pattern of peripheral chromatin in the nuclei of cells extracted from liver and thyroid specimens is associated with malignancy. We also show the method can correctly recover biologically interpretable and statistically significant differences in translocation imaging assays in a completely automated fashion.

Sliced Wasserstein Kernels for Probability Distributions

Optimal transport distances, otherwise known as Wasserstein distances, have recently drawn ample attention in computer vision and machine learning as powerful discrepancy measures for probability distributions. The recent developments on alternative formulations of the optimal transport have allowed for faster solutions to the problem and have revamped their practical applications in machine learning. In this paper, we exploit the widely used kernel methods and provide a family of provably positive definite kernels based on the Sliced Wasserstein distance and demonstrate the benefits of these kernels in a variety of learning tasks. Our work provides a new perspective on the application of optimal transport flavored distances through kernel methods in machine learning tasks.

The Radon Cumulative Distribution Transform and Its Application to Image Classification

Invertible image representation methods (transforms) are routinely employed as low-level image processing operations based on which feature extraction and recognition algorithms are developed. Most transforms in current use (e.g., Fourier, wavelet, and so on) are linear transforms and, by themselves, are unable to substantially simplify the representation of image classes for classification. Here, we describe a nonlinear, invertible, low-level image processing transform based on combining the well-known Radon transform for image data, and the 1D cumulative distribution transform proposed earlier. We describe a few of the properties of this new transform, and with both theoretical and experimental results show that it can often render certain problems linearly separable in a transform space.

Optimal Mass Transport: Signal processing and machine-learning applications

Transport-based techniques for signal and data analysis have recently received increased interest. Given their ability to provide accurate generative models for signal intensities and other data distributions, they have been used in a variety of applications, including content-based retrieval, cancer detection, image superresolution, and statistical machine learning, to name a few, and they have been shown to produce state-of-the-art results. Moreover, the geometric characteristics of transport-related metrics have inspired new kinds of algorithms for interpreting the meaning of data distributions. Here, we provide a practical overview of the mathematical underpinnings of mass transport-related methods, including numerical implementation, as well as a review, with demonstrations, of several applications. Software accompanying this article is available from [43].

A continuous linear optimal transport approach for pattern analysis in image datasets

We present a new approach to facilitate the application of the optimal transport metric to pattern recognition on image databases. The method is based on a linearized version of the optimal transport metric, which provides a linear embedding for the images. Hence, it enables shape and appearance modeling using linear geometric analysis techniques in the embedded space. In contrast to previous work, we use Monges formulation of the optimal transport problem, which allows for reasonably fast computation of the linearized optimal transport embedding for large images. We demonstrate the application of the method to recover and visualize meaningful variations in a supervised-learning setting on several image datasets, including chromatin distribution in the nuclei of cells, galaxy morphologies, facial expressions, and bird species identification. We show that the new approach allows for high-resolution construction of modes of variations and discrimination and can enhance classification accuracy in a variety of image discrimination problems.

Detection of malignant mesothelioma using nuclear structure of mesothelial cells in effusion cytology specimens

Mesothelioma is a form of cancer generally caused from previous exposure to asbestos. Although it was considered a rare neoplasm in the past, its incidence is increasing worldwide due to extensive use of asbestos. In the current practice of medicine, the gold standard for diagnosing mesothelioma is through a pleural biopsy with subsequent histologic examination of the tissue. The diagnostic tissue should demonstrate the invasion by the tumor and is obtained through thoracoscopy or open thoracotomy, both being highly invasive surgical operations. On the other hand, thoracocentesis, which is removal of effusion fluid from the pleural space, is a far less invasive procedure that can provide material for cytological examination. In this study, we aim at detecting and classifying malignant mesothelioma based on the nuclear chromatin distribution from digital images of mesothelial cells in effusion cytology specimens. Accordingly, a computerized method is developed to determine whether a set of nuclei belonging to a patient is benign or malignant. The quantification of chromatin distribution is performed by using the optimal transport‐based linear embedding for segmented nuclei in combination with the modified Fisher discriminant analysis. Classification is then performed through a k‐nearest neighborhood approach and a basic voting strategy. Our experiments on 34 different human cases result in 100% accurate predictions computed with blind cross validation. Experimental comparisons also show that the new method can significantly outperform standard numerical feature‐type methods in terms of agreement with the clinical diagnosis gold standard. According to our results, we conclude that nuclear structure of mesothelial cells alone may contain enough information to separate malignant mesothelioma from benign mesothelial proliferations.

Zero Shot Learning via Multi-scale Manifold Regularization

We address zero-shot learning using a new manifold alignment framework based on a localized multi-scale transform on graphs. Our inference approach includes a smoothness criterion for a function mapping nodes on a graph (visual representation) onto a linear space (semantic representation), which we optimize using multi-scale graph wavelets. The robustness of the ensuing scheme allows us to operate with automatically generated semantic annotations, resulting in an algorithm that is entirely free of manual supervision, and yet improves the state-of-the-art as measured on benchmark datasets.

Novel computer-aided diagnosis of mesothelioma using nuclear structure of mesothelial cells in effusion cytology specimens

diagnostic standard is a pleural biopsy with subsequent histologic examination of the tissue demonstrating invasion by the tumor. The diagnostic tissue is obtained through thoracoscopy or open thoracotomy, both being highly invasive procedures. Thoracocenthesis, or removal of effusion fluid from the pleural space, is a far less invasive procedure that can provide material for cytological examination. However, it is insufficient to definitively confirm or exclude the diagnosis of malignant mesothelioma, since tissue invasion cannot be determined. In this study, we present a computerized method to detect and classify malignant mesothelioma based on the nuclear chromatin distribution from digital images of mesothelial cells in effusion cytology specimens. Our method aims at determining whether a set of nuclei belonging to a patient, obtained from effusion fluid images using image segmentation, is benign or malignant, and has a potential to eliminate the need for tissue biopsy. This method is performed by quantifying chromatin morphology of cells using the optimal transportation (Kantorovich–Wasserstein) metric in combination with the modified Fisher discriminant analysis, a k-nearest neighborhood classification, and a simple voting strategy. Our results show that we can classify the data of 10 different human cases with 100% accuracy after blind cross validation. We conclude that nuclear structure alone contains enough information to classify the malignant mesothelioma. We also conclude that the distribution of chromatin seems to be a discriminating feature between nuclei of benign and malignant mesothelioma cells.

Discovery and visualization of structural biomarkers from MRI using transport-based morphometry

&NA; Disease in the brain is often associated with subtle, spatially diffuse, or complex tissue changes that may lie beneath the level of gross visual inspection, even on magnetic resonance imaging (MRI). Unfortunately, current computer‐assisted approaches that examine pre‐specified features, whether anatomically‐defined (i.e. thalamic volume, cortical thickness) or based on pixelwise comparison (i.e. deformation‐based methods), are prone to missing a vast array of physical changes that are not well‐encapsulated by these metrics. In this paper, we have developed a technique for automated pattern analysis that can fully determine the relationship between brain structure and observable phenotype without requiring any a priori features. Our technique, called transport‐based morphometry (TBM), is an image transformation that maps brain images losslessly to a domain where they become much more separable. The new approach is validated on structural brain images of healthy older adult subjects where even linear models for discrimination, regression, and blind source separation enable TBM to independently discover the characteristic changes of aging and highlight potential mechanisms by which aerobic fitness may mediate brain health later in life. TBM is a generative approach that can provide visualization of physically meaningful shifts in tissue distribution through inverse transformation. The proposed framework is a powerful technique that can potentially elucidate genotype‐structural‐behavioral associations in myriad diseases. HighlightsNew image transformation framework to facilitate discrimination tasks from MRI.Automated discovery of image biomarkers using the novel pattern analysis technique.Lossless visualization of the morphology underlying statistical associations.