Gustavo K. Rohde

The adaptive bases algorithm for intensity-based nonrigid image registration

Nonrigid registration of medical images is important for a number of applications such as the creation of population averages, atlas-based segmentation, or geometric correction of functional magnetic resonance imaging (IMRI) images to name a few. In recent years, a number of methods have been proposed to solve this problem, one class of which involves maximizing a mutual information (Ml)-based objective function over a regular grid of splines. This approach has produced good results but its computational complexity is proportional to the compliance of the transformation required to register the smallest structures in the image. Here, we propose a method that permits the spatial adaptation of the transformations compliance. This spatial adaptation allows us to reduce the number of degrees of freedom in the overall transformation, thus speeding up the process and improving its convergence properties. To develop this method, we introduce several novelties: 1) we rely on radially symmetric basis functions rather than B-splines traditionally used to model the deformation field; 2) we propose a metric to identify regions that are poorly registered and over which the transformation needs to be improved; 3) we partition the global registration problem into several smaller ones; and 4) we introduce a new constraint scheme that allows us to produce transformations that are topologically correct. We compare the approach we propose to more traditional ones and show that our new algorithm compares favorably to those in current use.

Deformation‐based nuclear morphometry: Capturing nuclear shape variation in HeLa cells

The empirical characterization of nuclear shape distributions is an important unsolved problem with many applications in biology and medicine. Numerous genetic diseases and cancers have alterations in nuclear morphology, and methods for characterization of morphology could aid in both diagnoses and fundamental understanding of these disorders. Automated approaches have been used to measure features related to the size and shape of the cell nucleus, and statistical analysis of these features has often been performed assuming an underlying Euclidean (linear) vector space. We discuss the difficulties associated with the analysis of nuclear shape in light of the fact that shape spaces are nonlinear, and demonstrate methods for characterizing nuclear shapes and shape distributions based on spatial transformations that map one nucleus to another. By combining large deformation metric mapping with multidimensional scaling we offer a flexible approach for elucidating the intrinsic nonlinear degrees of freedom of a distribution of nuclear shapes. More specifically, we demonstrate approaches for nuclear shape interpolation and computation of mean nuclear shape. We also provide a method for estimating the number of free parameters that contribute to shape as well as an approach for visualizing most representative shape variations within a distribution of nuclei. The proposed methodology can be completely automated, is independent of the dimensionality of the images, and can handle complex shapes. Results obtained by analyzing two sets of images of HeLa cells are shown. In addition to identifying the modes of variation in normal HeLa nuclei, the effects of lamin A/C on nuclear morphology are quantitatively described.

Detection and classification of thyroid follicular lesions based on nuclear structure from histopathology images.

Follicular lesions of the thyroid are traditionally difficult and tedious challenges in diagnostic surgical pathology in part due to lack of obvious discriminatory cytological and microarchitectural features. We describe a computerized method to detect and classify follicular adenoma of the thyroid, follicular carcinoma of the thyroid, and normal thyroid based on the nuclear chromatin distribution from digital images of tissue obtained by routine histological methods. Our method is based on determining whether a set of nuclei, obtained from histological images using automated image segmentation, is most similar to sets of nuclei obtained from normal or diseased tissues. This comparison is performed utilizing numerical features, a support vector machine, and a simple voting strategy. We also describe novel methods to identify unique and defining chromatin patterns pertaining to each class. Unlike previous attempts in detecting and classifying these thyroid lesions using computational imaging, our results show that our method can automatically classify the data pertaining to 10 different human cases with 100% accuracy after blind cross validation using at most 43 nuclei randomly selected from each patient. We conclude that nuclear structure alone contains enough information to automatically classify the normal thyroid, follicular carcinoma, and follicular adenoma, as long as groups of nuclei (instead of individual ones) are used. We also conclude that the distribution of nuclear size and chromatin concentration (how tightly packed it is) seem to be discriminating features between nuclei of follicular adenoma, follicular carcinoma, and normal thyroid.

A Linear Optimal Transportation Framework for Quantifying and Visualizing Variations in Sets of Images

Transportation-based metrics for comparing images have long been applied to analyze images, especially where one can interpret the pixel intensities (or derived quantities) as a distribution of ‘mass’ that can be transported without strict geometric constraints. Here we describe a new transportation-based framework for analyzing sets of images. More specifically, we describe a new transportation-related distance between pairs of images, which we denote as linear optimal transportation (LOT). The LOT can be used directly on pixel intensities, and is based on a linearized version of the Kantorovich-Wasserstein metric (an optimal transportation distance, as is the earth mover’s distance). The new framework is especially well suited for computing all pairwise distances for a large database of images efficiently, and thus it can be used for pattern recognition in sets of images. In addition, the new LOT framework also allows for an isometric linear embedding, greatly facilitating the ability to visualize discriminant information in different classes of images. We demonstrate the application of the framework to several tasks such as discriminating nuclear chromatin patterns in cancer cells, decoding differences in facial expressions, galaxy morphologies, as well as sub cellular protein distributions.

An Optimal Transportation Approach for Nuclear Structure-Based Pathology

Nuclear morphology and structure as visualized from histopathology microscopy images can yield important diagnostic clues in some benign and malignant tissue lesions. Precise quantitative information about nuclear structure and morphology, however, is currently not available for many diagnostic challenges. This is due, in part, to the lack of methods to quantify these differences from image data. We describe a method to characterize and contrast the distribution of nuclear structure in different tissue classes (normal, benign, cancer, etc.). The approach is based on quantifying chromatin morphology in different groups of cells using the optimal transportation (Kantorovich-Wasserstein) metric in combination with the Fisher discriminant analysis and multidimensional scaling techniques. We show that the optimal transportation metric is able to measure relevant biological information as it enables automatic determination of the class (e.g., normal versus cancer) of a set of nuclei. We show that the classification accuracies obtained using this metric are, on average, as good or better than those obtained utilizing a set of previously described numerical features. We apply our methods to two diagnostic challenges for surgical pathology: one in the liver and one in the thyroid. Results automatically computed using this technique show potentially biologically relevant differences in nuclear structure in liver and thyroid cancers.

A flexible and robust approach for segmenting cell nuclei from 2D microscopy images using supervised learning and template matching.

We describe a new supervised learning‐based template matching approach for segmenting cell nuclei from microscopy images. The method uses examples selected by a user for building a statistical model that captures the texture and shape variations of the nuclear structures from a given dataset to be segmented. Segmentation of subsequent, unlabeled, images is then performed by finding the model instance that best matches (in the normalized cross correlation sense) local neighborhood in the input image. We demonstrate the application of our method to segmenting nuclei from a variety of imaging modalities, and quantitatively compare our results to several other methods. Quantitative results using both simulated and real image data show that, while certain methods may work well for certain imaging modalities, our software is able to obtain high accuracy across several imaging modalities studied. Results also demonstrate that, relative to several existing methods, the template‐based method we propose presents increased robustness in the sense of better handling variations in illumination, variations in texture from different imaging modalities, providing more smooth and accurate segmentation borders, as well as handling better cluttered nuclei.

Robust efficient estimation of heart rate pulse from video

We describe a simple but robust algorithm for estimating the heart rate pulse from video sequences containing human skin in real time. Based on a model of light interaction with human skin, we define the change of blood concentration due to arterial pulsation as a pixel quotient in log space, and successfully use the derived signal for computing the pulse heart rate. Various experiments with different cameras, different illumination condition, and different skin locations were conducted to demonstrate the effectiveness and robustness of the proposed algorithm. Examples computed with normal illumination show the algorithm is comparable with pulse oximeter devices both in accuracy and sensitivity.

Toward the virtual cell: Automated approaches to building models of subcellular organization “learned” from microscopy images

We review state‐of‐the‐art computational methods for constructing, from image data, generative statistical models of cellular and nuclear shapes and the arrangement of subcellular structures and proteins within them. These automated approaches allow consistent analysis of images of cells for the purposes of learning the range of possible phenotypes, discriminating between them, and informing further investigation. Such models can also provide realistic geometry and initial protein locations to simulations in order to better understand cellular and subcellular processes. To determine the structures of cellular components and how proteins and other molecules are distributed among them, the generative modeling approach described here can be coupled with high throughput imaging technology to infer and represent subcellular organization from data with few a priori assumptions. We also discuss potential improvements to these methods and future directions for research.

A generative model of microtubule distributions, and indirect estimation of its parameters from fluorescence microscopy images

The microtubule network plays critical roles in many cellular processes, and quantitative models of how its organization varies across cell types and conditions are required for understanding those roles and as input to cell simulations. High‐throughput image acquisition technologies are potentially valuable for this purpose, but do not provide sufficient resolution for current analysis methods that rely on tracing of individual microtubules. We describe a parametric conditional model of microtubule distribution that can generate a microtubule network in intact cells using a persistent random walk approach. The model parameters are physically meaningful as they directly describe the spatial distribution of microtubules and include the number of microtubules as well as the mean of the length distribution. We also present an indirect method for estimating the parameters of the model from three‐dimensional fluorescence microscope images of cells that relies on comparing acquired images with simulated images generated from the model. Our results show that our method can reasonably recover parameters for a given query image, and we present the distributions of parameters estimated by our method for a collection of HeLa cell images.

Transport-based single frame super resolution of very low resolution face images

Extracting high-resolution information from highly degraded facial images is an important problem with several applications in science and technology. Here we describe a single frame super resolution technique that uses a transport-based formulation of the problem. The method consists of a training and a testing phase. In the training phase, a nonlinear Lagrangian model of high-resolution facial appearance is constructed fully automatically. In the testing phase, the resolution of a degraded image is enhanced by finding the model parameters that best fit the given low resolution data. We test the approach on two face datasets, namely the extended Yale Face Database B and the AR face datasets, and compare it to state of the art methods. The proposed method outperforms existing solutions in problems related to enhancing images of very low resolution.