Joshua Stough

A method and software for segmentation of anatomic object ensembles by deformable m‐reps

Deformable shape models (DSMs) comprise a general approach that shows great promise for automatic image segmentation. Published studies by others and our own research results strongly suggest that segmentation of a normal or near-normal object from 3D medical images will be most successful when the DSM approach uses (1) knowledge of the geometry of not only the target anatomic object but also the ensemble of objects providing context for the target object and (2) knowledge of the image intensities to be expected relative to the geometry of the target and contextual objects. The segmentation will be most efficient when the deformation operates at multiple object-related scales and uses deformations that include not just local translations but the biologically important transformations of bending and twisting, i.e., local rotation, and local magnification. In computer vision an important class of DSM methods uses explicit geometric models in a Bayesian statistical framework to provide a priori information used in posterior optimization to match the DSM against a target image. In this approach a DSM of the object to be segmented is placed in the target image data and undergoes a series of rigid and nonrigid transformations that deform the model to closely match the target object. The deformation process is driven by optimizing an objective function that has terms for the geometric typicality and model-to-image match for each instance of the deformed model. The success of this approach depends strongly on the object representation, i.e., the structural details and parameter set for the DSM, which in turn determines the analytic form of the objective function. This paper describes a form of DSM called m-reps that has or allows these properties, and a method of segmentation consisting of large to small scale posterior optimization of m-reps. Segmentation by deformable m-reps, together with the appropriate data representations, visualizations, and user interface, has been implemented in software that accomplishes 3D segmentations in a few minutes. Software for building and training models has also been developed. The methods underlying this software and its abilities are the subject of this paper.

Regional appearance in deformable model segmentation

Automated medical image segmentation is a challenging task that benefits from the use of effective image appearance models. In this paper, we compare appearance models at three regional scales for statistically characterizing image intensity near object boundaries in the context of segmentation via deformable models. The three models capture appearance in the form of regional intensity quantile functions. These distribution-based regional image descriptors are amenable to Euclidean methods such as principal component analysis, which we use to build the statistical appearance models. The first model uses two regions, the interior and exterior of the organ of interest. The second model accounts for exterior inhomogeneity by clustering on object-relative local intensity quantile functions to determine tissue-consistent regions relative to the organ boundary. The third model analyzes these image descriptors per geometrically defined local region. To evaluate the three models, we present segmentation results on bladders and prostates in CT in the context of day-to-day adaptive radiotherapy for the treatment of prostate cancer. Results show improved segmentations with more local regions, probably because smaller regions better represent local inhomogeneity in the intensity distribution near the organ boundary.

Histogram statistics of local model-relative image regions

We present a novel approach to statistically characterize histograms of model-relative image regions. A multiscale model is used as an aperture to define image regions at multiple scales. We use this image description to define an appearance model for deformable model segmentation. Appearance models measure the likelihood of an object given a target image. To determine this likelihood we compute pixel intensity histograms of local model-relative image regions from a 3D image volume near the object boundary. We use a Gaussian model to statistically characterize the variation of non-parametric histograms mapped to Euclidean space using the Earth Movers distance. The new method is illustrated and evaluated in a deformable model segmentation study on CT images of the human bladder, prostate, and rectum. Results show improvement over a previous profile based appearance model, out-performance of statistically modeled histograms over simple histogram measurements, and advantages of regional histograms at a fixed local scale over a fixed global scale.

Clustering on image boundary regions for deformable model segmentation

We present a novel approach, clustering on local image profiles, for statistically characterizing image intensity in object boundary regions. In deformable model segmentation, a driving consideration is the geometry to image match, the degree to which the target image conforms to some template within the object boundary regions. The template should account for variation over a training set and yet be specific enough to drive an optimization to a desirable result. Using clustering, a template can be built that is optimal over the training data in the metric used, such as normalized correlation. We present a method that first determines local cross-boundary image profile types in the space of training data and then builds a template of optimal types. Also presented are the results of a study using this approach on the human kidney in the context of medial representation deformable model segmentation. The results show an improvement in the automatic segmentations using the cluster template, over a previously built template.

A statistical appearance model based on intensity quantile histograms

We present a novel histogram method for statistically characterizing the appearance of deformable models. In deformable model segmentation, appearance models measure the likelihood of an object given a target image. To determine this likelihood we compute pixel intensity quantile histograms of object-relative image regions from a weighted 3D image volume near the object boundary. We use a Gaussian model to statistically characterize the variation of histograms understood in Euclidean space via the Mallows distance. The probability of gas and bone tissue intensities are separately modeled to leverage a priori information on their expected distributions. The method is illustrated and evaluated in a segmentation study on CT images of the human left kidney. Results show improvement over a profile based appearance model and that the global maximum of the MAP estimate gives clinically acceptable segmentations in almost all of the cases studied

Conditional-mean initialization using neighboring objects in deformable model segmentation

Most model-based segmentation methods find a target object in a new image by constructing an objective function and optimizing it using a standard minimization algorithm. In general, the objective function has two penalty terms: 1) for deforming a template model and 2) for mismatch between the trained image intensities relative to the template model and the observed image intensities relative to the deformed template model in the target image. While it is difficult to establish an objective function with a global minimum at the desired segmentation result, even such an objective function is typically non-convex due to the complexity of the intensity patterns and the many structures surrounding the target object. Thus, it is critical that the optimization starts at a point close to the global minimum of the objective function in deformable model-based segmentation framework. For a segmentation method in maximum a posteriori framework a good objective function can be obtained by learning the probability distributions of the population shape deformations and their associated image intensities because each penalty term can be simplified to a squared function of some distance metric defined in the shape space. The mean shape and intensities of the learned probability distributions also provide a good initialization for segmentation. However, a major concern in estimating the shape prior is the stability of the estimated shape distributions from given training samples because the feature space of a shape model is usually very high dimensional while the number of training samples is limited. A lot of effort in that regard have been made to attain a stable estimation of shape probability distribution. In this paper, we describe our approach to stably estimate a shape probability distribution when good segmentations of objects adjacent to the target object are available. Our approach is to use a conditional shape probability distribution (CSPD) to take into account in the shape distribution the relation of the target object to neighboring objects. In particular, we propose a new method based on principal component regression (PCR) in reflecting in the conditional term of the CSPD the effect of neighboring objects on the target object. The resulting approach is able to give a better and robust initialization with training samples of a few cases. To demonstrate the potential of our approach, we apply it first to training of a simulated data of known deformations and second to male pelvic organs, using the CSPD in m-rep segmentations of the prostate in CT images. Our results show a clear improvement in initializing the prostate by its conditional mean given the bladder and the rectum as neighboring objects, as measured both by volume overlap and average surface distance.

CLUSTERING ON LOCAL APPEARANCE FOR DEFORMABLE MODEL SEGMENTATION

We present a novel local region approach for statistically characterizing appearance in the context of medical image segmentation via deformable models. Our appearance model reflects the inhomogeneity of tissue mixtures around the exterior of the object of interest by determining mixture-consistent local region types relative to the object boundary. The region types are formed by clustering local regional image descriptors. We partition the object boundary according to region type and apply principal component analysis on the cluster populations to acquire a statistical model of object appearance that accounts for local variability in the object exterior. We present results using this approach to segment bladders and prostates in CT in the context of day-to-day adaptive radiotherapy for prostate cancer. Results show improved fits versus those obtained with a previously developed method

Deep structure of images in populations via geometric models in populations

We face the question of how to produce a scale space of image intensities relative to a scale space of objects or other characteristic image regions filling up the image space, when both images and objects are understood to come from a population. We argue for a schema combining a multi-scale image representation with a multi-scale representation of objects or regions. The objects or regions at one scale level are produced using soft-edged apertures, which are subdivided into sub-regions. The intensities in the regions are represented using histograms. Relevant probabilities of region shape and inter-relations between region geometry and of histograms are described, and the means is given of inter-relating the intensity probabilities and geometric probabilities by producing the probabilities of intensities conditioned on geometry.

WE‐C‐I‐609‐07: On Constructing Priors and Likelihoods for Deformable Shape Models

Purpose: Explicit deformable shape models (DSMs) can be used in a Bayesian statistical framework to provide a prioriinformation for posterior optimization to match the DSM against a target image for automatic segmentation. In this approach a DSM is initialized in the target image and undergoes a series of deformations to closely match the target object. Deformation is driven by optimizing an objective function with terms for geometric typicality (prior) and model‐to‐image match (likelihood). The purpose of this work was to develop strategy, methodology, and tools for constructing the geometric prior and intensity likelihood for a particular form of DSM called m‐reps. Method and Materials: Geometric truth is defined for an object of interest by a statistically significant collection of expert human segmentations of training images. M‐reps are fit to the human drawn contours by minimizing the distance between the surfaces of the m‐rep and the contours under added conditions that lead to positional correspondence across training cases. The geometry of the resulting set of training m‐reps is analyzed in non‐Euclidean space using an approach called principal geodesic analysis (PGA) to yield a set of eigenmodes that define the geometric prior. The intensity likelihood is constructed by registering each training m‐rep with the corresponding gray scale image and collecting regional intensity information that is statistically characterized over all training cases. The intensity information can be in several forms including linear profiles and regional histograms. Results: PGA produces modes that include natural deformations such as local twisting, bending, bulging, and constricting. Unlike analysis in Euclidean space, improper shapes are avoided. The form of the intensity prior can be customized to each object of interest for optimal performance. Conclusion: These methods are powerful, robust and generalizable to other DSMs. Conflict of Interest: The presenting author has a financial interest in Morphormics, Inc.