Ravi Malladi

Images as Embedded Maps and Minimal Surfaces: Movies, Color, Texture, and Volumetric Medical Images

We extend the geometric framework introduced in Sochen et al. (IEEE Trans. on Image Processing, 7(3):310–318, 1998) for image enhancement. We analyze and propose enhancement techniques that selectively smooth images while preserving either the multi-channel edges or the orientation-dependent texture features in them. Images are treated as manifolds in a feature-space. This geometrical interpretation lead to a general way for grey level, color, movies, volumetric medical data, and color-texture image enhancement.We first review our framework in which the Polyakov action from high-energy physics is used to develop a minimization procedure through a geometric flow for images. Here we show that the geometric flow, based on manifold volume minimization, yields a novel enhancement procedure for color images. We apply the geometric framework and the general Beltrami flow to feature-preserving denoising of images in various spaces.Next, we introduce a new method for color and texture enhancement. Motivated by Gabors geometric image sharpening method (Gabor, Laboratory Investigation, 14(6):801–807, 1965), we present a geometric sharpening procedure for color images with texture. It is based on inverse diffusion across the multi-channel edge, and diffusion along the edge.

Image processing: flows under min/max curvature and mean curvature

We present a class of PDE-based algorithms suitable for image denoising and enhancement. The techniques are applicable to both salt-and-pepper gray-scale noise and full-image continuous noise present in black and white images, gray-scale images, texture images, and color images. At the core, the techniques rely on two fundamental ideas. First, a level set formulation is used for evolving curves; use of this technique to flow isointensity contours under curvature is known to remove noise and enhance images. Second, the particular form of the curvature flow is governed by a min/max switch which selects a range of denoising dependent on the size of switching window. Our approach has several virtues. First, it contains only one enhancement parameter, which in most cases is automatically chosen. Second, the scheme automatically stops smoothing at a point which depends on the switching window size; continued application of the scheme produces no further change. Third, the method is one of the fastest possible schemes based on a curvature-controlled approach.

Evolutionary fronts for topology-independent shape modeling and recovery

This paper presents a novel framework for shape modeling and shape recovery based on ideas developed by Osher & Sethian for interface motion. In this framework, shapes are represented by propagating fronts, whose motion is governed by a “Hamilton-Jacobi” type equation. This equation is written for a function in which the interface is a particular level set. Unknown shapes are modeled by making the front adhere to the object boundary of interest under the influence of a synthesized halting criterion. The resulting equation of motion is solved using a narrow-band algorithm designed for rapid front advancement. Our techniques can be applied to model arbitrarily complex shapes, which include shapes with significant protrusions, and to situations where no a priori assumption about the objects topology can be made. We demonstrate the scheme via examples of shape recovery in 2D and 3D from synthetic and low contrast medical image data.

Segmentation of nuclei and cells using membrane related protein markers

Segmenting individual cell nuclei from microscope images normally involves volume labelling of the nuclei with a DNA stain. However, this method often fails when the nuclei are tightly clustered in the tissue, because there is little evidence from the images on where the borders of the nuclei are. In this paper we present a method which solves this limitation and furthermore enables segmentation of whole cells. Instead of using volume stains, we used stains that specifically label the surface of nuclei or cells: lamins for the nuclear envelope and alpha‐6 or beta‐1 integrins for the cellular surface. The segmentation is performed by identifying unique seeds for each nucleus/cell and expanding the boundaries of the seeds until they reach the limits of the nucleus/cell, as delimited by the lamin or integrin staining, using gradient‐curvature flow techniques. We tested the algorithm using computer‐generated objects to evaluate its robustness against noise and applied it to cells in culture and to tissue specimens. In all the cases that we present the algorithm gave accurate results.

From High Energy Physics to Low Level Vision

A geometric framework for image scale space, enhancement, and segmentation is presented. We consider intensity images as surfaces in the (x, I) space. The image is thereby a 2D surface in 3D space for gray level images, and a 2D surface in 5D for color images. The new formulation unifies many classical schemes and algorithms via a simple scaling of the intensity contrast, and results in new and efficient schemes. Extensions to multi dimensional signals become natural and lead to powerful denoising and scale space algorithms. Here, we demonstrate the proposed framework by applying it to denoise and improve gray level and color images.

A real-time algorithm for medical shape recovery

In this paper, we present a shape recovery technique in 2D and 3D with specific applications in visualizing and measuring anatomical shapes from medical images. This algorithm models extremely corrugated structures like the brain, is topologically adaptable, is robust, and runs in O(N log N) time where N is the total number of points in the domain. Our two-stage technique is based on the level set shape recovery scheme and the fast marching method for computing solutions to static Hamilton-Jacobi equations.

A geometric approach to segmentation and analysis of 3D medical images

A geometric scheme for detecting, representing, and measuring 3D medical data is presented. The technique based on deforming 3D surfaces, represented via level-sets, towards the medical objects, according to intrinsic geometric measures of the data. The 3D medical object is represented as a (weighted) minimal surface in a Riemannian space whose metric is induced from the image. This minimal surface is computed using the level-set methodology for propagating interfaces, combined with a narrow band technique which allows fast implementation. This computation technique automatically handles topological changes. Measurements like volume and area are performed on the surface, exploiting the representation and the high accuracy intrinsic to the algorithm.

Subjective Surfaces: A Geometric Model for Boundary Completion

We present a geometric model and a computational method for segmentation of images with missing boundaries. In many situations, the human visual system fills in missing gaps in edges and boundaries, building and completing information that is not present. Boundary completion presents a considerable challenge in computer vision, since most algorithms attempt to exploit existing data. A large body of work concerns completion models, which postulate how to construct missing data; these models are often trained and specific to particular images. In this paper, we take the following, alternative perspective: we consider a given reference point within the image, and then develop an algorithm which tries to build missing information on the basis of the given point of view and the available information as boundary data to the algorithm. Starting from this point of view, a surface is constructed. It is then evolved with the mean curvature flow in the metric induced by the image until a piecewise constant solution is reached. We test the computational model on modal completion, amodal completion, and texture segmentation. We extend the geometric model and the algorithm to 3D in order to extract shapes from low signal/noise ratio ultrasound image volumes. Results in 3D echocardiography and 3D fetal echography are also presented.

A geometric model for 3-D confocal image analysis

The authors use partial-differential-equation-based filtering as a preprocessing and post processing strategy for computer-aided cytology. They wish to accurately extract and classify the shapes of nuclei from confocal microscopy images, which is a prerequisite to an accurate quantitative intranuclear (genotypic and phenotypic) and internuclear (tissue structure) analysis of tissue and cultured specimens. First, the authors study the use of a geometry-driven edge-preserving image smoothing mechanism before nuclear segmentation. They show how this filter outperforms other widely-used filters in that it provides higher edge fidelity. Then they apply the same filter with a different initial condition, to smooth nuclear surfaces and obtain sub-pixel accuracy. Finally the authors use another instance of the geometrical filter to correct for misinterpretations of the nuclear surface by the segmentation algorithm. Their prefiltering and post filtering nicely complements their initial segmentation strategy, in that it provides substantial and measurable improvement in the definition of the nuclear surfaces.

Level set methods for curvature flow, image enhancement, and shape recovery in medical images

Level set methods are powerful numerical techniques for tracking the evolution of interfaces moving under a variety of complex motions. They are based on computing viscosity solutions to the appropriate equations of motion, using techniques borrowed from hyperbolic conservation laws. In this paper, we review some of the applications of this work to curvature motion, the construction of minimal surfaces, image enhancement, and shape recovery. We introduce new schemes for denoising three-dimensional shapes and images, as well as a fast shape recovery techniques for three-dimensional images.