Nir A. Sochen

A general framework for low level vision

We introduce a new geometrical framework based on which natural flows for image scale space and enhancement are presented. We consider intensity images as surfaces in the (x, I) space. The image is, thereby, a two-dimensional (2-D) surface in three-dimensional (3-D) space for gray-level images, and 2-D surfaces in five dimensions for color images. The new formulation unifies many classical schemes and algorithms via a simple scaling of the intensity contrast, and results in new and efficient schemes. Extensions to multidimensional signals become natural and lead to powerful denoising and scale space algorithms.

Image enhancement and denoising by complex diffusion processes

The linear and nonlinear scale spaces, generated by the inherently real-valued diffusion equation, are generalized to complex diffusion processes, by incorporating the free Schrodinger equation. A fundamental solution for the linear case of the complex diffusion equation is developed. Analysis of its behavior shows that the generalized diffusion process combines properties of both forward and inverse diffusion. We prove that the imaginary part is a smoothed second derivative, scaled by time, when the complex diffusion coefficient approaches the real axis. Based on this observation, we develop two examples of nonlinear complex processes, useful in image processing: a regularized shock filter for image enhancement and a ramp preserving denoising process.

Forward-and-backward diffusion processes for adaptive image enhancement and denoising

Signal and image enhancement is considered in the context of a new type of diffusion process that simultaneously enhances, sharpens, and denoises images. The nonlinear diffusion coefficient is locally adjusted according to image features such as edges, textures, and moments. As such, it can switch the diffusion process from a forward to a backward (inverse) mode according to a given set of criteria. This results in a forward-and-backward (FAB) adaptive diffusion process that enhances features while locally denoising smoother segments of the signal or image. The proposed method, using the FAB process, is applied in a super-resolution scheme. The FAB method is further generalized for color processing via the Beltrami flow, by adaptively modifying the structure tensor that controls the nonlinear diffusion process. The proposed structure tensor is neither positive definite nor negative, and switches between these states according to image features. This results in a forward-and-backward diffusion flow where different regions of the image are either forward or backward diffused according to the local geometry within a neighborhood.

Images as Embedded Maps and Minimal Surfaces: Movies, Color, Texture, and Volumetric Medical Images

We extend the geometric framework introduced in Sochen et al. (IEEE Trans. on Image Processing, 7(3):310–318, 1998) for image enhancement. We analyze and propose enhancement techniques that selectively smooth images while preserving either the multi-channel edges or the orientation-dependent texture features in them. Images are treated as manifolds in a feature-space. This geometrical interpretation lead to a general way for grey level, color, movies, volumetric medical data, and color-texture image enhancement.We first review our framework in which the Polyakov action from high-energy physics is used to develop a minimization procedure through a geometric flow for images. Here we show that the geometric flow, based on manifold volume minimization, yields a novel enhancement procedure for color images. We apply the geometric framework and the general Beltrami flow to feature-preserving denoising of images in various spaces.Next, we introduce a new method for color and texture enhancement. Motivated by Gabors geometric image sharpening method (Gabor, Laboratory Investigation, 14(6):801–807, 1965), we present a geometric sharpening procedure for color images with texture. It is based on inverse diffusion across the multi-channel edge, and diffusion along the edge.

Free water elimination and mapping from diffusion MRI

Relating brain tissue properties to diffusion tensor imaging (DTI) is limited when an image voxel contains partial volume of brain tissue with free water, such as cerebrospinal fluid or edema, rendering the DTI indices no longer useful for describing the underlying tissue properties. We propose here a method for separating diffusion properties of brain tissue from surrounding free water while mapping the free water volume. This is achieved by fitting a bi‐tensor model for which a mathematical framework is introduced to stabilize the fitting. Applying the method on datasets from a healthy subject and a patient with edema yielded corrected DTI indices and a more complete tract reconstruction that passed next to the ventricles and through the edema. We were able to segment the edema into areas according to the condition of the underlying tissue. In addition, the volume of free water is suggested as a new quantitative contrast of diffusion MRI. The findings suggest that free water is not limited to the borders of the brain parenchyma; it therefore contributes to the architecture surrounding neuronal bundles and may indicate specific anatomical processes. The analysis requires a conventional DTI acquisition and can be easily merged with existing DTI pipelines. Magn Reson Med, 2009.

Integrated active contours for texture segmentation

We address the issue of textured image segmentation in the context of the Gabor feature space of images. Gabor filters tuned to a set of orientations, scales and frequencies are applied to the images to create the Gabor feature space. A two-dimensional Riemannian manifold of local features is extracted via the Beltrami framework. The metric of this surface provides a good indicator of texture changes and is used, therefore, in a Beltrami-based diffusion mechanism and in a geodesic active contours algorithm for texture segmentation. The performance of the proposed algorithm is compared with that of the edgeless active contours algorithm applied for texture segmentation. Moreover, an integrated approach, extending the geodesic and edgeless active contours approaches to texture segmentation, is presented. We show that combining boundary and region information yields more robust and accurate texture segmentation results.

Variational denoising of partly textured images by spatially varying constraints

Denoising algorithms based on gradient dependent regularizers, such as nonlinear diffusion processes and total variation denoising, modify images towards piecewise constant functions. Although edge sharpness and location is well preserved, important information, encoded in image features like textures or certain details, is often compromised in the process of denoising. We propose a mechanism that better preserves fine scale features in such denoising processes. A basic pyramidal structure-texture decomposition of images is presented and analyzed. A first level of this pyramid is used to isolate the noise and the relevant texture components in order to compute spatially varying constraints based on local variance measures. A variational formulation with a spatially varying fidelity term controls the extent of denoising over image regions. Our results show visual improvement as well as an increase in the signal-to-noise ratio over scalar fidelity term processes. This type of processing can be used for a variety of tasks in partial differential equation-based image processing and computer vision, and is stable and meaningful from a mathematical viewpoint

From High Energy Physics to Low Level Vision

A geometric framework for image scale space, enhancement, and segmentation is presented. We consider intensity images as surfaces in the (x, I) space. The image is thereby a 2D surface in 3D space for gray level images, and a 2D surface in 5D for color images. The new formulation unifies many classical schemes and algorithms via a simple scaling of the intensity contrast, and results in new and efficient schemes. Extensions to multi dimensional signals become natural and lead to powerful denoising and scale space algorithms. Here, we demonstrate the proposed framework by applying it to denoise and improve gray level and color images.

Towards recognition-based variational segmentation using shape priors and dynamic labeling

We propose a novel variational approach based on a level set formulation of the Mumford-Shah functional and shape priors. We extend the functional by a labeling function which indicates image regions in which the shape prior is enforced. By minimizing the proposed functional with respect to both the level set function and the labeling function, the algorithm selects image regions where it is favorable to enforce the shape prior. By this, the approach permits to segment multiple independent objects in an image, and to discriminate familiar objects from unfamiliar ones by means of the labeling function. Numerical results demonstrate the performance of our approach.

A Multiphase Dynamic Labeling Model for Variational Recognition-driven Image Segmentation

We propose a variational framework for the integration of multiple competing shape priors into level set based segmentation schemes. By optimizing an appropriate cost functional with respect to both a level set function and a (vector-valued) labeling function, we jointly generate a segmentation (by the level set function) and a recognition-driven partition of the image domain (by the labeling function) which indicates where to enforce certain shape priors. Our framework fundamentally extends previous work on shape priors in level set segmentation by directly addressing the central question of where to apply which prior. It allows for the seamless integration of numerous shape priors such that—while segmenting both multiple known and unknown objects—the level set process may selectively use specific shape knowledge for simultaneously enhancing segmentation and recognizing shape.