Guy Gilboa

Nonlocal Operators with Applications to Image Processing

We propose the use of nonlocal operators to define new types of flows and functionals for image processing and elsewhere. A main advantage over classical PDE-based algorithms is the ability to handle better textures and repetitive structures. This topic can be viewed as an extension of spectral graph theory and the diffusion geometry framework to functional analysis and PDE-like evolutions. Some possible applications and numerical examples are given, as is a general framework for approximating Hamilton–Jacobi equations on arbitrary grids in high demensions, e.g., for control theory.

Structure-Texture Image Decomposition--Modeling, Algorithms, and Parameter Selection

This paper explores various aspects of the image decomposition problem using modern variational techniques. We aim at splitting an original image f into two components u and ρ, where u holds the geometrical information and ρ holds the textural information. The focus of this paper is to study different energy terms and functional spaces that suit various types of textures. Our modeling uses the total-variation energy for extracting the structural part and one of four of the following norms for the textural part: L2, G, L1 and a new tunable norm, suggested here for the first time, based on Gabor functions. Apart from the broad perspective and our suggestions when each model should be used, the paper contains three specific novelties: first we show that the correlation graph between u and ρ may serve as an efficient tool to select the splitting parameter, second we propose a new fast algorithm to solve the TV − L1 minimization problem, and third we introduce the theory and design tools for the TV-Gabor model.

Image enhancement and denoising by complex diffusion processes

The linear and nonlinear scale spaces, generated by the inherently real-valued diffusion equation, are generalized to complex diffusion processes, by incorporating the free Schrodinger equation. A fundamental solution for the linear case of the complex diffusion equation is developed. Analysis of its behavior shows that the generalized diffusion process combines properties of both forward and inverse diffusion. We prove that the imaginary part is a smoothed second derivative, scaled by time, when the complex diffusion coefficient approaches the real axis. Based on this observation, we develop two examples of nonlinear complex processes, useful in image processing: a regularized shock filter for image enhancement and a ramp preserving denoising process.

Nonlocal Linear Image Regularization and Supervised Segmentation

A nonlocal quadratic functional of weighted differences is examined. The weights are based on image features and represent the affinity between different pixels in the image. By prescribing different formulas for the weights, one can generalize many local and nonlocal linear denoising algorithms, including the nonlocal means filter and the bilateral filter. In this framework one can easily show that continuous iterations of the generalized filter obey certain global characteristics and converge to a constant solution. The linear operator associated with the Euler–Lagrange equation of the functional is closely related to the graph Laplacian. We can thus interpret the steepest descent for minimizing the functional as a nonlocal diffusion process. This formulation allows a convenient framework for nonlocal variational minimizations, including variational denoising, Bregman iterations, and the recently proposed inverse scale space. It is also demonstrated how the steepest descent flow can be used for segmenta...

Forward-and-backward diffusion processes for adaptive image enhancement and denoising

Signal and image enhancement is considered in the context of a new type of diffusion process that simultaneously enhances, sharpens, and denoises images. The nonlinear diffusion coefficient is locally adjusted according to image features such as edges, textures, and moments. As such, it can switch the diffusion process from a forward to a backward (inverse) mode according to a given set of criteria. This results in a forward-and-backward (FAB) adaptive diffusion process that enhances features while locally denoising smoother segments of the signal or image. The proposed method, using the FAB process, is applied in a super-resolution scheme. The FAB method is further generalized for color processing via the Beltrami flow, by adaptively modifying the structure tensor that controls the nonlinear diffusion process. The proposed structure tensor is neither positive definite nor negative, and switches between these states according to image features. This results in a forward-and-backward diffusion flow where different regions of the image are either forward or backward diffused according to the local geometry within a neighborhood.

Variational denoising of partly textured images by spatially varying constraints

Denoising algorithms based on gradient dependent regularizers, such as nonlinear diffusion processes and total variation denoising, modify images towards piecewise constant functions. Although edge sharpness and location is well preserved, important information, encoded in image features like textures or certain details, is often compromised in the process of denoising. We propose a mechanism that better preserves fine scale features in such denoising processes. A basic pyramidal structure-texture decomposition of images is presented and analyzed. A first level of this pyramid is used to isolate the noise and the relevant texture components in order to compute spatially varying constraints based on local variance measures. A variational formulation with a spatially varying fidelity term controls the extent of denoising over image regions. Our results show visual improvement as well as an increase in the signal-to-noise ratio over scalar fidelity term processes. This type of processing can be used for a variety of tasks in partial differential equation-based image processing and computer vision, and is stable and meaningful from a mathematical viewpoint

Estimation of optimal PDE-based denoising in the SNR sense

This paper is concerned with finding the best partial differential equation-based denoising process, out of a set of possible ones. We focus on either finding the proper weight of the fidelity term in the energy minimization formulation or on determining the optimal stopping time of a nonlinear diffusion process. A necessary condition for achieving maximal SNR is stated, based on the covariance of the noise and the residual part. We provide two practical alternatives for estimating this condition by observing that the filtering of the image and the noise can be approximated by a decoupling technique, with respect to the weight or time parameters. Our automatic algorithm obtains quite accurate results on a variety of synthetic and natural images, including piecewise smooth and textured ones. We assume that the statistics of the noise were previously estimated. No a priori knowledge regarding the characteristics of the clean image is required. A theoretical analysis is carried out, where several SNR performance bounds are established for the optimal strategy and for a widely used method, wherein the variance of the residual part equals the variance of the noise

Regularized Shock Filters and Complex Diffusion

We address the issue of regularizing Osher and Rudins shock filter, used for image deblurring, in order to allow processes that are more robust against noise. Previous solutions to the problem suggested adding some sort of diffusion term to the shock equation. We analyze and prove some properties of coupled shock and diffusion processes. Finally we propose an original solution of adding a complex diffusion term to the shock equation. This new term is used to smooth out noise and indicate inflection points simultaneously. The imaginary value, which is an approximated smoothed second derivative scaled by time, is used to control the process. This results in a robust deblurring process that performs well also on noisy signals.

Nonlinear inverse scale space methods for image restoration

In this paper we generalize the iterated refinement method, introduced by the authors in [8],to a time-continuous inverse scale-space formulation. The iterated refinement procedure yields a sequence of convex variational problems, evolving toward the noisy image. The inverse scale space method arises as a limit for a penalization parameter tending to zero, while the number of iteration steps tends to infinity. For the limiting flow, similar properties as for the iterated refinement procedure hold. Specifically, when a discrepancy principle is used as the stopping criterion, the error between the reconstruction and the noise-free image decreases until termination, even if only the noisy image is available and a bound on the variance of the noise is known. The inverse flow is computed directly for one-dimensional signals, yielding high quality restorations. In higher spatial dimensions, we introduce a relaxation technique using two evolution equations. These equations allow accurate, efficient and straightforward implementation.

Constrained and SNR-Based Solutions for TV-Hilbert Space Image Denoising

We examine the general regularization model which is based on total-variation for the structural part and a Hilbert-space norm for the oscillatory part. This framework generalizes the Rudin-Osher-Fatemi and the Osher-Sole-Vese models and opens way for new denoising or decomposition methods with tunable norms, which are adapted to the nature of the noise or textures of the image. We give sufficient conditions and prove the convergence of an iterative numerical implementation, following Chambolle’s projection algorithm.In this paper we focus on the denoising problem. In order to provide an automatic solution, a systematic method for choosing the weight between the energies is imperative. The classical method for selecting the weight parameter according to the noise variance is reformulated in a Hilbert space sense. Moreover, we generalize a recent study of Gilboa-Sochen-Zeevi where the weight parameter is selected such that the denoised result is close to optimal, in the SNR sense. A broader definition of SNR, which is frequency weighted, is formulated in the context of inner products. A necessary condition for maximal SNR is provided. Lower and upper bounds on the SNR performance of the classical and optimal strategies are established, under quite general assumptions.