Coloma Ballester

Filling-in by joint interpolation of vector fields and gray levels

A variational approach for filling-in regions of missing data in digital images is introduced. The approach is based on joint interpolation of the image gray levels and gradient/isophotes directions, smoothly extending in an automatic fashion the isophote lines into the holes of missing data. This interpolation is computed by solving the variational problem via its gradient descent flow, which leads to a set of coupled second order partial differential equations, one for the gray-levels and one for the gradient orientations. The process underlying this approach can be considered as an interpretation of the Gestaltists principle of good continuation. No limitations are imposed on the topology of the holes, and all regions of missing data can be simultaneously processed, even if they are surrounded by completely different structures. Applications of this technique include the restoration of old photographs and removal of superimposed text like dates, subtitles, or publicity. Examples of these applications are given. We conclude the paper with a number of theoretical results on the proposed variational approach and its corresponding gradient descent flow.

A Variational Model for P+XS Image Fusion

We propose an algorithm to increase the resolution of multispectral satellite images knowing the panchromatic image at high resolution and the spectral channels at lower resolution. Our algorithm is based on the assumption that, to a large extent, the geometry of the spectral channels is contained in the topographic map of its panchromatic image. This assumption, together with the relation of the panchromatic image to the spectral channels, and the expression of the low-resolution pixel in terms of the high-resolution pixels given by some convolution kernel followed by subsampling, constitute the elements for constructing an energy functional (with several variants) whose minima will give the reconstructed spectral images at higher resolution. We discuss the validity of the above approach and describe our numerical procedure. Finally, some experiments on a set of multispectral satellite images are displayed.

A TV Based Restoration Model with Local Constraints

Abstract We propose in this paper a total variation based restoration model which incorporates the image acquisition model z=h*U+n (where z represents the observed sampled image, U is the ideal undistorted image, h denotes the blurring kernel and n is a white Gaussian noise) as a set of local constraints. These constraints, one for each pixel of the image, express the fact that the variance of the noise can be estimated from the residuals z−h*U if we use a neighborhood of each pixel. This is motivated by the fact that the usual inclusion of the image acquisition model as a single constraint expressing a bound for the variance of the noise does not give satisfactory results if we wish to simultaneously recover textured regions and obtain a good denoising of the image. We use Uzawa’s algorithm to minimize the total variation subject to the proposed family of local constraints and we display some experiments using this model.

A variational model for filling-in gray level and color images

A variational approach for filling-in regions of missing data in gray-level and color images is introduced in this paper. The approach is based on joint interpolation of the image gray-levels and gradient/isophores directions, smoothly extending in an automatic fashion the isophote lines into the holes of missing data. This interpolation is computed solving the variational problem via its gradient descent flow, which lends to a set of coupled second order partial differential equations, one for the gray-levels and one for the gradient orientations. The process underlying this approach can be considered as an interpretation of the Gestaltists principle of good continuation. No limitations are imposed on the topology of the holes, and all regions of missing data can be simultaneously processed, even if they are surrounded by completely different structures. Applications of this technique include the restoration of old photographs and removal of superimposed text like dates, subtitles, or publicity. Examples of these applications are given.

MINIMIZING TOTAL VARIATION FLOW

We prove existence and uniqueness of weak solutions for the minimizing Total Variation flow with initial data in L1 under Neumann boundary conditions. We prove that the HN−1 measure of the boundaries of level sets of the solution decreases with time, as one would expect. We also prove that local maxima (minima) strictly decrease (increase) their level with time. We shall also consider the Dirichlet problem which presents some particular difficulties for general initial data in L1.

Level Lines Selection with Variational Models for Segmentation and Encoding

This paper discusses the interest of the Tree of Shapes of an image as a region oriented image representation. The Tree of Shapes offers a compact and structured representation of the family of level lines of an image. This representation has been used for many processing tasks such as filtering, registration, or shape analysis. In this paper we show how this representation can be used for segmentation, rate distortion optimization, and encoding. We address the problem of segmentation and rate distortion optimization using Guigues algorithm on a hierarchy of partitions constructed using the simplified Mumford-Shah multiscale energy. To segment an image, we minimize the simplified Mumford-Shah energy functional on the set of partitions represented in this hierarchy. The rate distortion problem is also solved in this hierarchy of partitions. In the case of encoding, we propose a variational model to select a family of level lines of a gray level image in order to obtain a minimal description of it. Our energy functional represents the cost in bits of encoding the selected level lines while controlling the maximum error of the reconstructed image. In this case, a greedy algorithm is used to minimize the corresponding functional. Some experiments are displayed.

Affine Invariant Texture Segmentation and Shape from Texture by Variational Methods

We address the problem of texture segmentation by using a novel affine invariant model. The introduction of affine invariance as a requirement for texture analysis goes beyond what is known of the human performance and also beyond the psychophysical theories. We propose to compute texture features using affine invariant intrinsic neighborhoods and affine invariant intrinsic orientation matrices. We discuss several possibilities for the definition of the channels and give comparative experimental results where an affine invariant Mumford-Shah type energy functional is used to compute the multichannel affine invariant segmentation. We prove that the method is able to retrieve faithfully the texture regions and to recover the shape from texture information in images where several textures are present. The numerical algorithm is multiscale.

An Inpainting- Based Deinterlacing Method

Video is usually acquired in interlaced format, where each image frame is composed of two image fields, each field holding same parity lines. However, many display devices require progressive video as input; also, many video processing tasks perform better on progressive material than on interlaced video. In the literature, there exist a great number of algorithms for interlaced to progressive video conversion, with a great tradeoff between the speed and quality of the results. The best algorithms in terms of image quality require motion compensation; hence, they are computationally very intensive. In this paper, we propose a novel de interlacing algorithm based on ideas from the image in painting arena. We view the lines to interpolate as gaps that we need to in paint. Numerically, this is implemented using a dynamic programming procedure, which ensures a complexity of O(S), where S is the number of pixels in the image. The results obtained with our algorithm compare favorably, in terms of image quality, with state-of-the-art methods, but at a lower computational cost, since we do not need to perform motion field estimation.

Affine invariant segmentation by variational method

Affine invariance, viewed as a simplified projective invariance, is an essential requirement for analysis of natural scenes. We propose an affine invariant analogue of Mumford and Shah energy functional to segment images and we discuss it. We present a simple multiscale algorithm to minimize it, based on region growing methods, and we display our first numerical experiments.

Disocclusion by Joint Interpolation of Vector Fields and Gray Levels

In this paper we study a variational approach for filling in regions of missing data in two-dimensional and three-dimensional digital images. Applications of this technique include the restoration of old photographs and removal of superimposed text like dates, subtitles, or publicity, or the zooming of images. The approach presented here, initially introduced in [IEEE Trans. Image Process., 10 (2001), pp. 1200--1211] is based on a joint interpolation of the image gray levels and gradient/isophotes directions, smoothly extending the isophote lines into the holes of missing data. The process underlying this approach can be considered as an interpretation of the Gestaltists principle of good continuation. We study the existence of minimizers of our functional and its approximation by minima of smoother functionals. Then we present the numerical algorithm used to minimize it and display some numerical experiments.