Patrick L. Combettes

SIGNAL RECOVERY BY PROXIMAL FORWARD-BACKWARD SPLITTING ∗

We show that various inverse problems in signal recovery can be formulated as the generic problem of minimizing the sum of two convex functions with certain regularity properties. This formulation makes it possible to derive existence, uniqueness, characterization, and stability results in a unified and standardized fashion for a large class of apparently disparate problems. Recent results on monotone operator splitting methods are applied to establish the convergence of a forward-backward algorithm to solve the generic problem. In turn, we recover, extend, and provide a simplified analysis for a variety of existing iterative methods. Applications to geometry/texture image decomposition schemes are also discussed. A novelty of our framework is to use extensively the notion of a proximity operator, which was introduced by Moreau in the 1960s.

Proximal Splitting Methods in Signal Processing

The proximity operator of a convex function is a natural extension of the notion of a projection operator onto a convex set. This tool, which plays a central role in the analysis and the numerical solution of convex optimization problems, has recently been introduced in the arena of inverse problems and, especially, in signal processing, where it has become increasingly important. In this paper, we review the basic properties of proximity operators which are relevant to signal processing and present optimization methods based on these operators. These proximal splitting methods are shown to capture and extend several well-known algorithms in a unifying framework. Applications of proximal methods in signal recovery and synthesis are discussed.

The foundations of set theoretic estimation

Explains set theoretic estimation, which is governed by the notion of feasibility and produces solutions whose sole property is to be consistent with all information arising from the observed data and a priori knowledge. Each piece of information is associated with a set in the solution space, and the intersection of these sets, the feasibility set, represents the acceptable solutions. The practical use of the set theoretic framework stems from the existence of efficient techniques for finding these solutions. Many scattered problems in systems science and signal processing have been approached in set theoretic terms over the past three decades. The author synthesizes a single, general framework from these various approaches, examines its fundamental philosophy, goals, and analytical techniques, and relates it to conventional methods. >

A Douglas–Rachford Splitting Approach to Nonsmooth Convex Variational Signal Recovery

Under consideration is the large body of signal recovery problems that can be formulated as the problem of minimizing the sum of two (not necessarily smooth) lower semicontinuous convex functions in a real Hilbert space. This generic problem is analyzed and a decomposition method is proposed to solve it. The convergence of the method, which is based on the Douglas-Rachford algorithm for monotone operator-splitting, is obtained under general conditions. Applications to non-Gaussian image denoising in a tight frame are also demonstrated.

Solving monotone inclusions via compositions of nonexpansive averaged operators

A unified fixed point theoretic framework is proposed to investigate the asymptotic behavior of algorithms for finding solutions to monotone inclusion problems. The basic iterative scheme under consideration involves nonstationary compositions of perturbed averaged nonexpansive operators. The analysis covers proximal methods for common zero problems as well as for various splitting methods for finding a zero of the sum of monotone operators.A unified fixed point theoretic framework is proposed to investigate the asymptotic behavior of algorithms for finding solutions to monotone inclusion problems. The basic iterative scheme under consideration involves nonstationary compositions of perturbed averaged nonexpansive operators. The analysis covers proximal methods for common zero problems as well as for various splitting methods for finding a zero of the sum of monotone operators.

A Weak-to-Strong Convergence Principle for Fejé-Monotone Methods in Hilbert Spaces

We consider a wide class of iterative methods arising in numerical mathematics and optimization that are known to converge only weakly. Exploiting an idea originally proposed by Haugazeau, we present a simple modification of these methods that makes them strongly convergent without additional assumptions. Several applications are discussed.

The Convex Feasibility Problem in Image Recovery

Publisher Summary Image recovery is a broad discipline that encompasses the large body of inverse problems, in which an image h is to be inferred from the observation of data x consisting of signals physically or mathematically related to it. Image restoration and image reconstruction are the two main sub-branches of image recovery. The term “image restoration” usually applies to the problem of estimating the original form h of a degraded image x . The following four basic elements are required to solve an image recovery problem: (1) a data formation model, (2) a priori information, (3) a recovery criterion, and (4) a solution method. The recovery criterion defines the class of images that are acceptable as solutions to the problem. It is chosen by the user on grounds that may include experience, compatibility with the available a priori knowledge, personal convictions on the best way to solve the problem, and ease of implementation. The traditional approach has been to use a criterion of optimality, which usually leads to a single best solution. An alternative approach is to use a criterion of feasibility, in which consistency with all prior information and the data defines a set of equally acceptable solutions. The solution method is a numerical algorithm that will produce a solution to the recovery problem—that is, an image that satisfies the recovery criterion. Modification can be made in two directions: in the conventional image recovery framework, one seeks to preserve the notion of an optimal solution, whereas in the set theoretic framework the emphasis is placed on feasibility.

A proximal decomposition method for solving convex variational inverse problems

A broad range of inverse problems can be abstracted into the problem of minimizing the sum of several convex functions in a Hilbert space. We propose a proximal decomposition algorithm for solving this problem with an arbitrary number of nonsmooth functions and establish its weak convergence. The algorithm fully decomposes the problem in that it involves each function individually via its own proximity operator. A significant improvement over the methods currently in use in the area of inverse problems is that it is not limited to two nonsmooth functions. Numerical applications to signal and image processing problems are demonstrated.

Image restoration subject to a total variation constraint

Total variation has proven to be a valuable concept in connection with the recovery of images featuring piecewise smooth components. So far, however, it has been used exclusively as an objective to be minimized under constraints. In this paper, we propose an alternative formulation in which total variation is used as a constraint in a general convex programming framework. This approach places no limitation on the incorporation of additional constraints in the restoration process and the resulting optimization problem can be solved efficiently via block-iterative methods. Image denoising and deconvolution applications are demonstrated.

A variational formulation for frame-based inverse problems

A convex variational framework is proposed for solving inverse problems in Hilbert spaces with a priori information on the representation of the target solution in a frame. The objective function to be minimized consists of a separable term penalizing each frame coefficient individually, and a smooth term modelling the data formation model as well as other constraints. Sparsity-constrained and Bayesian formulations are examined as special cases. A splitting algorithm is presented to solve this problem and its convergence is established in infinite-dimensional spaces under mild conditions on the penalization functions, which need not be differentiable. Numerical simulations demonstrate applications to frame-based image restoration.