Emilie Chouzenoux

A block coordinate variable metric forward---backward algorithm

A number of recent works have emphasized the prominent role played by the Kurdyka-Łojasiewicz inequality for proving the convergence of iterative algorithms solving possibly nonsmooth/nonconvex optimization problems. In this work, we consider the minimization of an objective function satisfying this property, which is a sum of two terms: (i) a differentiable, but not necessarily convex, function and (ii) a function that is not necessarily convex, nor necessarily differentiable. The latter function is expressed as a separable sum of functions of blocks of variables. Such an optimization problem can be addressed with the Forward–Backward algorithm which can be accelerated thanks to the use of variable metrics derived from the Majorize–Minimize principle. We propose to combine the latter acceleration technique with an alternating minimization strategy which relies upon a flexible update rule. We give conditions under which the sequence generated by the resulting Block Coordinate Variable Metric Forward–Backward algorithm converges to a critical point of the objective function. An application example to a nonconvex phase retrieval problem encountered in signal/image processing shows the efficiency of the proposed optimization method.

A Majorize-Minimize Subspace Approach for

In this work, we consider a class of differentiable criteria for sparse image computing problems, where a nonconvex regularization is applied to an arbitrary linear transform of the target image. As special cases, it includes edge-preserving measures or frame-analysis potentials commonly used in image processing. As shown by our asymptotic results, the l2-l0 penalties we consider may be employed to provide approximate solutions to l0-penalized optimization problems. One of the advantages of the proposed approach is that it allows us to derive an efficient Majorize-Minimize subspace algorithm. The convergence of the algorithm is investigated by using recent results in nonconvex optimization. The fast convergence properties of the proposed optimization method are illustrated through image processing examples. In particular, its effectiveness is demonstrated on several data recovery problems.

\ell_2-\ell_0

The ℓ₁/ℓ₂ ratio regularization function has shown good performance for retrieving sparse signals in a number of recent works, in the context of blind deconvolution. Indeed, it benefits from a scale invariance property much desirable in the blind context. However, the ℓ₁/ℓ₂ function raises some difficulties when solving the nonconvex and nonsmooth minimization problems resulting from the use of such a penalty term in current restoration methods. In this paper, we propose a new penalty based on a smooth approximation to the ℓ₁/ℓ₂ function. In addition, we develop a proximal-based algorithm to solve variational problems involving this function and we derive theoretical convergence results. We demonstrate the effectiveness of our method through a comparison with a recent alternating optimization strategy dealing with the exact ℓ₁/ℓ₂ term, on an application to seismic data blind deconvolution.

Image Regularization

This paper deals with the reconstruction of T1-T2 correlation spectra in nuclear magnetic resonance relaxometry. The ill-posed character and the large size of this inverse problem are the main difficulties to tackle. While maximum entropy is retained as an adequate regularization approach, the choice of an efficient optimization algorithm remains a challenging task. Our proposal is to apply a truncated Newton algorithm with two original features. First, a theoretically sound line search strategy suitable for the entropy function is applied to ensure the convergence of the algorithm. Second, an appropriate preconditioning structure based on a singular value decomposition of the forward model matrix is used to speed up the algorithm convergence. Furthermore, we exploit the specific structures of the observation model and the Hessian of the criterion to reduce the computation cost of the algorithm. The performances of the proposed strategy are illustrated by means of synthetic and real data processing.

Euclid in a Taxicab: Sparse Blind Deconvolution with Smoothed

Modern signal processing (SP) methods rely very heavily on probability and statistics to solve challenging SP problems. SP methods are now expected to deal with ever more complex models, requiring ever more sophisticated computational inference techniques. This has driven the development of statistical SP methods based on stochastic simulation and optimization. Stochastic simulation and optimization algorithms are computationally intensive tools for performing statistical inference in models that are analytically intractable and beyond the scope of deterministic inference methods. They have been recently successfully applied to many difficult problems involving complex statistical models and sophisticated (often Bayesian) statistical inference techniques. This survey paper offers an introduction to stochastic simulation and optimization methods in signal and image processing. The paper addresses a variety of high-dimensional Markov chain Monte Carlo (MCMC) methods as well as deterministic surrogate methods, such as variational Bayes, the Bethe approach, belief and expectation propagation and approximate message passing algorithms. It also discusses a range of optimization methods that have been adopted to solve stochastic problems, as well as stochastic methods for deterministic optimization. Subsequently, areas of overlap between simulation and optimization, in particular optimization-within-MCMC and MCMC-driven optimization are discussed.

{\ell _1}/{\ell _2}

This paper proposes accelerated subspace optimization methods in the context of image restoration. Subspace optimization methods belong to the class of iterative descent algorithms for unconstrained optimization. At each iteration of such methods, a stepsize vector allowing the best combination of several search directions is computed through a multidimensional search. It is usually obtained by an inner iterative second-order method ruled by a stopping criterion that guarantees the convergence of the outer algorithm. As an alternative, we propose an original multidimensional search strategy based on the majorize-minimize principle. It leads to a closed-form stepsize formula that ensures the convergence of the subspace algorithm whatever the number of inner iterations. The practical efficiency of the proposed scheme is illustrated in the context of edge-preserving image restoration.

Regularization

The Poisson-Gaussian model can accurately describe the noise present in a number of imaging systems. However most existing restoration methods rely on approximations of the Poisson-Gaussian noise statistics. We propose a convex optimization strategy for the reconstruction of images degraded by a linear operator and corrupted with a mixed Poisson-Gaussian noise. The originality of our approach consists of considering the exact, mixed continuous-discrete model corresponding to the data statistics. After establishing the Lipschitz differentiability and convexity of the Poisson-Gaussian neg-log-likelihood, we derive a primal-dual iterative scheme for minimizing the associated penalized criterion. The proposed method is applicable to a large choice of convex penalty terms. The robustness of our scheme allows us to handle computational difficulties due to infinite sums arising from the computation of the gradient of the criterion. We propose finite bounds for these sums, that are dependent on the current image estimate, and thus adapted to each iteration of our algorithm. The proposed approach is validated on image restoration examples. Then, the exact data fidelity term is used as a reference for studying some of its various approximations. We show that in a variational framework the Shifted Poisson and Exponential approximations lead to very good restoration results.

Efficient Maximum Entropy Reconstruction of Nuclear Magnetic Resonance T1-T2 Spectra

A Poisson-Gaussian model accurately describes the noise present in many imaging systems such as CCD cameras or fluorescence microscopy. However most existing restoration strategies rely on approximations of the Poisson-Gaussian noise statistics. We propose a convex optimization algorithm for the reconstruction of signals degraded by a linear operator and corrupted with mixed Poisson-Gaussian noise. The originality of our approach consists of considering the exact continuous-discrete model corresponding to the data statistics. After establishing the Lipschitz differentiability of the Poisson-Gaussian log-likelihood, we derive a primal-dual iterative scheme for minimizing the associated penalized criterion. The proposed method is applicable to a large choice of penalty terms. The robustness of our scheme allows us to handle computational difficulties due to infinite sums arising from the computation of the gradient of the criterion. The proposed approach is validated on image restoration examples.

A Survey of Stochastic Simulation and Optimization Methods in Signal Processing

Complex-valued data are encountered in many application areas of signal and image processing. In the context of the optimization of functions of real variables, subspace algorithms have recently attracted much interest, owing to their efficiency for solving large-size problems while simultaneously offering theoretical convergence guarantees. The goal of this paper is to show how some of these methods can be successfully extended to the complex case. More precisely, we investigate the properties of the proposed complex-valued Majorize-Minimize Memory Gradient (3MG) algorithm. Important practical applications of these results arise in inverse problems. Here, we focus on image reconstruction in Parallel Magnetic Resonance Imaging (PMRI). The linear operator involved in the observation model then includes a subsampling operator over the k-space (2D Fourier domain) the choice of which is analyzed through our numerical results. In addition, sensitivity matrices associated with the multiple channel coils come into play. Comparisons with existing optimization methods confirm the better performance of the proposed algorithm. HighlightsAn extension of the Majorize-Minimize Memory Gradient algorithm for minimizing functions of complex variables is proposed.The convergence of the algorithm is proved under weak assumptions.A novel application of the algorithm to parallel Magnetic Resonance Imaging reconstruction is proposed.Through simulations on real data, the algorithm is shown to outperform recent optimization strategies in terms of convergence speed.The algorithm can handle various subsampling schemes, both convex and nonconvex penalization functions and different possibly redundant frame representations.

A Majorize–Minimize Strategy for Subspace Optimization Applied to Image Restoration

Hyperspectral data unmixing aims at identifying the components (endmembers) of an observed surface and at determining their fractional abundances inside each pixel area. Assuming that the spectral signatures of the surface components have been previously determined by an endmember extraction algorithm, or to be part of an available spectral library, the main problem is reduced to the estimation of the fractional abundances. For large hyperspectral image data sets, the estimation of the abundance maps requires the resolution of a large-scale optimization problem subject to linear constraints such as non-negativity and sum less or equal to one. This paper proposes a primal-dual interior-point optimization algorithm allowing a constrained least squares estimation approach. In comparison with existing methods, the proposed algorithm is more flexible since it can handle any linear equality and/or inequality constraint and has the advantage of a reduced computational cost. It also presents an algorithmic structure suitable for a parallel implementation on modern intensive computing devices such as Graphics Processing Units (GPU). The implementation issues are discussed and the applicability of the proposed approach is illustrated with the help of examples on synthetic and real hyperspectral data.