Nélida E. Echebest

Active-set strategy in Powell's method for optimization without derivatives

In this article we present an algorithm for solving bound constrained optimization problems without derivatives based on Powells method [38] for derivative-free optimization. First we consider the unconstrained optimization problem. At each iteration a quadratic interpolation model of the objective function is constructed around the current iterate and this model is minimized to obtain a new trial point. The whole process is embedded within a trust-region framework. Our algorithm uses infinity norm instead of the Euclidean norm and we solve a box constrained quadratic subproblem using an active-set strategy to explore faces of the box. Therefore, a bound constrained optimization algorithm is easily extended. We compare our im_ plementation with NEWUOA and BOBYQA, Powells algorithms for unconstrained and bound constrained derivative free optimization respectively. Numerical experiments show that, in general, our algorithm require less functional evaluations than Powells algorithms.

A class of optimized row projection methods for solving large nonsymmetric linear systems

We present in this paper optimal and accelerated row projection algorithms arising from the use of quadratic programming, that allow us to define the iterate xk+1 as the projection of xk onto a hyperplane which minimizes its distance to the solution x*. These algorithms also use a novel partition strategy into blocks based on sequential estimations of their condition numbers.

Inexact Restoration method for nonlinear optimization without derivatives

A derivative-free optimization method is proposed for solving a general nonlinear programming problem. It is assumed that the derivatives of the objective function and the constraints are not available. The new method is based on the Inexact Restoration scheme, where each iteration is decomposed in two phases. In the first one, the violation of the feasibility is reduced. In the second one, the objective function is minimized onto a linearization of the nonlinear constraints. At both phases, polynomial interpolation models are used in order to approximate the objective function and the constraints. At the first phase a derivative-free solver for box constrained optimization can be used. For the second phase, we propose a new method ad-hoc based on trust-region strategy that uses the projection of the simplex gradient on the tangent space. Under suitable assumptions, the algorithm is well defined and convergence results are proved. A numerical implementation is described and numerical experiments are presented to validate the theoretical results.

Acceleration Scheme for Parallel Projected Aggregation Methods for Solving Large Linear Systems

The Projected Aggregation methods generate the new point xk+1 as the projection of xk onto an “aggregate” hyperplane usually arising from linear combinations of the hyperplanes defined by the blocks. The aim of this paper is to improve the speed of convergence of a particular kind of them by projecting the directions given by the blocks onto the aggregate hyperplane defined in the last iteration. For that purpose we apply the scheme introduced in “A new method for solving large sparse systems of linear equations using row projections” [11], for a given block projection algorithm, to some new methods here introduced whose main features are related to the fact that the projections do not need to be accurately computed. Adaptative splitting schemes are applied which take into account the structure and conditioning of the matrix. It is proved that these new highly parallel algorithms improve the original convergence rate and present numerical results which show their computational efficiency.

An Acceleration Scheme for Solving Convex Feasibility Problems Using Incomplete Projection Algorithms

The Projected Aggregation Methods (PAM) for solving linear systems of equalities and/or inequalities, generate a new iterate xk+1 by projecting the current point xk onto a separating hyperplane generated by a given linear combination of the original hyperplanes or halfspaces. In [12] we introduced acceleration schemes for solving systems of linear equations by applying optimization techniques to the problem of finding the optimal combination of the hyperplanes within a PAM like framework. In this paper we generalize those results, introducing a new accelerated iterative method for solving systems of linear inequalities, together with the corresponding theoretical convergence results. In order to test its efficiency, numerical results obtained applying the new acceleration scheme to two algorithms introduced by García-Palomares and González-Castaño [6] are given.

An Accelerated Iterative Method with Diagonally Scaled Oblique Projections for Solving Linear Feasibility Problems

The Projected Aggregation Methods (PAM) for solving linear systems of equalities and/or inequalities, generate a new iterate xk+1 by projecting the current point xk onto a separating hyperplane generated by a given linear combination of the original hyperplanes or halfspaces. In Scolnik et al. (2001, 2002a) and Echebest et al. (2004) acceleration schemes for solving systems of linear equations and inequalities respectively were introduced, within a PAM like framework. In this paper we apply those schemes in an algorithm based on oblique projections reflecting the sparsity of the matrix of the linear system to be solved. We present the corresponding theoretical convergence results which are a generalization of those given in Echebest et al. (2004). We also present the numerical results obtained applying the new scheme to two algorithms introduced by Garcí a-Palomares and González-Castaño (1998) and also the comparison of its efficiency with that of Censor and Elfving (2002).

New Optimized and Accelerated Pam Methods for Solving Large Non-Symmetric Linear Systems: Theory and Practice

The Projected Aggregation Methods generate the new point x k+1 as the projection of x k onto an ” aggregate” hyperplane usually arising from linear combinations of the hyperplanes planes defined by the blocks. In [13] an acceleration scheme was introduced for algorithms in which an optimized search direction arises from the solution of small quadratic subproblems. In this paper we extend that theory to classical methods like Cimminos and to the generalized convex combination as defined in [5]. We prove that the resulting new highly parallel, algorithms improve the original convergence rate and present numerical results which show their outstanding computational efficiency.

Two derivative-free methods for solving underdetermined nonlinear systems of equations

In this paper, two different approaches to solve underdetermined nonlinear system of equations are proposed. In one of them, the derivative-free method defined by La Cruz, Martinez and Raydan for solving square nonlinear systems is modified and extended to cope with the underdetermined case. The other approach is a Quasi-Newton method that uses the Broyden update formula and the globalized line search that combines the strategy of Grippo, Lampariello and Lucidi with the Li and Fukushima one. Global convergence results for both methods are proved and numerical experiments are presented.

Incomplete oblique projections method for solving regularized least-squares problems in image reconstruction

In this paper we improve on the incomplete oblique projections (IOP) method introduced previously by the authors for solving inconsistent linear systems, when applied to image reconstruction problems. That method uses IOP onto the set of solutions of the augmented system Ax−r=b, and converges to a weighted least-squares solution of the system Ax=b. In image reconstruction problems, systems are usually inconsistent and very often rank-deficient because of the underlying discretized model. Here we have considered a regularized least-squares objective function that can be used in many ways such as incorporating blobs or nearest-neighbor interactions among adjacent pixels, aiming at smoothing the image. Thus, the oblique incomplete projections algorithm has been modified for solving this regularized model. The theoretical properties of the new algorithm are analyzed and numerical experiments are presented showing that the new approach improves the quality of the reconstructed images.

Implicit regularization of the incomplete oblique projections method

The aim of this paper is to improve the performance of the incomplete oblique projections method (IOP), previously introduced by the authors for solving inconsistent linear systems, when applied to image reconstruction problems. That method employs incomplete oblique projections onto the set of solutions of the augmented system Ax−r=b, and converges to a weighted least squares solution of the system Ax=b. Many tomographic image reconstruction problems are such that the limitation of the range of rays makes the model underdetermined, the discretized linear system is rank-deficient, the nullspace is non-trivial, and the minimal norm least squares solution may be far away from the true image. In a previous paper, we have added a quadratic term reflecting neighboring pixel information to the standard least squares model for improving the quality of the reconstructed images. In this paper we replace the quadratic function by a more general regularizing function avoiding the modification of the original system. The key idea is to perform a joint optimization of the norm of the residual and of the regularizing function in each iteration. The theoretical properties of this new algorithm are analyzed, and numerical experiments are presented comparing its performance with other well-known methods. They show that the new approach improves the quality of the reconstructed images.