José Luis Morales

An interior algorithm for nonlinear optimization that combines line search and trust region steps

Abstract.An interior-point method for nonlinear programming is presented. It enjoys the flexibility of switching between a line search method that computes steps by factoring the primal-dual equations and a trust region method that uses a conjugate gradient iteration. Steps computed by direct factorization are always tried first, but if they are deemed ineffective, a trust region iteration that guarantees progress toward stationarity is invoked. To demonstrate its effectiveness, the algorithm is implemented in the Knitro [6,28] software package and is extensively tested on a wide selection of test problems.

Remark on “algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound constrained optimization”

This remark describes an improvement and a correction to Algorithm 778. It is shown that the performance of the algorithm can be improved significantly by making a relatively simple modification to the subspace minimization phase. The correction concerns an error caused by the use of routine dpmeps to estimate machine precision.

On the geometry phase in model-based algorithms for derivative-free optimization

A numerical study of model-based methods for derivative-free optimization is presented. These methods typically include a geometry phase whose goal is to ensure the adequacy of the interpolation set. The paper studies the performance of an algorithm that dispenses with the geometry phase altogether (and therefore does not attempt to control the position of the interpolation set). Data are presented describing the evolution of the condition number of the interpolation matrix and the accuracy of the gradient estimate. The experiments are performed on smooth unconstrained optimization problems with dimensions ranging between 2 and 15.

Enriched Methods for Large-Scale Unconstrained Optimization

This paper describes a class of optimization methods that interlace iterations of the limited memory BFGS method (L-BFGS) and a Hessian-free Newton method (HFN) in such a way that the information collected by one type of iteration improves the performance of the other. Curvature information about the objective function is stored in the form of a limited memory matrix, and plays the dual role of preconditioning the inner conjugate gradient iteration in the HFN method and of providing an initial matrix for L-BFGS iterations. The lengths of the L-BFGS and HFN cycles are adjusted dynamically during the course of the optimization. Numerical experiments indicate that the new algorithms are both effective and not sensitive to the choice of parameters.

An algorithm for the fast solution of symmetric linear complementarity problems

This paper studies algorithms for the solution of mixed symmetric linear complementarity problems. The goal is to compute fast and approximate solutions of medium to large sized problems, such as those arising in computer game simulations and American options pricing. The paper proposes an improvement of a method described by Kocvara and Zowe (Numer Math 68:95–106, 1994) that combines projected Gauss–Seidel iterations with subspace minimization steps. The proposed algorithm employs a recursive subspace minimization designed to handle severely ill-conditioned problems. Numerical tests indicate that the approach is more efficient than interior-point and gradient projection methods on some physical simulation problems that arise in computer game scenarios.

Robust and efficient density fitting

In this paper we propose an iterative method for solving the inhomogeneous systems of linear equations associated with density fitting. The proposed method is based on a version of the conjugate gradient method that makes use of automatically built quasi-Newton preconditioners. The paper gives a detailed description of a parallel implementation of the new method. The computational performance of the new algorithms is analyzed by benchmark calculations on systems with up to about 35,000 auxiliary functions. Comparisons with the standard, direct approach show no significant differences in the computed solutions.

Algorithm 809: PREQN: Fortran 77 subroutines for preconditioning the conjugate gradient method

PREQN is a package of Fortran 77 subroutins for automatically generating preconditioners for the conjugate gradient method. It is designed for solving a sequence of linear systems <italic>A<subscrpt>i</subscrpt>x</italic> = <italic>b<subscrpt>i</subscrpt></italic>, <italic>i</italic> = 1…, <italic>t</italic>, where the coefficient matrices <italic>A<subscrpt>i<subscrpt></italic> are symmetric and positive definite and vary slowly. Problems of this type arise, for example, in nonlinear optimization. The preconditioners are based on limited-memory quasi-Newton updating and are recommended for problems in which (i) the coefficient matrices are not explicitly known and only matrix-vector products of the form <italic>A<subscrpt>i</subscrpt>v</italic> can be computed; or (ii) the coefficient matrices are not sparse. PREQN is written so that a single call from a conjugate gradient routine performs the preconditioning operation and stores information needed for the generation of a new preconditioner.

On the solution of complementarity problems arising in American options pricing

In the Black–Scholes–Merton model, as well as in more general stochastic models in finance, the price of an American option solves a parabolic variational inequality. When the variational inequality is discretized, one obtains a linear complementarity problem (LCP) that must be solved at each time step. This paper presents an algorithm for the solution of these types of LCPs that is significantly faster than the methods currently used in practice. The new algorithm is a two-phase method that combines the active-set identification properties of the projected successive over relaxation (SOR) iteration with the second-order acceleration of a (recursive) reduced-space phase. We show how to design the algorithm so that it exploits the structure of the LCPs arising in these financial applications and present numerical results that show the effectiveness of our approach.

An Algorithm for Linear Complementarity and its Application in American Options Pricing

In this talk, we consider the numerical solution of American options pricing problems. Most options traded on options exchanges world-wide and a large fraction of options traded over-the-counter are of the American-style, including options on stocks of individual companies, stock indexes, foreign currencies, interest rates, commodities, and energy. Options books of a large financial institution may contain options on thousands of different underlying assets, and perhaps several dozen different contracts (with expiration dates ranging from days to years, and different strike prices). As the underlying asset prices change throughout the trading day, the options prices change as well. Re-pricing a large options book in real time may thus require re-computing thousands of options prices quickly. For such large scale applications, fast numerical algorithms are essential. When the prices of underlying assets are assumed to follow a diffusion process, such as in the classical BlackScholes-Merton model based on the geometric Brownian motion process, or in extensions such as Hestons stochastic volatihty model, the pricing function of an American-style option satisfies a system of parabolic partial differential variational inequahties. After this system is discretized in space and time, it yields a linear complementarity problem, which must be solved at each time step. Thus, the fast solution of linear complementarity problems (LCPs) is of great practical importance in computational finance. The most popular LCP method at present is the projected SOR iteration, or the closely related variant, the projected Gauss-Seidel iteration [2]. The standard treatment of LCPs for American option pricing can be found, for example, in [8] for the simple case of the Black-Scholes-Merton model and in [4] for several more complicated settings. Several new active-set methods [1,7] have recently been proposed for solving these LCPs more efficiently. Some of the most promising results are reported by Borici and Luethi [1], who developed a variant of the simplex-like method for LCPs with Z-matrices [2]. We argue in this talk that much greater speedups can be obtained with an algorithm that combines iterations of the projected Gauss-Seidel (or SOR) method with reduced-space steps. This two-phase approach exploits the fact that the projected Gauss-Seidel iteration often makes a quick estimation of the optimal active set, while the reduced-space iteration can dramatically improve upon this estimate and yield a fast rate of convergence. We illustrate the performance of this algorithm on both the Black-Scholes-Merton model (using various values of volatility and maturity) and the Heston model [3] with stochastic volatility. Let us begin by describing the Black-Scholes-Merton model. Consider an American put option with strike price K > 0 and maturity time T > 0. If the option is exercised when the underlying asset price is S, the option holder receives the payoff W{S) = {K -S)^ = max{K -S,0). Similarly, the payoff function for an American call option is ^{S) = {SK)+. Let V{t, S) be the option value at time t G [0, T] when the asset price is S. We assume that V solves the following partial differential variational inequahty (see, e.g., [5]):