William W. Hager

Updating the inverse of a matrix

The Sherman–Morrison–Woodbury formulas relate the inverse of a matrix after a small-rank perturbation to the inverse of the original matrix. The history of these fomulas is presented and various applications to statistics, networks, structural analysis, asymptotic analysis, optimization, and partial differential equations are discussed. The Sherman-Morrison-Woodbury formulas express the inverse of a matrix after a small rank perturbation in terms of the inverse of the original matrix. This paper surveys the history of these formulas and we examine some applications where these formulas are helpful

Algorithm 887: CHOLMOD, Supernodal Sparse Cholesky Factorization and Update/Downdate

CHOLMOD is a set of routines for factorizing sparse symmetric positive definite matrices of the form A or AAT, updating/downdating a sparse Cholesky factorization, solving linear systems, updating/downdating the solution to the triangular system Lx = b, and many other sparse matrix functions for both symmetric and unsymmetric matrices. Its supernodal Cholesky factorization relies on LAPACK and the Level-3 BLAS, and obtains a substantial fraction of the peak performance of the BLAS. Both real and complex matrices are supported. CHOLMOD is written in ANSI/ISO C, with both C and MATLABTM interfaces. It appears in MATLAB 7.2 as x = A\b when A is sparse symmetric positive definite, as well as in several other sparse matrix functions.

Brief paper: A unified framework for the numerical solution of optimal control problems using pseudospectral methods

A unified framework is presented for the numerical solution of optimal control problems using collocation at Legendre-Gauss (LG), Legendre-Gauss-Radau (LGR), and Legendre-Gauss-Lobatto (LGL) points. It is shown that the LG and LGR differentiation matrices are rectangular and full rank whereas the LGL differentiation matrix is square and singular. Consequently, the LG and LGR schemes can be expressed equivalently in either differential or integral form, while the LGL differential and integral forms are not equivalent. Transformations are developed that relate the Lagrange multipliers of the discrete nonlinear programming problem to the costates of the continuous optimal control problem. The LG and LGR discrete costate systems are full rank while the LGL discrete costate system is rank-deficient. The LGL costate approximation is found to have an error that oscillates about the true solution and this error is shown by example to be due to the null space in the LGL discrete costate system. An example is considered to assess the accuracy and features of each collocation scheme.

A Nonmonotone Line Search Technique and Its Application to Unconstrained Optimization

A new nonmonotone line search algorithm is proposed and analyzed. In our scheme, we require that an average of the successive function values decreases, while the traditional nonmonotone approach of Grippo, Lampariello, and Lucidi [SIAM J. Numer. Anal., 23 (1986), pp. 707--716] requires that a maximum of recent function values decreases. We prove global convergence for nonconvex, smooth functions, and R-linear convergence for strongly convex functions. For the L-BFGS method and the unconstrained optimization problems in the CUTE library, the new nonmonotone line search algorithm used fewer function and gradient evaluations, on average, than either the monotone or the traditional nonmonotone scheme.

Runge-Kutta methods in optimal control and the transformed adjoint system

Summary. The convergence rate is determined for Runge-Kutta discretizations of nonlinear control problems. The analysis utilizes a connection between the Kuhn-Tucker multipliers for the discrete problem and the adjoint variables associated with the continuous minimum principle. This connection can also be exploited in numerical solution techniques that require the gradient of the discrete cost function.

Joint transceiver design for MIMO communications using geometric mean decomposition

In recent years, considerable attention has been paid to the joint optimal transceiver design for multi-input multi-output (MIMO) communication systems. In this paper, we propose a joint transceiver design that combines the geometric mean decomposition (GMD) with either the conventional zero-forcing VBLAST decoder or the more recent zero-forcing dirty paper precoder (ZFDP). Our scheme decomposes a MIMO channel into multiple identical parallel subchannels, which can make it rather convenient to design modulation/demodulation and coding/decoding schemes. Moreover, we prove that our scheme is asymptotically optimal for (moderately) high SNR in terms of both channel throughput and bit error rate (BER) performance. This desirable property is not shared by any other conventional schemes. We also consider the subchannel selection issues when some of the subchannels are too poor to be useful. Our scheme can also be combined with orthogonal frequency division multiplexing (OFDM) for intersymbol interference (ISI) suppression. The effectiveness of our approaches has been validated by both theoretical analyses and numerical simulations.

Uniform channel decomposition for MIMO communications

Assuming the availability of the channel state information at transmitter (CSIT), we propose a transceiver design which contains a linear precoder and a MMSE-VBLAST detector. We show that this transceiver design can decompose, in a capacity lossless manner, a MIMO channel into multiple subchannels with identical capacities. This uniform channel decomposition (UCD) scheme can provide much convenience for the modulation/demodulation and coding/decoding procedures due to obviating the need of bit allocation. The simulations show that the UCD scheme has excellent performance even without using any error correcting coding.

Algorithm 851: CG_DESCENT, a conjugate gradient method with guaranteed descent

Recently, a new nonlinear conjugate gradient scheme was developed which satisfies the descent condition <b>g</b>^T_<i>k</i><b>d</b>_<i>k</i> ≤ −7/8 ‖<b>g</b>_<i>k</i>‖² and which is globally convergent whenever the line search fulfills the Wolfe conditions. This article studies the convergence behavior of the algorithm; extensive numerical tests and comparisons with other methods for large-scale unconstrained optimization are given.

Direct trajectory optimization and costate estimation of finite-horizon and infinite-horizon optimal control problems using a Radau pseudospectral method

A method is presented for direct trajectory optimization and costate estimation of finite-horizon and infinite-horizon optimal control problems using global collocation at Legendre-Gauss-Radau (LGR) points. A key feature of the method is that it provides an accurate way to map the KKT multipliers of the nonlinear programming problem to the costates of the optimal control problem. More precisely, it is shown that the dual multipliers for the discrete scheme correspond to a pseudospectral approximation of the adjoint equation using polynomials one degree smaller than that used for the state equation. The relationship between the coefficients of the pseudospectral scheme for the state equation and for the adjoint equation is established. Also, it is shown that the inverse of the pseudospectral LGR differentiation matrix is precisely the matrix associated with an implicit LGR integration scheme. Hence, the method presented in this paper can be thought of as either a global implicit integration method or a pseudospectral method. Numerical results show that the use of LGR collocation as described in this paper leads to the ability to determine accurate primal and dual solutions for both finite and infinite-horizon optimal control problems.

Lipschitz Continuity for Constrained Processes

We study Lipschitz continuity properties for “constrained processes”. As applications of our general theory, we consider mathematical programs and optimal control problems. We show that if thegradients of the binding constraints satisfy an independence condition, then the solution and the dual multipliers of a convex mathematical program are a Lipschitz continuous function of the data. Similarly, it is proved that the optimal control and the dual multipliers for strictly convex control problems with convex constraints on the state and the control are Lipschitz continuous in time. In both applications, estimates of the Lipschitz constant are given.