Homer F. Walker

Mixture Densities, Maximum Likelihood and the EM Algorithm

The problem of estimating the parameters which determine a mixture density has been the subject of a large, diverse body of literature spanning nearly ninety years. During the last two decades, the method of maximum likelihood has become the most widely followed approach to this problem, thanks primarily to the advent of high speed electronic computers. Here, we first offer a brief survey of the literature directed toward this problem and review maximum-likelihood estimation for it. We then turn to the subject of ultimate interest, which is a particular iterative procedure for numerically approximating maximum-likelihood estimates for mixture density problems. This procedure, known as the EM algorithm, is a specialization to the mixture density context of a general algorithm of the same name used to approximate maximum-likelihood estimates for incomplete data problems. We discuss the formulation and theoretical and practical properties of the EM algorithm for mixture densities, focussing in particular on ...

Globally Convergent Inexact Newton Methods

Inexact Newton methods for finding a zero of

Implementation of the GMRES method using householder transformations

F:\mathbf{R}^n \to \mathbf{R}^n

NITSOL: A Newton Iterative Solver for Nonlinear Systems

are variations of Newton’s method in which each step only approximately satisfies the linear Newton equation but still reduces the norm of the local linear model of F. Here, inexact Newton methods are formulated that incorporate features designed to improve convergence from arbitrary starting points. For each method, a basic global convergence result is established to the effect that, under reasonable assumptions, if a sequence of iterates has a limit point at which

GMRES On (Nearly) Singular Systems

F^\prime

Algorithm 777: HOMPACK90: a suite of Fortran 90 codes for globally convergent homotopy algorithms

is invertible, then that limit point is a solution and the sequence converges to it. When appropriate, it is shown that initial inexact Newton steps are taken near the solution, and so the convergence can ultimately be made as fast as desired, up to the rate of Newton’s method, by forcing the initial linear residuals to be appropriately small. The primary goal is to introduce and analyze new inexact Newton methods, but consideration is also given to “gl...

Convergence Theorems for Least-Change Secant Update Methods,

The standard implementation of the GMRES method for solving large nonsymmetric linear systems involves a Gram-Schmidt process which is a potential source of significant numerical error. An alternative implementation is outlined here in which orthogonalization by Householder transformations replaces the Gram-Schmidt process. This implementation requires slightly less storage but somewhat more arithmetic than the standard one; however, numerical experiments suggest that it is more stable, especially as the limits of residual reduction are reached. The extra arithmetic required may be less significant when products of the coefficient matrix with vectors are expensive or on vector and, in particular, parallel machines.

Anderson Acceleration for Fixed-Point Iterations

We introduce a well-developed Newton iterative (truncated Newton) algorithm for solving large-scale nonlinear systems. The framework is an inexact Newton method globalized by backtracking. Trial steps are obtained using one of several Krylov subspace methods. The algorithm is implemented in a Fortran solver called NITSOL that is robust yet easy to use and provides a number of useful options and features. The structure offers the user great flexibility in addressing problem specificity through preconditioning and other means and allows easy adaptation to parallel environments. Features and capabilities are illustrated in numerical experiments.

Residual smoothing techniques for iterative methods

We consider the behavior of the GMRES method for solving a linear system

Simulating cyclic artery compression using a 3D unsteady model with fluid–structure interactions

Ax = b