Stacy D. Hill

Optimization over discrete sets via SPSA

A fixed gain version of the SPSA (simultaneous perturbation stochastic approximation) method (J.C. Spall, 1992) for function minimization is developed and the error process is characterized. The new procedure is applicable to optimization problems over /spl Zscr//sup p/., the grid of points in /spl Rscr//sup p/ with integer components. Simulation results and a closely related application, a resource allocation problem, is also described.

Theoretical framework for comparing several popular stochastic optimization approaches

This paper establishes a framework for formal comparisons of several leading optimization algorithms, establishing guidance to practitioners for when to use or not to use a particular method. The focus in this paper are five general algorithm forms: random search, simultaneous perturbation stochastic approximation, simulated annealing, evolutionary strategies, and genetic algorithms. We summarize the available theoretical results on rates of convergence for the five algorithm forms and then use the theoretical results to draw some preliminary conclusions on the relative efficiency. Our aim is to sort out some of the competing claims of efficiency and to suggest a structure for comparison that is more general and transferable than the usual problem-specific numerical studies.

Simulation optimization of air traffic delay cost

The cost of delay is a serious and increasing problem in the airline industry. Air travel is increasing, and already domestic airports incur thousands of hours of delay daily, costing the industry

SPSA/SIMMOD optimization of air traffic delay cost

2 billion a year. One strategy for reducing total delay costs is to hold planes for a short time at the gate in order to reduce costly airborne congestion. In a network of airports involving thousands of flights, it is difficult to determine the amount to hold each flight at the gate. This paper discusses how the optimization procedure simultaneous perturbation stochastic approximation (SPSA) can be used to process delay cost measurements from air traffic simulation packages and produce an optimal gate holding strategy. As a test case, the SIMMOD air traffic simulation package was used to model a simple four-airport network.

Theoretical comparisons of evolutionary computation and other optimization approaches

The cost of delay is a serious and increasing problem in the airline industry. Air travel is increasing, and already domestic airports incur thousands of hours of delay daily, costing the industry

Sensitivity of a Bayesian analysis to the prior distribution

2 billion a year. One strategy for reducing total delay costs is to hold planes for a short time at the gate in order to reduce costly airborne congestion. In a network of airports involving hundreds of flights, it is difficult to determine the amount to hold each flight at the gate. This paper discusses how the optimization procedure simultaneous perturbation stochastic approximation (SPSA) can be used to process delay cost measurements from air traffic simulation packages and produce an optimal gate holding schedule. As a test case, the SIMMOD air traffic simulation package was used to model a simple four-airport network. Initial delay costs are reduced up to 10.3%.

Constrained optimization over discrete sets via SPSA with application to non-separable resource allocation

This paper is a first step to formal comparisons of several leading optimization algorithms, establishing guidance to practitioners for when to use or not use a particular method. The focus in this paper is four general algorithm forms: random search, simultaneous perturbation stochastic approximation, simulated annealing, and evolutionary computation. We summarize the available theoretical results on rates of convergence for the four algorithm forms and then use the theoretical results to draw some preliminary conclusions on the relative efficiency. Our aim is to sort out some of the competing claims of efficiency and to suggest a structure for comparison that is more general and transferable than the usual problem-specific numerical studies. Much work remains to be done to generalize and extend the results to problems and algorithms of the type frequently seen in practice.

Some theoretical comparisons of stochastic optimization approaches

Consider the problem of eliciting and specifying a prior probability distribution for a Bayesian analysis. There will generally be some uncertainty in the choice of prior, especially when there is little information from which to construct such a distribution, or when there are several priors elicited, say, from different experts. It is of interest, then, to characterize the sensitivity of a posterior distribution (or posterior mean) to prior. We characterize this sensitivity in terms of bounds on the difference between posterior distributions corresponding to different priors. Further, we illustrate the results on two distinct problems: a) determining least-informative (vague) priors and b) estimating statistical quantiles for a problem in analyzing projectile accuracy. >

Transfer optimization via simultaneous perturbation stochastic approximation

This paper presents a version of the simultaneous perturbation stochastic approximation (SPSA) algorithm for optimizing non-separable functions over discrete sets under given constraints. The primary motivation for discrete SPSA is to solve a class of resource allocation problems wherein the goal is to distribute a finite number of discrete resources to finitely many users in such a way as to optimize a specified objective function. The basic algorithm and the application of the algorithm to the optimal resource allocation problem is discussed and simulation, results are presented which illustrate its performance.

Simulation optimization via simultaneous perturbation stochastic approximation

This paper is a first step to formal comparisons of several leading optimization algorithms, establishing guidance to practitioners for when to use or not use a particular method. It focuses on four general algorithm forms: random search, simultaneous perturbation stochastic approximation simulated annealing, and evolutionary computation. We summarize the available theoretical results on rates of convergence for the four algorithm forms and then use the theoretical results to draw some preliminary conclusions on the relative efficiency. Our aim is to sort out some of the competing claims of efficiency and suggest a structure for comparison that is more general and transferable than the usual problem-specific numerical studies. Much work remains to be done to generalize and extend the results to problems and algorithms of the type frequently seen in practice.