Jichuan Yang

Stochastic approximation with averaging of the iterates: optimal asymptotic rate of convergence for general processes

Consider the stochastic approximation algorithm \[X_{n + 1} = X_n + a_n g(X_n ,\xi _n ).\] In an important paper, Polyak and Juditsky [SIAM J. Control Optim., 30 (1992), pp. 838–855] showed that (loosely speaking) if the coefficients

Controlled and optimally controlled multiplexing systems: A numerical exploration

a_n

A Monte Carlo method for sensitivity analysis and parametric optimization of nonlinear stochastic systems

go to zero slower than

A Numerical Method for Controlled Routing in Large Trunk Line Networks via Stochastic Control Theory

O({1 / n})

A Monte Carlo method for sensitivity analysis and parametric optimization of nonlinear stochastic systems: the ergodic case

, then the averaged sequence

Stochastic Approximation with Averaging and Feedback: Faster Convergence

\sum\nolimits_{i = 1}^n {{{X_i } / n}}

An effective numerical method for controlled routing in large trunk line networks

converged to its limit, at an optimum rate, for any coefficient sequence. The conditions were rather special, and direct constructions were used. Here a rather simple proof is given that results of this type are generic to stochastic approximation, and essentially hold any time that the classical asymptotic normality of the normalized and centered iterates holds. Considerable intuitive insight is provided into the procedure. Simulations have well borne out the importance of the method.

Analysis of adaptive step-size SA algorithms for parameter tracking

Large controlled multiplexing systems are approximated by diffusion type processes yielding a very efficient way of approximation and good numerical methods. The “limit” equations are an efficient aggregation of the original system, and provide the basis of the actual numerical approximation to the control problem. The numerical approximations have the structure of the original problem, but are generally much simpler. The control can occur in a variety of places; e.g., “leaky bucket” controllers, control of “marked cells” at the transmitter buffer, or control of the transmitter speed. From the point of view of the limit equations, those are equivalent. Various forms of the optimal control problem are explored, where the main aim is to control or balance the losses at the control with those due to buffer overflow. It is shown that much can be saved via the use of optimal controls or reasonable approximations to them. We discuss systems with one to three classes of sources, various aggregation methods and control approximation schemes. There are qualitative comparisons of various systems with and without control and a discussion of the variations of control and performance as the systems data and control bounds vary. The approach is a very useful tool for providing both qualitative and quantitative information which would be hard to get otherwise. The results have applications to various forms of the ATM and broadband integrated data networks.

Stochastic approximation with averaging and feedback: rapidly convergent "on-line" algorithms

For high-dimensional or nonlinear problems there are serious limitations on the power of available computational methods for the optimization or parametric optimization of stochastic systems. The paper develops an effective Monte Carlo method for obtaining good estimators of systems sensitivities with respect to system parameters under quite general conditions on the systems and cost functions. The value of the method is borne out by numerical experiments, and the computational requirements are favorable with respect to competing methods when the dimension is high or the nonlinearities “severe.” The method is a type of “derivative of likelihood ratio” method. Jump-diffusion, functional diffusion, and reflected diffusion models of broad types are covered by the basic technique (e.g., the type of limit model that arises in the analysis of queueing systems under heavy traffic, where the boundary reflection conditions are discontinuous). For a wide class of problems, the cost function or dynamics need not be ...

Stochastic approximation with averaging and feedback: rapidly convergent (quote)on-line(quote) algorithms

The paper discusses a powerful approach to the routing problem in large networks of the trunk line type. The approximations are based on heavy traffic limit theorems. The sequence of suitably scaled available circuits converges to a reflected diffusion process as the network size grows, under reasonable conditions. This limit model contains the basic features of the original network, and provides a very useful basis for a good control strategy for the physical system. The optimal ergodic cost problem for a three (link) dimensional system is solved numerically via the Markov chain approximation method to get the optimal controls. These “three link” results can be approximated in such a way that they can be applied to a physical network of arbitrary size, using only “local” information. Indeed, the numerical approximations have the interpretation of a type of simplified or “aggregated” network, which allows the use of physical intuition in its application. The resulting polities are compared in simulations (on systems with hundreds of links) to other current approaches, and found to be quite competitive and have many advantages. INFORMS Journal on Computing , ISSN 1091-9856, was published as ORSA Journal on Computing from 1989 to 1995 under ISSN 0899-1499.