Oleg N. Granichin

Simultaneous Perturbation Stochastic Approximation for Tracking Under Unknown but Bounded Disturbances

Multi-dimensional stochastic optimization plays an important role in analysis and control of many technical systems. To solve the challenging multidimensional problems of nonstationary optimization, it is suggested to use a stochastic approximation algorithm (like SPSA) with perturbed input and constant step-size which has simple form. We get a finite bound of residual between estimates and time-varying unknown parameters when observations are made under an unknown but bounded noise. Applications of the algorithm are considered for a random walk, an optimization of UAVs flight, and a load balancing problem.

Adaptive autonomous soaring of multiple UAVs using Simultaneous Perturbation Stochastic Approximation

This paper presents a new algorithm for maximizing the flight duration of a single UAV (Uninhabited Air Vehicle) and UAVs group using the thermal model developed by Allen at NASA Dryden.

Discrete-time minimum tracking based on stochastic approximation algorithm with randomized differences

In this paper application of the stochastic approximation algorithm with randomized differences to the minimum tracking problem for the non-constrained optimization is considered. The upper bound of mean-squared estimation error is derived in the case of once differentiable functional and almost arbitrary observation noise. Numerical simulation of the estimates stabilization for the multidimensional optimization with unknown but bounded deterministic noise is provided. Stabilization bound has sufficiently small level comparing to significant level of noise.

Adaptive control of siso plant with time-varying coefficients based on random test perturbation

Indirect adaptive control problem in closed loop time-varying linear system is addressed. Small random test signals are added to the plants inputs in order to identify and track its parameters. The noise is assumed to be unknown but bounded. It may not have good statistical properties which is the case in many applications. Simulation example of non minimum-phase plant of second order is provided.

Algorithm for stochastic approximation with trial input perturbation in the nonstationary problem of optimization

Consideration was given to the randomized stochastic approximation algorithm with simultaneous trial input perturbation and two measurements used to optimize the unconstrained nonstationary functional. The upper boundary of the mean-square residual was established under conditions of single differentiability of the functional and almost arbitrary noise. Efficiency of the algorithm was illustrated by an example of stabilization of the resulting estimates for the multidimensional case under dependent observation noise.

Local voting protocol in decentralized load balancing problem with switched topology, noise, and delays

In this paper the applicability of the local voting protocol with nonvanishing step-size for decentralized stochastic network load balancing is studied under nonstationary problem formulation. The network system was considered to have a switched topology, and the control strategy uses noisy and delayed measurements. Nonvanishing (for example, constant) step-size allows to achieve the better convergence rate and copes with time-varying loads and productivities of agents (nodes). Conditions for achieving a suboptimal level of loading agents are established, and an estimate of the appropriate level of suboptimality is given depending on the step-size of the control algorithm, the structure of the averaged network and the statistical properties of noise and delays in measurements. Obtained theoretical results are illustrated by simulations of simultaneously processing of 106 tasks by 1024 agents with 2048 links. It is examined that the performance of the adaptive multi-agent strategy with redistribution of tasks among “connected” neighbors is significantly better than the performance of the strategy without redistribution.

Randomized Algorithm for UAVs Group Flight Optimization

Abstract The problem of small UAVs flight optimization is considered. To solve this problem thermal updrafts are used. For the precise detection of the thermal updrafts center the simultaneous perturbation stochastic approximation (SPSA) type algorithm is proposed. If UAVs use thermal updrafts so they can save the energy during the flight. Therefore the flight time will be vary for different UAVs. In order to optimize the area monitoring, the consensus approach has been proposed.

Randomized Algorithms for Stochastic Approximation under Arbitrary Disturbances

New algorithms for stochastic approximation under input disturbance are designed. For the multidimensional case, they are simple in form, generate consistent estimates for unknown parameters under “almost arbitrary” disturbances, and are easily “incorporated” in the design of quantum devices for estimating the gradient vector of a function of several variables.

Multi-agent Stochastic Systems with Switched Topology and Noise

In this paper the approximate consensus problem in multi-agent stochastic systems with noisy information about the current state of the nodes and randomly switched topology for agents with nonlinear dynamics is considered. The control is formed by the local voting protocol with step size not tending to zero. To analyze closed loop system we propose to use method of continuous models (ODE approach or Derevitskii-Fradkov-Ljung (DFL)-scheme). The usage of this method allows one to reduce the computation load. The bounds of the mean proximity of trajectories of the discrete stochastic system to its continuous deterministic model are obtained. Based on those bounds the conditions for achieving mean square ε-consensus are established. The method is applied to the load balancing problem in decentralized stochastic dynamic network with incomplete information about the current state of nodes and changing set of communication links is considered. The load balancing problem is reformulated as consensus problem in noisy model with switched topology. The conditions to achieve the optimal level of nodes load are obtained. The performance of the system is evaluated both analytically and by simulation. Obtained results are important for control of production networks, multiprocessor or multicomputer networks, etc.

Stochastic approximation search algorithms with randomization at the input

This work presents a comprehensive survey of the development of pseudogradient stochastic approximation algorithms with randomized input disturbance, considers the problems of their applicability in optimization problems with linear constraints, and discusses new possibilities to use them for multiagent control for load balancing of nodes in computational networks. Justifications of the algorithms’ correctness and their optimal convergence rate are based on the foundational works of B.T. Polyak.