Jorge I. Poveda

Shahshahani gradient-like extremum seeking

In this paper we present novel extremum seeking controllers designed to achieve online constrained optimization of uncertain input-to-output maps of a class of nonlinear dynamical systems. The algorithms, inspired by a class of evolutionary dynamics that emerge in the context of population games, generate on-line Shahshahani gradient-like systems able to achieve extremum seeking under simplex-like constraints on the positive orthant. The method attains semi-global practical convergence to the optimal point, even when the initial conditions are not in the feasible set, and it can be naturally adapted to address distributed extremum seeking problems in multi-agent systems where an all-to-all communication structure is not available. Potential applications include problems on distributed dynamic resource allocation, congestion and flow control in networked systems, as well as portfolio optimization in financial markets. Via simulations, we illustrate our results under different scenarios.

Flexible Nash seeking using stochastic difference inclusions

We present a novel algorithm designed to achieve robust convergence to Nash equilibria in non-cooperative games, where players are not required to participate in the game for all time, neither to know the exact mathematical form of their cost function. In this algorithm each player employs stochastic probing dynamics that only require measurements of its own cost function, together with a dynamic time-ratio mechanism that enforces its frequency of participation in the game to satisfy a time-ratio constraint. The algorithm is modeled by a constrained stochastic difference inclusion with non-unique solutions that encompass a complete set of admissible behaviors for each player. To characterize the convergence and stability properties of the system we introduce the notion of mean-square practical exponential stability for constrained stochastic difference inclusions, as well as sufficient Lyapunov conditions that certify this property. Simulation examples are used to demonstrate the results.

A framework for a class of hybrid extremum seeking controllers with dynamic inclusions

Abstract This paper presents a prescriptive framework for the design and analysis of a class of deterministic extremum seeking controllers (ESCs) based on hybrid dynamic inclusions. This type of ESCs combines continuous and discrete-time dynamics during the seeking process, and its evolution in time is characterized by differential and difference inclusions rather than standard difference and differential equations. Examples of hybrid ESCs include, but are not limited to, purely continuous ESC with optimizers described by set-valued mappings, ESC with arbitrarily fast and slow switching modes, ESC with weakly-jumping parameters, as well as distributed ESCs for multi-agent systems with time-varying graphs. Since solutions of systems described by set-valued mappings are usually not unique, we do not insist on this property, but rather we characterize the behavior of all possible solutions generated by the closed-loop system. Making use of recent results in singular perturbations, averaging, and Ω -limit sets of sets for hybrid dynamical systems, we establish a practical asymptotic stability result. Some examples of common classes of seeking dynamics described by hybrid systems are presented, as well as some numerical simulations illustrating their application in standard maximization problems and Nash seeking problems in game theoretical scenarios.

A Robust Event-Triggered Approach for Fast Sampled-Data Extremization and Learning

This paper presents a general framework for the analysis and design of a class of model-free, robust, and efficient sampled-data-based algorithms for extremization and learning in continuous-time nonlinear systems that generate response maps with an optimal operational set. In particular, we consider plants described by differential inclusions, interconnected in a sampled-data setting with a robust learning algorithm characterized by a constrained difference inclusion. In contrast to standard sampled-data-based approaches, where the learning dynamics are updated after a fixed sufficiently long sampling time has passed, we design a robust dynamic event-based mechanism that triggers the control action as soon as the rate of change of the output of the plant is sufficiently small. By using this event-based update rule, a significant improvement in the convergence time of the closed-loop system can be achieved. Using the framework of set-valued hybrid dynamical systems, we establish for the closed-loop system the existence of a uniformly asymptotically stable compact set, which, by an appropriate tuning of the control parameters, can be made arbitrarily close to the optimal operational set. Our results generalize existing results for periodic sampled-data extremum seeking, and can be used to solve model-free multivariable smooth/nonsmooth constrained optimization problems, as well as learning problems in game theoretical scenarios.

Event-triggered based on-line optimization for a class of nonlinear systems

We consider the problem of robust on-line optimization of a class of continuous-time nonlinear systems by using a discrete-time controller/optimizer, interconnected with the plant in a sampled-data structure. In contrast to classic approaches where the controller is updated after a fixed sufficiently long waiting time has passed, we design an event-based mechanism that triggers the control action only when the rate of change of the output of the plant is sufficiently small. By using this event-based update rule, a significant improvement in the convergence rate of the closed-loop dynamics is achieved. Since the closed-loop system combines discrete-time and continuous-time dynamics, and in order to guarantee robustness and semi-continuous dependence of solutions on parameters and initial conditions, we use the framework of hybrid set-valued dynamical systems to analyze the stability properties of the system. Numerical simulations illustrate the results.

A hybrid systems approach to global synchronization and coordination of multi-agent sampled-data systems

Abstract A global synchronization algorithm for a network of resetting clocks is presented. Each clock signals its neighbors when it resets to zero after reaching the end of its period. Based on where a neighbor is located at the time of the signal, the neighboring clock adjusts its value instantaneously to synchronize with the signaling clock. Subsequently, the sychronization algorithm is used in the context of networked, multi-agent sampled-data systems. Here, agents update their internal state, based on the values of the internal states of their neighbors, when their clock variable resets to zero. The synchronization algorithm, together with additional sampling of the internal states of neighbors, is used to synchronize and coordinate the agents of the networked sampled-data system. These actions enable achieving the stability property that would result if the agents performed their state updates simultaneously. The models associated with these algorithms are presented in terms of hybrid dynamical systems. Well-posed models are developed that exhibit robustness properties, including robustness to small communication delays and to small mismatch in the natural frequency of each clock, and for which extensive analysis tools are available, including an invariance principle and a reduction principle.

A hybrid seeking approach for robust learning in multi-agent systems

This work presents a hybrid seeking control designed to achieve flexible robust learning in distributed multi-agent systems. We consider a set of n agents, with individual unknown but well behaved dynamics, seeking for an optimal equilibrium by using a hybrid control that requires only measurements of their cost functions. This equilibrium may correspond to a classic maximizer of a global potential function or a Nash equilibrium in a Lyapunov game. Making use of recent results in singular perturbation and averaging theory for hybrid dynamical systems we show convergence of the algorithm to a neighborhood of the optimal equilibrium.

Distributed robust stochastic learning in asynchronous networks of sampled-data systems

This paper presents a stochastic distributed algorithm for robust learning in networks of asynchronous sampled-data systems characterized by strongly connected directed graphs, where the response map of each sampled-data system has a quadratic structure, and the interactions between systems describe a Nash game. It is assumed that each sampled-data system has an individual resetting clock, as well as plant dynamics modeled by a differential inclusion. In order to achieve convergence in a practical mean-square sense to the Nash equilibrium of the game, we propose a robust distributed stochastic hybrid algorithm that coordinates and synchronizes the actions of the agents by using only local information from the network. A numerical example illustrates the results.

Robust constrained model predictive control with persistent model adaptation

In this paper we present a novel approach for robust model predictive control (MPC) for constrained linear discrete-time systems with bounded time-varying structural uncertainties. The proposed method combines a dead-beat observer with a robust tube-based MPC for plants with structural uncertainties in the model, in order to persistently learn in finite time the uncertainties by using only input and output measurements of the plant. The time-varying uncertainties are assumed to satisfy a dwell-time constraint, and under the given assumptions recursive feasibility as well as asymptotic stability for the closed-loop system are established.

A neuro-adaptive architecture for extremum seeking control using hybrid learning dynamics

This paper presents a novel approach to achieve online multivariable hybrid optimization of response maps associated to set-valued dynamical systems, without requiring the use of averaging theory. In particular, we propose a prescriptive framework for the analysis and design of a class of adaptive control architectures based on neural networks (NN) and learning dynamics described by hybrid dynamical systems (HDS). The NNs are used as model-free gradient approximators that are online tuned in order to obtain an arbitrarily precise estimation on a compact set of the gradient of the response map of the system under control. For the closed-loop system a semi-global practical asymptotic stability result is obtained, and the results are illustrated via numerical examples.