Paul Frihauf

Finite-horizon LQ control for unknown discrete-time linear systems via extremum seeking

We present a non-model based approach for asymptotic, locally exponentially stable attainment of the optimal open-loop control sequence for unknown, discrete-time linear systems with a scalar input, where not even the dimension of the system is known. This control sequence minimizes the finite-time horizon cost function, which is quadratic in the measured output and in the input. We make no assumptions on the stability of the unknown system, but we do assume that the system is reachable. The proposed algorithm employs the multi-variable discrete-time extremum seeking approach to minimize the cost function, extending results established for the scalar discrete-time extremum seeking method. Simulation results show that the Hessians condition number, used as a measure of the optimization problems level of difficulty, increases with both the systems level of instability and the length of the finite horizon for a scalar system. Thus, we suggest solving well-conditioned, shorter time horizon optimal control problems to obtain good initial control estimates for longer time horizon problems. We also show that the algorithm accommodates input constraints by employing the projection operator.

Nash Equilibrium Seeking with Finitely-and Infinitely-Many Players *

Abstract We introduce a non-model based approach for stable attainment of Nash equilibria in noncooperative games. Unlike classical game theory, which requires some amount of modeling information, this approach employs deterministic extremum seeking to enable the players to maximize their payoff functions without knowing the underlying model of the game. The players only need to measure their own payoff values. We present results for games for some basic models of economic competition with two players, N players, and infinitely-many players.

Nash Equilibrium Seeking for Dynamic Systems with Non-quadratic Payoffs

We consider general, stable nonlinear differential equations with N inputs and N outputs, where in the steady state, the output signals represent the payoff functions of a noncooperative game played by the steady-state values of the input signals. To achieve locally stable convergence to the resulting steady-state Nash equilibria, we introduce a non-model-based approach, where the players determine their actions based only on their own payoff values. This strategy is based on the extremum seeking approach, which has previously been developed for standard optimization problems and employs sinusoidal perturbations to estimate the gradient. Since non-quadratic payoffs create the possibility of multiple, isolated Nash equilibria, our convergence results are local. Specifically, the attainment of any particular Nash equilibrium is not assured for all initial conditions, but only for initial conditions in a set around that specific stable Nash equilibrium. For non-quadratic costs, the convergence to a Nash equilibrium is not perfect, but is biased in proportion to the perturbation amplitudes and the higher derivatives of the payoff functions. We quantify the size of these residual biases.

Rapidly convergent leader-enabled multi-agent deployment into planar curves

We introduce an approach for stable deployment of agents into planar curves (1-D formations in 2-D space) parameterized by the agent index. Stability is ensured by leader feedback, which is designed in a manner similar to boundary control of PDEs. By discretizing the model and the PDE controllers with respect to the continuous agent index, we obtain control laws for the discrete follower agents and the leader agent. The class of PDEs that motivates our design is the reaction-advection-diffusion class (broader than the standard heat equation, which is stable and does not necessitate leader feedback), which allows a much broader family of deployment profiles. Many of these profiles, however, are open-loop unstable. We stabilize them with leader feedback.

A Single Forward-Velocity Control Signal for Stochastic Source Seeking With Multiple Nonholonomic Vehicles

With a single stochastic extremum seeking control signal, we steer multiple autonomous vehicles, modeled as nonholonomic unicycles, toward the maximum of an unknown, spatially distributed signal field. The vehicles, whose angular velocities are constant and distinct, travel at the same forward velocity, which is controlled by the stochastic extremum seeking controller. To determine the vehicles’ velocity, the controller uses measurements of the signal field at the respective vehicle positions and excitation based on filtered white noise. The positions of the vehicles are not measured. We prove local exponential convergence, both almost surely and in probability, to a small neighborhood near the source and provide a numerical example to illustrate the effectiveness of the algorithm. [DOI: 10.1115/1.4027577]

Stochastic source seeking with tuning of forward velocity

Using the method of stochastic extremum seeking, we navigate an autonomous vehicle, modeled as a nonholonomic unicycle, towards the maximum of an unknown, spatially distributed signal field by measuring only the signal at the vehicles position. The vehicle position is not measured. Keeping the angular velocity constant, we control the forward velocity by designing a stochastic source seeking control law, which employs excitation based on filtered white noise rather than sinusoidal perturbations used in previous works. We prove local exponential convergence, both almost surely and in probability, to a small neighborhood near the source and provide numerical simulations to illustrate the effectiveness of the algorithm.

Stochastic Source Seeking With Multiple Nonholonomic Vehicles via a Single Forward-Velocity Control Signal

With a single stochastic extremum seeking control signal, we steer multiple autonomous vehicles, modeled as nonholonomic unicycles, toward the maximum of an unknown, spatially distributed signal field. The vehicles, whose angular velocities are constant and distinct, travel at the same forward velocity, which is controlled by the stochastic extremum seeking controller. To determine the vehicles’ velocity, the controller uses measurements of the signal field at the respective vehicle positions and excitation based on filtered white noise. The positions of the vehicles are not measured. We prove local exponential convergence, both almost surely and in probability, to a small neighborhood near the source and provide a numerical example to illustrate the effectiveness of the algorithm.Copyright

Laser Pulse Shaping via Iterative Learning Control and High-Dimensional Extremum Seeking

We investigate pulse shaping and optimization for a laser amplifier. Due to the complex character of the nonlinear PDE dynamics involved in the laser model, it is of interest to consider non-model based methods for pulse shaping. We determine input pulse shapes for an unknown laser dynamics model using iterative learning control (ILC) and high-dimensional extremum seeking (ES), which is a real-time optimization strategy. We utilize ILC to obtain the input pulse shape that generates a desired output pulse shape and ES to find the input pulse that maximizes the energy amplifier gain. Both single-pass and double-pass laser models are investigated. The effectiveness of these approaches is illustrated via numerical simulations.Copyright