Yorai Wardi

Optimal control of a class of hybrid systems

We present a modeling framework for hybrid systems intended to capture the interaction of event-driven and time-driven dynamics. This is motivated by the structure of many manufacturing environments where discrete entities (termed jobs) are processed through a network of workcenters so as to change their physical characteristics. Associated with each job is a temporal state subject to event-driven dynamics and a physical state subject to time-driven dynamics. Based on this framework, we formulate and analyze a class of optimal control problems for single-stage processes. First-order optimality conditions are derived and several properties of optimal state trajectories (sample paths) are identified which significantly simplify the task of obtaining explicit optimal control policies.

Transition-time optimization for switched-mode dynamical systems

This note considers the problem of determining optimal switching times at which mode transitions should occur in multimodal, hybrid systems. It derives a simple formula for the gradient of the cost functional with respect to the switching times, and uses it in a gradient-descent algorithm. Much of the analysis is carried out in the setting of optimization problems involving fixed switching-mode sequences, but a possible extension is pointed out for the case where the switching-mode sequence is a part of the variable. Numerical examples testify to the viability of the proposed approach.

Perturbation analysis for online control and optimization of stochastic fluid models

Uses stochastic fluid models (SFMs) for control and optimization (rather than performance analysis) of communication networks, focusing on problems of buffer control. We derive gradient estimators for packet loss and workload related performance metrics with respect to threshold parameters. These estimators are shown to be unbiased and directly observable from a sample path without any knowledge of underlying stochastic characteristics, including traffic and processing rates (i.e., they are nonparametric). This renders them computable in online environments and easily implementable for network management and control. We further demonstrate their use in buffer control problems where our SFM-based estimators are evaluated based on data from an actual system.

Optimal control of switching times in switched dynamical systems

This paper considers an optimal control problem for switched dynamical systems, where the objective is to minimize a cost functional defined on the state, and where the control variable consists of the switching times. The gradient of the cost functional is derived on an especially simple form, which lends itself to be directly used in gradient-descent algorithms. This special structure of the gradient furthermore allows for the number of switching points to become part of the control variable, instead of being a given constant. Numerical examples testify to the viability of the proposed approach.

Perturbation analysis and control of two-class stochastic fluid models for communication networks

This paper uses stochastic fluid models (SFMs) for the control and optimization (rather than performance analysis) of communication network nodes processing two classes of traffic: one is uncontrolled and the other is subject to threshold-based buffer control. We derive gradient estimators for packet loss and workload related performance metrics with respect to threshold parameters. These estimators are shown to be unbiased and directly observable from a sample path without any knowledge of underlying stochastic characteristics of the traffic processes. This renders them computable in online environments and easily implementable for network management and control. We further demonstrate their use in buffer control problems where our SFM-based estimators are evaluated based on data from an actual system.

Perturbation Analysis and Optimization of Stochastic Hybrid Systems

We present a general framework for carrying out perturbation analysis in Stochastic Hybrid Systems (SHS) of arbitrary structure. In particular, Infinitesimal Perturbation Analysis (IPA) is used to provide unbiased gradient estimates of performance metrics with respect to various controllable parameters. These can be combined with standard gradient-based algorithms for optimization purposes and implemented on line with little or no distributional information regarding the stochastic processes involved. We generalize an earlier concept of “induced events” for this framework to include system features such as delays in control signals or modeling multiple user classes sharing a resource. We apply this generalized IPA to two SHS with different characteristics. First, we develop a gradient estimator for the performance of a linear switched system with control signal delays and a safety constraint and show that it is independent of the random delays distributional characteristics. Second, we derive closed-form unbiased IPA estimators for a Stochastic Flow Model (SFM) of systems executing tasks subject to either hard or soft real-time constraints. These estimators are incorporated in a gradient-based algorithm to optimize performance by controlling a task admission threshold parameter. Simulation results are included to illustrate this optimization approach.

Optimal Control of Switching Surfaces in Hybrid Dynamical Systems

This paper concerns an optimal control problem defined on a class of switched-mode hybrid dynamical systems. The systems mode is changed (switched) whenever the state variable crosses a certain surface in the state space, henceforth called a switching surface. These switching surfaces are parameterized by finite-dimensional vectors called the switching parameters. The optimal control problem is to minimize a cost functional, defined on the state trajectory, as a function of the switching parameters. The paper derives the gradient of the cost functional in a costate-based formula that reflects the special structure of hybrid systems. It then uses the formula in a gradient-descent algorithm for solving an obstacle-avoidance problem in robotics.

Online IPA Gradient Estimators in Stochastic Continuous Fluid Models

This paper applies infinitesimal perturbation analysis (IPA) to loss-related and workload-related metrics in a class of stochastic flow models (SFM). It derives closed-form formulas for the gradient estimators of these metrics with respect to various parameters of interest, such as buffer size, service rate, and inflow rate. The IPA estimators derived are simple and fast to compute, and are further shown to be unbiased and nonparametric, in the sense that they can be computed directly from the observed data without any knowledge of the underlying probability law. These properties hold out the promise of utilizing IPA gradient estimates as ingredients of online management and control of telecommunications networks. While this paper considers single-node SFMs, the analysis method developed is amenable to extensions to networks of SFM nodes with more general topologies.

On-Line Optimization of Switched-Mode Dynamical Systems

This paper considers an optimization problem in the setting of switched-mode hybrid dynamical systems, where the control variable (independent variable) consists of the mode-switching times, and the performance criterion is a cost functional defined on the systems state trajectory. The system is deterministic, nonlinear, and autonomous, and its state variable cannot be measured and hence it has to be estimated. We propose an on-line, Newton-like optimization algorithm that recomputes the control variable by attempting to optimize the cost-to-go at equally-spaced epochs. The main result concerns the algorithms convergence rate, which can vary from sublinear to quadratic depending on its computing rate and the state estimation error.

Perturbation analysis and optimization of stochastic flow networks

We consider a stochastic fluid model of a network consisting of several single-class nodes in tandem and perform perturbation analysis for the node queue contents and associated event times with respect to a threshold parameter at the first node. We then derive infinitesimal perturbation analysis (IPA) derivative estimators for loss and buffer occupancy performance metrics with respect to this parameter and show that these estimators are unbiased. We also show that the estimators depend only on data directly observable from a sample path of the actual underlying discrete event system, without any knowledge of the stochastic characteristics of the random processes involved. This renders them computable in online environments and easily implementable for network management and optimization. This is illustrated by combining the IPA estimators with standard gradient based stochastic optimization methods and providing simulation examples.