Vicente Costanza

Approximation procedures for the optimal control of bilinear and nonlinear systems

Optimal control problems for bilinear systems are studied and solved with a view to approximating analogous problems for general nonlinear systems. For a given bilinear optimal control problem, a sequence of linear problems is constructed, and their solutions are shown to converge to the desired solution. Also, the direct solution to the Hamilton-Jacobi equation is analyzed. A power-series approach is presented which requires offline calculations as in the linear case (Riccati equation). The methods are compared and illustrated. Relations to classical linear systems theory are discussed.

Brief paper: Finite-horizon dynamic optimization of nonlinear systems in real time

A novel scheme for constructing and tracking the solution trajectories to regular, finite-horizon, deterministic optimal control problems with nonlinear dynamics is devised. The optimal control is obtained from the states and costates of Hamiltonian ODEs, integrated online. In the one-dimensional case the initial costate is found by successively solving two first-order, quasi-linear, partial differential equations, whose independent variables are the time-horizon duration T and the final penalty coefficient S. These PDEs should in general be integrated off-line, the solution rendering not only the missing initial condition sought in the particular (T,S)-situation, but additional information on the boundary values of the whole two-parameter family of control problems, which can be used for designing the definitive objective functional. Optimal trajectories for the model are then generated in real time and used as references to be followed by the physical system. Numerical improvements are suggested for accurate integration of naturally unstable Hamiltonian dynamics, and strategies are proposed for tracking their results, in finite time or asymptotically, when perturbations in the state of the system appear. The whole procedure is tested in models arising in aero-navigation optimization.

A closed-loop approach to antiretroviral therapies for HIV infection

Abstract Antiretroviral therapies allowing for regular medical intervention in an optimal feedback role are substantiated. Individual therapies are evaluated from a multi-objective cost perspective, under different time and history constrains. The dynamics of the control system is governed by three state variables: healthy T-cells, infected T-cells, and viral particles. The drug dose administrated to the patient is taken as the manipulated or control variable. Illustrations of the methodology employed are provided for a fixed time-horizon of 180 days and variable prescription intervals, inverse discount factors, state discretization, and thresholds. Two cases are discussed: (i) beginning of the infection and (ii) near endemic equilibrium. Both open-loop and closed-loop results are presented. Dynamic Programming has been employed in the numerical treatment of the problem. Safety recommendations are also designed to cope with the chaotic behavior of the free dynamics’ flow in critical regions of the states’ domain. A software package is envisioned to assist physicians in assessing the patient “state”, estimating antiretroviral dose prescriptions for the whole treatment period, simulating the patient’s evolution, eventually correcting the original sequence in presence of incoming data, and evaluating combined costs of alternative treatment strategies.

Optimizing thymic recovery in HIV patients through multidrug therapies

Abstract An optimal control approach based on an enlarged nonlinear model for the dynamics of HIV infection and thymic function is composed to simulate and evaluate antiretroviral therapies. In addition to the relevant biological agents, an extra state variable is included, associated with the thymus capacity for healthy cells production. The methodology contemplates eventual deleterious effects of drugs over childrens thymus recovery. The intake of ‘Reverse Transcriptase Inhibitors’ and ‘Protease Inhibitors’ are modeled as two independent control variables, each affecting a different term in the dynamics, so extending the prevailing pure-HAART-therapy analysis. The objective function designed here is also more inclusive than usual, accounting for the costs of the two drug families involved and for the thymus deterioration, in addition to penalizing eventual virus excess and healthy cells deficits. The search for the best combined therapy is treated as an optimal control problem. A hybrid version of Dynamic Programming for continuous and discrete variables is used to treat the problem numerically. Long time-horizons are explored, aiming to avoid typical peaks in drug prescriptions found at the beginning and at the end of the optimization periods. Results indicate that certain combinations of drugs are more convenient than pure protocols when the value of thymus functioning is relevant, specially for children patients.

Mathematical Modeling of HIV Dynamics After Antiretroviral Therapy Initiation: A Review

Abstract This review shows the potential ground-breaking impact that mathematical tools may have in the analysis and the understanding of the HIV dynamics. In the first part, early diagnosis of immunological failure is inferred from the estimation of certain parameters of a mathematical model of the HIV infection dynamics. This method is supported by clinical research results from an original clinical trial: data just after 1 month following therapy initiation are used to carry out the model identification. The diagnosis is shown to be consistent with results from monitoring of the patients after 6 months. In the second part of this review, prospective research results are given for the design of individual anti-HIV treatments optimizing the recovery of the immune system and minimizing side effects. In this respect, two methods are discussed. The first one combines HIV population dynamics with pharmacokinetics and pharmacodynamics models to generate drug treatments using impulsive control systems. The second one is based on optimal control theory and uses a recently published differential equation to model the side effects produced by highly active antiretroviral therapy therapies. The main advantage of these revisited methods is that the drug treatment is computed directly in amounts of drugs, which is easier to interpret by physicians and patients.

Flexible operation through optimal tracking in nonlinear processes

Abstract In flexible process operation there exist physical variables which are allowed to take a finite number of stationary values. Each of these targets play the role of a typical set-point from the process control point of view. Changes in set-point are handled by industrial regulators. Such controllers, including PIDs, are commonly tuned when put into service, and their parameters remain the same for different set-point values, which is inpractical for linear and nonlinear systems. Here this problem is solved from the tracking — instead of regulation — point of view, and an optimality criteria is introduced. Opposite to the classical approach the formulation of an ‘absolute’ control cost is emphasized. This means that the energy employed in controlling or tracking should be measured as a whole, and not as a difference with the effort needed to maintain in equilibrium one particular set-point. The main purpose of this article is to deal with the nonlinear dynamics case, where the process is approximated by a general bilinear state equation and the cost is quadratic as usual. This is called the bilinear-quadratic optimal tracking problem. In the paper analytical expressions are found for the solution to this optimal tracking problem under mild simplifying hypothesis, and the traditional optimal control versus the new tracking approaches are compared. Also the effect of nonlinearities over both problems are discussed. In the tracking context the time horizon must necessarily be finite for the optimization problem to be mathematically well posed. Although this finite horizon is dictated by flexibility costraints, once the corresponding optimal tracking strategy is obtained, then it is applied repeatedly without need for further calculations.

Taking Side Effects into Account for HIV Medication

A control-theoretic approach to the problem of designing “low-side-effects” therapies for HIV patients based on highly active drugs is substantiated here. The evolution of side effects during treatment is modeled by an extra differential equation coupled to the dynamics of virions, healthy T-cells, and infected ones. The new equation reflects the dependence of collateral damages on the amount of each dose administered to the patient and on the evolution of the viral load detected by periodical blood analysis. The cost objective accounts for recommended bounds on healthy cells and virions, and also penalizes the appearance of collateral morbidities caused by the medication. The optimization problem is solved by a hybrid dynamic programming scheme that adhere to discrete-time observation and control actions, but by maintaining the continuous-time setup for predicting states and side effects. The resulting optimal strategies employ less drugs than those prescribed by previous optimization studies, but maintaining high doses at the beginning and the end of each period of six months. If an inverse discount rate is applied to favor early actions, and under a mild penalization of the final viral load, then the optimal doses are found to be high at the beginning and decrease afterward, thus causing an apparent stabilization of the main variables. But in this case, the final viral load turns higher than acceptable.

Online suboptimal control of linearized models

A novel approach to approximately solving the restricted-control linear quadratic regulator problem online is substantiated and applied in two case studies. The first example is a one-dimensional system whose exact solution is known. The other one refers to the temperature control of a metallic strip at the exit of a multi-stand rolling mill. The new (online-feedback) strategy employs a convenient version of the gradient method, where partial derivatives of the cost are taken with respect to the final penalization matrix coefficients and to the switching times where the control (de)saturates. The calculations are based on exact algebraic formula, which do not involve trajectory simulations, and so reducing in principle the computational effort associated with receding horizon or nonlinear programming methods.

Initial values for Riccati ODEs from variational PDEs

The recently discovered variational PDEs (partial differential equations) for finding missing boundary conditions in Hamilton equations of optimal control are applied to the extended-space transformation of time-variant linear-quadratic regulator (LQR) problems. These problems become autonomous but with nonlinear dynamics and costs. The numerical solutions to the PDEs are checked against the analytical solutions to the original LQR problem. This is the first validation of the PDEs in the literature for a nonlinear context. It is also found that the initial value of the Riccati matrix can be obtained from the spatial derivative of the Hamiltonian flow, which satisfies the variational equation. This last result has practical implications when implementing two-degrees-of freedom control strategies for nonlinear systems with generalized costs.

Rotational speed tracking for hydraulic-motor-driven machine-tools

Reference LA-ARTICLE-1996-001View record in Web of Science Record created on 2004-11-26, modified on 2017-05-10