Pablo S. Rivadeneira

Brief paper: Finite-horizon dynamic optimization of nonlinear systems in real time

A novel scheme for constructing and tracking the solution trajectories to regular, finite-horizon, deterministic optimal control problems with nonlinear dynamics is devised. The optimal control is obtained from the states and costates of Hamiltonian ODEs, integrated online. In the one-dimensional case the initial costate is found by successively solving two first-order, quasi-linear, partial differential equations, whose independent variables are the time-horizon duration T and the final penalty coefficient S. These PDEs should in general be integrated off-line, the solution rendering not only the missing initial condition sought in the particular (T,S)-situation, but additional information on the boundary values of the whole two-parameter family of control problems, which can be used for designing the definitive objective functional. Optimal trajectories for the model are then generated in real time and used as references to be followed by the physical system. Numerical improvements are suggested for accurate integration of naturally unstable Hamiltonian dynamics, and strategies are proposed for tracking their results, in finite time or asymptotically, when perturbations in the state of the system appear. The whole procedure is tested in models arising in aero-navigation optimization.

Impulsive control of single-input nonlinear systems with application to HIV dynamics

Abstract In this paper, some fundamental analysis and results are introduced for the accessibility of impulsive control systems (ICS). The main result is the characterization of accessibility for nonlinear ICS based on the ‘number of impulses’ which is required. These results naturally generalize and correct some earlier results obtained for linear ICS. The theory developed is applied to an impulsive model of the dynamics of human immunodeficiency virus (HIV) subject to medication. It is shown that HIV system fulfills the accessibility criterion. Finally, an impulsive control strategy is designed based on exact linearization to improve the response immune system of a patient of HIV.

A closed-loop approach to antiretroviral therapies for HIV infection

Abstract Antiretroviral therapies allowing for regular medical intervention in an optimal feedback role are substantiated. Individual therapies are evaluated from a multi-objective cost perspective, under different time and history constrains. The dynamics of the control system is governed by three state variables: healthy T-cells, infected T-cells, and viral particles. The drug dose administrated to the patient is taken as the manipulated or control variable. Illustrations of the methodology employed are provided for a fixed time-horizon of 180 days and variable prescription intervals, inverse discount factors, state discretization, and thresholds. Two cases are discussed: (i) beginning of the infection and (ii) near endemic equilibrium. Both open-loop and closed-loop results are presented. Dynamic Programming has been employed in the numerical treatment of the problem. Safety recommendations are also designed to cope with the chaotic behavior of the free dynamics’ flow in critical regions of the states’ domain. A software package is envisioned to assist physicians in assessing the patient “state”, estimating antiretroviral dose prescriptions for the whole treatment period, simulating the patient’s evolution, eventually correcting the original sequence in presence of incoming data, and evaluating combined costs of alternative treatment strategies.

A control systems analysis of HIV prevention model using impulsive input

We investigate a control systems analysis on HIV infection dynamics with regard to enhancement of the immune response. The HIV dynamic model is modified to include the pharmaco-kinetics and pharmacodynamics of antiretroviral HIV drugs, and the intake of drug is considered as impulsive control input. As it is administrated at discrete time instants, we assume that this yields an impulsive control problem for a nonlinear continuous-time system. Based on this new model, we study clinical experiments about antiretoviral treatments via numerical simulation and analyse the experimental results. It is noted that this modeling approach can help to provide a theoretical explanation of the clinical results. The analysis result in the paper could imply that the protocol of the experiment might enhance the immune response against HIV.

Observability criteria for impulsive control systems with applications to biomedical engineering processes

One of the fundamental properties of the impulsive systems is analyzed: observability. Algebraic criteria for testing this property are obtained for the nonlinear case, considering continuous and discrete outputs. For this class of systems, observability is explored not only through time derivatives of the output, but also considering few discrete measurements at different time-instants. In this context, it is shown that nonlinear impulsive control systems can be strongly observable or observable over a finite time interval. A new rank condition based on the structure of the impulses is found to characterize observability of linear impulsive systems. It generalizes the celebrated Kalman criterion, for both kind of outputs, discrete and continuous. Finally, these results are tested and illustrated both on academic examples and on two impulsive dynamical models from biomedical engineering science.

Optimizing thymic recovery in HIV patients through multidrug therapies

Abstract An optimal control approach based on an enlarged nonlinear model for the dynamics of HIV infection and thymic function is composed to simulate and evaluate antiretroviral therapies. In addition to the relevant biological agents, an extra state variable is included, associated with the thymus capacity for healthy cells production. The methodology contemplates eventual deleterious effects of drugs over childrens thymus recovery. The intake of ‘Reverse Transcriptase Inhibitors’ and ‘Protease Inhibitors’ are modeled as two independent control variables, each affecting a different term in the dynamics, so extending the prevailing pure-HAART-therapy analysis. The objective function designed here is also more inclusive than usual, accounting for the costs of the two drug families involved and for the thymus deterioration, in addition to penalizing eventual virus excess and healthy cells deficits. The search for the best combined therapy is treated as an optimal control problem. A hybrid version of Dynamic Programming for continuous and discrete variables is used to treat the problem numerically. Long time-horizons are explored, aiming to avoid typical peaks in drug prescriptions found at the beginning and at the end of the optimization periods. Results indicate that certain combinations of drugs are more convenient than pure protocols when the value of thymus functioning is relevant, specially for children patients.

Mathematical Modeling of HIV Dynamics After Antiretroviral Therapy Initiation: A Review

Abstract This review shows the potential ground-breaking impact that mathematical tools may have in the analysis and the understanding of the HIV dynamics. In the first part, early diagnosis of immunological failure is inferred from the estimation of certain parameters of a mathematical model of the HIV infection dynamics. This method is supported by clinical research results from an original clinical trial: data just after 1 month following therapy initiation are used to carry out the model identification. The diagnosis is shown to be consistent with results from monitoring of the patients after 6 months. In the second part of this review, prospective research results are given for the design of individual anti-HIV treatments optimizing the recovery of the immune system and minimizing side effects. In this respect, two methods are discussed. The first one combines HIV population dynamics with pharmacokinetics and pharmacodynamics models to generate drug treatments using impulsive control systems. The second one is based on optimal control theory and uses a recently published differential equation to model the side effects produced by highly active antiretroviral therapy therapies. The main advantage of these revisited methods is that the drug treatment is computed directly in amounts of drugs, which is easier to interpret by physicians and patients.

Taking Side Effects into Account for HIV Medication

A control-theoretic approach to the problem of designing “low-side-effects” therapies for HIV patients based on highly active drugs is substantiated here. The evolution of side effects during treatment is modeled by an extra differential equation coupled to the dynamics of virions, healthy T-cells, and infected ones. The new equation reflects the dependence of collateral damages on the amount of each dose administered to the patient and on the evolution of the viral load detected by periodical blood analysis. The cost objective accounts for recommended bounds on healthy cells and virions, and also penalizes the appearance of collateral morbidities caused by the medication. The optimization problem is solved by a hybrid dynamic programming scheme that adhere to discrete-time observation and control actions, but by maintaining the continuous-time setup for predicting states and side effects. The resulting optimal strategies employ less drugs than those prescribed by previous optimization studies, but maintaining high doses at the beginning and the end of each period of six months. If an inverse discount rate is applied to favor early actions, and under a mild penalization of the final viral load, then the optimal doses are found to be high at the beginning and decrease afterward, thus causing an apparent stabilization of the main variables. But in this case, the final viral load turns higher than acceptable.

Online suboptimal control of linearized models

A novel approach to approximately solving the restricted-control linear quadratic regulator problem online is substantiated and applied in two case studies. The first example is a one-dimensional system whose exact solution is known. The other one refers to the temperature control of a metallic strip at the exit of a multi-stand rolling mill. The new (online-feedback) strategy employs a convenient version of the gradient method, where partial derivatives of the cost are taken with respect to the final penalization matrix coefficients and to the switching times where the control (de)saturates. The calculations are based on exact algebraic formula, which do not involve trajectory simulations, and so reducing in principle the computational effort associated with receding horizon or nonlinear programming methods.

Initial values for Riccati ODEs from variational PDEs

The recently discovered variational PDEs (partial differential equations) for finding missing boundary conditions in Hamilton equations of optimal control are applied to the extended-space transformation of time-variant linear-quadratic regulator (LQR) problems. These problems become autonomous but with nonlinear dynamics and costs. The numerical solutions to the PDEs are checked against the analytical solutions to the original LQR problem. This is the first validation of the PDEs in the literature for a nonlinear context. It is also found that the initial value of the Riccati matrix can be obtained from the spatial derivative of the Hamiltonian flow, which satisfies the variational equation. This last result has practical implications when implementing two-degrees-of freedom control strategies for nonlinear systems with generalized costs.