Tayfun Çimen

State-Dependent Riccati Equation (SDRE) Control: A Survey

Since the mid-90s, State-Dependent Riccati Equation (SDRE) strategies have emerged as general design methods that provide a systematic and effective means of designing nonlinear controllers, observers, and filters. These methods overcome many of the difficulties and shortcomings of existing methodologies, and deliver computationally simple algorithms that have been highly effective in a variety of practical and meaningful applications. In a special session at the 17th IFAC Symposium on Automatic Control in Aerospace 2007, theoreticians and practitioners in this area of research were brought together to discuss and present SDRE-based design methodologies as well as review the supporting theory. It became evident that the number of successful simulation, experimental and practical real-world applications of SDRE control have outpaced the available theoretical results. This paper reviews the theory developed to date on SDRE nonlinear regulation for solving nonlinear optimal control problems, and discusses issues that are still open for investigation. Existence of solutions as well as stability and optimality properties associated with SDRE controllers are the main contribution in the paper. The capabilities, design flexibility and art of systematically carrying out an effective SDRE design are also emphasized.

Nonlinear optimal tracking control with application to super-tankers for autopilot design

A new method is introduced to design optimal tracking controllers for a general class of nonlinear systems. A recently developed recursive approximation theory is applied to solve the nonlinear optimal tracking control problem explicitly by classical means. This reduces the nonlinear problem to a sequence of linear-quadratic and time-varying approximating problems which, under very mild conditions, globally converge in the limit to the nonlinear systems considered. The converged control input from the approximating sequence is then applied to the nonlinear system. The method is used to design an autopilot for the ESSO 190,000-dwt oil tanker. This multi-input-multi-output nonlinear super-tanker model is well established in the literature and represents a challenging problem for control design, where the design requirement is to follow a commanded maneuver at a desired speed. The performance index is selected so as to minimize: (a) the tracking error for a desired course heading, and (b) the rudder deflection angle to ensure that actuators operate within their operating limits. This will present a trade-off between accurate tracking and reduced actuator usage (fuel consumption) as they are both mutually dependent on each other. Simulations of the nonlinear super-tanker control model are conducted to illustrate the effectiveness of the nonlinear tracking controller.

Global optimal feedback control for general nonlinear systems with nonquadratic performance criteria

Abstract Optimal control of general nonlinear nonaffine controlled systems with nonquadratic performance criteria (that permit state- and control-dependent time-varying weighting parameters), is solved classically using a sequence of linear- quadratic and time-varying problems. The proposed method introduces an “approximating sequence of Riccati equations” (ASRE) to explicitly construct nonlinear time-varying optimal state-feedback controllers for such nonlinear systems. Under very mild conditions of local Lipschitz continuity, the sequences converge (globally) to nonlinear optimal stabilizing feedback controls. The computational simplicity and effectiveness of the ASRE algorithm is an appealing alternative to the tedious and laborious task of solving the Hamilton–Jacobi–Bellman partial differential equation. So the optimality of the ASRE control is studied by considering the original nonlinear-nonquadratic optimization problem and the corresponding necessary conditions for optimality, derived from Pontryagins maximum principle. Global optimal stabilizing state-feedback control laws are then constructed. This is compared with the optimality of the ASRE control by considering a nonlinear fighter aircraft control system, which is nonaffine in the control. Numerical simulations are used to illustrate the application of the ASRE methodology, which demonstrate its superior performance and optimality.

Asymptotically Optimal Nonlinear Filtering: Theory and Examples with Application to Target State Estimation

The State-Dependent Riccati Equation (SDRE) filter, which is derived by constructing the dual of the well-known SDRE nonlinear regulator control design technique, has been studied in various papers, with mainly practical investigations of the filter. Until recently, theoretical aspects of the filter had not been fully investigated, leaving many unanswered questions, such as stability and convergence of the filter. The authors conducted an investigation of the conditions under which the state estimate given by this algorithm converges asymptotically to the first order minimum variance estimate given by the extended Kalman filter (EKF). Conditions for determining a region of stability for the SDRE filter were also investigated. The analysis was based on stable manifold theory and Hamilton-Jacobi-Bellman (HJB) equations. In this paper, the motivation for introducing HJB equations is justified with mathematical rigor, which is given by reference to the maximum likelihood approach to deriving the EKF. The application of the SDRE filter is then demonstrated on challenging examples to illustrate the theoretical aspects and design flexibility (additional degrees of freedom) of the algorithm when loss of observability is encountered. In particular, a realistic and detailed evaluation of the filter is carried out for the problem of target state estimation in an advanced tactical missile guidance application for analysis in the optimal guidance problem for air-air engagements using only passive sensor (angle-only) information. Simulation results are presented which show dramatic tracking improvement using the SDRE target tracker.

SDRE CONTROL OF THE CONTROL ACTUATION SYSTEM OF A GUIDED MISSILE

Abstract This article illustrates the design and analysis of a nonlinear State-Dependent Riccati Equation (SDRE) controller on a control actuation system (CAS) of an aerodynamically controlled guided missile. The system consists of a DC-motor, the transmission mechanism, a control actuation surface (fin), and a position feedback device. The overall system has to satisfy the desired performance requirements in the presence of the nonlinear modeling characteristics: aerodynamic loading and backlash in the transmission mechanism, both of which are unknown to the controller. Moreover, there are uncertainties in DC-motor parameters and supplied power. The use of a state-dependent (nonquadratic) performance functional is strongly emphasized for control design purposes to show how the integration of mathematical synthesis and engineering objectives can be achieved. A series of numerical simulations are carried out to evaluate the effectiveness and robustness of the proposed nonlinear SDRE controller.

On the Existence of Solutions Characterized by Riccati Equations to Infinite-Time Horizon Nonlinear Optimal Control Problems

Abstract In a recent survey paper presented by the author during the 17th IFAC World Congress in Seoul, South Korea, in 2008, the theory developed to date on State-Dependent Riccati Equation (SDRE) control has been reviewed, discussing issues on existence of solutions as well as optimality and stability properties associated with SDRE controllers. In this study, existence of solutions associated with general infinite-time horizon nonlinear optimal control problems for nonlinear regulation of input-affine systems is considered and examined in detail, providing a link between the Hamilton-Jacobi-Bellman equation, Lagrangian manifolds and solutions characterized by Riccati equations, using stable manifold theory. The motivation for characterization of solutions to nonlinear optimal control problems by Riccati equations, in particular by symmetric positive-definite solutions, is also justified in hopes of providing a sound theoretical basis for existence of solutions of SDRE controls under very mild conditions.

ASYMPTOTICALLY OPTIMAL NONLINEAR FILTERING

Abstract In this paper, a theoretical investigation of the state-dependent Riccati equation (SDRE) filter is carried out, which is derived by constructing the dual of the well-known SDRE nonlinear regulator control design technique. The SDRE filter has been studied in various papers, with mainly practical investigations of the filter. However, the theoretical aspects of the filter have not been fully investigated and there remain many unanswered questions, such as stability and convergence of the filter. This paper investigates conditions under which the state estimate given by this algorithm converges asymptotically to the first order minimum variance estimate given by the extended Kalman filter (EKF). Conditions for determining a region of stability for the SDRE filter are also investigated. The analysis is based on stable manifold theory and Hamilton-Jacobi-Bellman (HJB) equations. The motivation for introducing HJB equations is given by reference to the maximum likelihood approach to deriving the EKF. The application of the SDRE filter will be demonstrated on a simple pendulum problem to illustrate the theory. The behavioral differences and similarities between the SDRE filter, the linearized Kalman filter (LKF) and the EKF are also discussed using this example.

APPROXIMATE NONLINEAR OPTIMAL SDRE TRACKING CONTROL

Abstract The intrinsic nonlinearities of aircraft dynamics become exceptionally significant when there are demands for large maneuvers, causing aircraft to operate near the stability limits of the system, where series problems can be encountered, since the real issue becomes dealing with nonlinearities that dominate system behavior. This has eminent consequences in aerospace applications where catastrophic failure in high-performance aircraft must be prevented. Recently, a novel algorithm has been derived for nonlinear suboptimal tracking control , which can be realized for real-time computer implementation. The algorithm is based on the State-Dependent Riccati Equation (SDRE) strategy that has become well-known within the control community. This paper focuses on illustrating the application, computational advantage and validity of the proposed tracking control methodology on a realistic simulation example of a ducted fan engine model for high-performance thrust-vectored aircraft. The proposed method delivers a computationally simple, yet effective, algorithm for constructively synthesizing nonlinear suboptimal feedback controls for trajectory tracking problems.

Optimal Control of Nonlinear Systems

In this paper we study a nonlinear optimization problem with non-linear dynamics and replace it with a sequence of time-varying linear-quadratic problem, which can be solved classically.

STOCHASTIC OPTIMAL CONTROL OF PARTIALLY OBSERVABLE NONLINEAR SYSTEMS

Abstract This paper presents a new theory for solving the continuous-time stochastic optimal control problem for a very general class of nonlinear (nonautonomous and nonaffine controlled) systems with partial state information. The proposed theory transforms the nonlinear problem into a sequence of linear-quadratic Gaussian (LQG) and time-varying problems, which converge (uniformly in time) under very mild conditions of local Lipschitz continuity. These results have been previously presented for deterministic nonlinear systems under perfect state measurements for finite horizons, but the present study shows how an additional class of nonlinear problems, involving partially observable stochastic systems, can be handled with the same theory. The method introduces an “approximating sequence of Riccati equations” ( ASRE ) to explicitly find the error covariance matrix and nonlinear time-varying optimal feedback controllers for such nonlinear systems, which is achieved using the framework of Kalman-Bucy filtering, separation principle and LQR theory. The paper shows a practical way of designing optimal feedback control systems for complex nonlinear stochastic problems using a combination of modern LQG estimation and LQ control-design methodologies.