Jack Baczynski

Optimal Control for Continuous-Time Linear Quadratic Problems with Infinite Markov Jump Parameters

The subject matter of this paper is the optimal control problem for continuous-time linear systems subject to Markovian jumps in the parameters and the usual infinite-time horizon quadratic cost. What essentially distinguishes our problem from previous ones, inter alia, is that the Markov chain takes values on a countably infinite set. To tackle our problem, we make use of powerful tools from semigroup theory in Banach space and a decomplexification technique. The solution for the problem relies, in part, on the study of a countably infinite set of coupled algebraic Riccati equations (ICARE). Conditions for existence and uniqueness of a positive semidefinite solution of the ICARE are obtained via the extended concepts of stochastic stabilizability (SS) and stochastic detectability (SD). These concepts are couched into the theory of operators in Banach space and, parallel to the classical linear quadratic (LQ) case, bound up with the spectrum of a certain infinite dimensional linear operator.

Lyapunov coupled equations for continuous-time infinite Markov jump linear systems

This paper deals with Lyapunov equations for continuous-time Markov jump linear systems (MJLS). Out of the bent which wends most of the literature on MJLS, we focus here on the case in which the Markov chain has a countably infinite state space. It is shown that the infinite MJLS is stochastically stabilizable (SS) if and only if the associated countably infinite coupled Lyapunov equations have a unique norm bounded strictly positive solution. It is worth mentioning here that this result do not hold for mean square stabilizability (MSS), since SS and MSS are no longer equivalent in our set up (see, e.g., [J. Baczynski, Optimal control for continuous time LQ-problems with infinite Markov jump parameters, Ph.D. Thesis, Federal University of Rio de Janeiro, UFRJ/COPPE, 2000]). To some extent, a decomplexification technique and tools from operator theory in Banach space and, in particular, from semigroup theory are the very technical underpinning of the paper.

Optimal linear mean square filter for continuous-time jump linear systems

We consider a class of hybrid systems which is modeled by continuous-time linear systems with Markovian jumps in the parameters (LSMJP). Our aim is to derive the best linear mean square estimator for such systems. The approach adopted here produces a filter which bears those desirable properties of the Kalman filter: A recursive scheme suitable for computer implementation which allows some offline computation that alleviates the computational burden. Apart from the intrinsic theoretical interest of the problem in its own right and the application-oriented motivation of getting more easily implementable filters, another compelling reason why the study here is pertinent has to do with the fact that the optimal nonlinear filter for our estimation problem is not computable via a finite computation (the filter is infinite dimensional). Our filter has dimension Nn, with n denoting the dimension of the state vector and N the number of states of the Markov chain.

Stochastic versus mean square stability in continuous time linear infinite Markov jump parameter systems

We deal with linear systems with Markovian Jump Parameters (LSMJP). Most of the literature on this matter adopts a finite state space for the Markov chain. In this paper we focus on the countably infinite state space case showing that, unlike the finite state space case, two important concepts in optimal control theory, namely, stochastic stability (SS) and mean square stability (MSS) are no longer equivalent in this setting. *Research supported in part by the Brazilian National Research Council—CNPq and PRONEX.

On a discrete-time linear jump stochastic dynamic game

In this paper we optimally solve a stochastic perfectly observed dynamic game for discrete-time linear systems with Markov jump parameters (LSMJPs). The results here encompass both the cooperative and non-cooperative case. Included also, is a verification theorem. Besides being interesting in its own right, the motivation here lies, inter alia, in the results of recent vintage, which show that, for the classical linear case, the risk-sensitive optimal control problem approach is intimately bound up with the H

Maximal versus strong solution to algebraic Riccati equations arising in infinite Markov jump linear systems

Abstract We deal with a perturbed algebraic Riccati equation in an infinite dimensional Banach space which appears, for instance, in the optimal control problem for infinite Markov jump linear systems (from now on iMJLS). Infinite or finite here has to do with the state space of the Markov chain being infinite countable or finite (see, e.g., [M.D. Fragoso, J. Baczynski, Optimal control for continuous time LQ—problems with infinite Markov jump parameters, SIAM J. Control Optim. 40(1) (2001) 270–297]). By using a certain concept of stochastic stability (a sort of L 2 -stability), we have proved in [J. Baczynski, M.D. Fragoso, Maximal solution to algebraic Riccati equations linked to infinite Markov jump linear systems, Internal Report LNCC, no. 6, 2006] existence (and uniqueness) of maximal solution for this class of equations. As it is noticed in this paper, unlike the finite case (including the linear case), we cannot guarantee anymore that maximal solution is a strong solution in this setting. Via a discussion on the main mathematical hindrance behind this issue, we devise some mild conditions for this implication to hold. Specifically, our main result here is that, under stochastic stability, along with a condition related with convergence in the infinite dimensional scenario, and another one related to spectrum—weaker than spectral continuity—we ensure the maximal solution to be also a strong solution. These conditions hold trivially in the finite case, allowing us to recover the result of strong solution of [C.E. de Souza, M.D. Fragoso, On the existence of maximal solution for generalized algebraic Riccati equations arising in stochastic control, Systems Control Lett. 14 (1990) 233–239] set for MJLS. The issue of whether the convergence condition is restrictive or not is brought to light and, together with some counterexamples, unveil further differences between the finite and the infinite countable case.

Maximal solution to algebraic Riccati equations linked to infinite Markov jump linear systems

In this paper, we deal with a perturbed algebraic Riccati equation in an infinite dimensional Banach space. Besides the interest in its own right, this class of equations appears, for instance, in the optimal control problem for infinite Markov jump linear systems (from now on iMJLS). Here, infinite or finite has to do with the state space of the Markov chain being infinite countable or finite (see Fragoso and Baczynski in SIAM J Control Optim 40(1):270–297, 2001). By using a certain concept of stochastic stability (a sort of L2-stability), we prove the existence (and uniqueness) of maximal solution for this class of equation and provide a tool to compute this solution recursively, based on an initial stabilizing controller. When we recast the problem in the finite setting (finite state space of the Markov chain), we recover the result of de Souza and Fragoso (Syst Control Lett 14:233–239, 1999) set to the Markovian jump scenario, now free from an inconvenient technical hypothesis used there, originally introduced in Wonham in (SIAM J Control 6(4):681–697).

Maximal Solution to Perturbed Algebraic Riccati Equations Arising in Markovian Jump Control Revisited

In this paper we revisit the maximal solution problem studied in [7]. It is shown that, for the Markovian jump scenario, we can get rid of an inconvenient technical hypothesis used in [7] (originally introduced in [20]). This is achieved, essentially, via the mean square stability concept.

On maximal solution to infinite dimensional perturbed Riccati differential equations arising in stochastic control

Finding the maximal solution for a certain class of infinite dimensional perturbed Riccati algebraic equations is the main concern of this paper. In addition, we provide a sufficient and necessary condition for stochastic stability. Also, we obtain necessary conditions which unveil some structural properties. Besides the interest in its own right, this class of equations turns out to be essential, for instance, when dealing with linear systems with infinite countable Markov jump parameters or infinite dimensional linear time-invariant systems with state-dependent noise.

A new finite difference method for pricing and hedging fixed income derivatives

We propose a second order accurate numerical finite difference method to replace the classical schemes used to solve PDEs in financial engineering. We name it Modified Fully Implicit method. The motivation for doing so stems from the accuracy loss while trying to stabilize the solution via the up-wind scheme in the convective term as well as the fact that spurious oscillations solutions occur when volatilities are low (this is actually the range that is commonly observed in interest rate markets). Unlike the classical schemes, our method covers the whole spectrum of volatilities in the interest rate dynamics.We obtain analytical and numerical results for pricing and hedging a zero-coupon bond and an Asian interest rate option. In the case of the Asian option, we compare the realistic discrete compounding interest rate scheme (associated with the Modified Fully Implicit method) with the continuous compounding scheme (often exploited in the literature), obtaining relative discrepancies between prices exceeding 50%. This indicates that the former scheme is more appropriate then the latter to price more complicate derivatives than straight bonds.