Mrinal K. Ghosh

Discrete-time controlled Markov processes with average cost criterion: a survey

This work is a survey of the average cost control problem for discrete-time Markov processes. The authors have attempted to put together a comprehensive account of the considerable research on this problem over the past three decades. The exposition ranges from finite to Borel state and action spaces and includes a variety of methodologies to find and characterize optimal policies. The authors have included a brief historical perspective of the research efforts in this area and have compiled a substantial yet not exhaustive bibliography. The authors have also identified several important questions that are still open to investigation.

Optimal control of switching diffusions with application to flexible manufacturing systems

A controlled switching diffusion model is developed to study the hierarchical control of flexible manufacturing systems. The existence of a homogeneous Markov nonrandomized optimal policy is established by a convex analytic method. Using the existence of such a policy, the existence of a unique solution in a certain class to the associated Hamilton-Jacobi-Bellman equations is established and the optimal policy is characterized as a minimizing selector of an appropriate Hamiltonian.

Ergodic Control of Switching Diffusions

We study the ergodic control problem of switching diffusions representing a typical hybrid system that arises in numerous applications such as fault-tolerant control systems, flexible manufacturing systems, etc. Under fairly general conditions, we establish the existence of a stable, nonrandomized Markov policy which almost surely minimizes the pathwise long-run average cost. We then study the corresponding Hamilton--Jacobi--Bellman (HJB) equation and establish the existence of a unique solution in a certain class. Using this, we characterize the optimal policy as a minimizing selector of the Hamiltonian associated with the HJB equations. As an example we apply the results to a failure-prone manufacturing system and obtain closed form solutions for the optimal policy.

Ergodic control of multidimensional diffusions 1: the existence results

The existence of optimal stable Markov relaxed controls for the ergodic control of multidimensional diffusions is established by direct probabilistic methods based on a characterization of a.s. limit sets of empirical measures. The optimality of the above is established in the strong (i.e., almost sure) sense among all admissible controls under very general conditions.

Ergodic control of degenerate diffusions

We study the ergodic control problem of degenerate diffusions on RdUnder a certain Liapunov type stability condition we establish the existence of an optimal control. We then study the correponding HJB equation and establish the existence of a unique viscosity solution in a certain class

Risk Minimizing Option Pricing in a Regime Switching Market

Abstract We study option pricing in a regime switching market where the risk free interest rate, growth rate and the volatility of a stock depends on a finite state Markov chain. Using a minimal martingale measure we show that the risk minimizing option price satisfies a system of Black–Scholes partial differential equations with weak coupling.

Ergodic control of multidimensional diffusions. II: Adaptive control

The self-tuning scheme for the adaptive control of a diffusion process is studied with long-run average cost criterion and maximum likelihood estimation of parameters. Asymptotic optimality under a suitable identifiability condition is established under two alternative sets of hypotheses—a Lyapunov-type stability criterion and a condition on cost which penalizes instability.

Denumerable state stochastic games with limiting average payoff

We study stochastic games with countable state space, compact action spaces, and limiting average payoff. ForN-person games, the existence of an equilibrium in stationary strategies is established under a certain Liapunov stability condition. For two-person zero-sum games, the existence of a value and optimal strategies for both players are established under the same stability condition.

Harnack's inequality for cooperative weakly coupled elliptic systems

Abstract : The authors consider cooperative, uniformly elliptic systems, with bounded coefficients and coupling in the zeroth-order terms. They establish two analogs of Harnacks inequality for this class of system. A weak version is obtained under fairly general conditions, while imposing an irreducibility condition on the coupling coefficients results in a stronger version of the inequality. This irreducibility condition also is necessary for the existence of a Harnack constant for this class of systems. A Harnack inequality also is obtained for a class of superharmonic functions.

Risk Minimizing Option Pricing in a Semi-Markov Modulated Market

We address risk minimizing option pricing in a semi-Markov modulated market where the floating interest rate depends on a finite state semi-Markov process. The growth rate and the volatility of the stock also depend on the semi-Markov process. Using the Follmer-Schweizer decomposition we find the locally risk minimizing price for European options and the corresponding hedging strategy. We develop suitable numerical methods for computing option prices.