Toshiharu Fujita

Controlled Markov Chains with Utility Functions

In this paper we consider finite-stage stochastic optimization problems of utility criterion, which is the stochastic evaluation of associative reward through a utility function. We optimize the expected value of a utility criterion not in the class of Markov policies but in the class of general policies. We show that, by expanding the state space, an invariant imbedding approach yields an recursive relation between two adjacent optimal value functions. We show that the utility problem with a general policy is equivalent to a terminal problem with a Markov policy on the augmented state space. Finally it is shown that the utility problem has an optimal policy in the class of general policies on the original state space.

An optimistic decision-making in fuzzy environment

Bellman and Zadeh have originated three systems of multistage decision processes in a fuzzy environment: deterministic, stochastic and fuzzy systems. In this article, we consider an optimization problem with an optimistic criterion on a fuzzy system. By making use of minimization-maximization expectation in a fuzzy environment, we derive a recursive equation for the fuzzy decision process through invariant imbedding approach. By illustrating a three-state, two-decision and two-stage model, we give an optimal solution through dynamic programming. The optimal solution is also verified by the method of multistage fuzzy decision tree-table.

Associative Criteria in Mutually Dependent Markov Decision Processes

In this paper, we consider associative criteria in mutually dependent Markov decision processes (MDMDP). The MDMDP model is structured upon two types of finite-stage Markov decision process: main-process and sub-process. At each stage, the reward in one process is given by the optimal value of the alternative process problem, whose initial state is determined by the current state and decision in the original process. We introduce an associative criterion to each MDMDP and derive mutually dependent recursive equations by dynamic programming with an invariant imbedding technique.

A Nondeterministic Dynamic Programming Model

In this paper we consider a dynamic programming model with nondeterministic system. Nondeterministic is a type of the transition systems. It means that a single state yields more than one state in the next stage. We newly introduce this nondeterministic system and study on related optimization problems. Nondeterministic dynamic programming covers traditional ones and has a strong possibility for applying the idea of dynamic programming to more various problems.