Gregory B. Miller

Dynamic Discrete Power Control in Cellular Networks

We consider an uplink power control problem where each mobile wishes to maximize its throughput (which depends on the transmission powers of all mobiles) but has a constraint on the average power consumption. A finite number of power levels are available to each mobile. The decision of a mobile to select a particular power level may depend on its channel state. We consider two frameworks concerning the state information of the channels of other mobiles: i) the case of full state information and ii) the case of local state information. In each of the two frameworks, we consider both cooperative as well as non-cooperative power control. We manage to characterize the structure of equilibria policies and, more generally, of best-response policies in the non-cooperative case. We present an algorithm to compute equilibria policies in the case of two non-cooperative players. Finally, we study the case where a malicious mobile, which also has average power constraints, tries to jam the communication of another mobile. Our results are illustrated and validated through various numerical examples.

Methods to design optimal control of Markov process with finite state set in the presence of constraints

The problem of optimal control of a nonuniform Markov process with a finite state set over a fixed interval in the presence of inequality-like constraints was considered. The design of control relies on the principle of dynamic programming in combination with the methods of convex programming and the duality theory. Two types of conditions under which it is possible to select a Markov optimal control were proposed.

Brief paper: Towards the optimal control of Markov chains with constraints

An optimal control problem with constraints is considered on a finite interval for a non-stationary Markov chain with a finite state space. The constraints are given as a set of inequalities. The optimal solution existence is proved under a natural assumption that the set of admissible controls is non-empty. The stochastic control problem is reduced to a deterministic one and it is shown that the optimal solution satisfies the maximum principle, moreover it can be chosen within a class of Markov controls. On the basis of this result an approach to the numerical solution is proposed and its implementation is illustrated by examples.

Hidden Markov model approach to TCP link state tracking

A new model of TCP link state evolution In the settings of hidden Markov models (HMM) is introduced. A class of special Markov processes, which can be considered as generalization of the classic Markov chains, is suggested. Solution of the optimal filtering problem given combined noised and unnoised indirect observations is also presented. The result is implemented to the tracking problem for the couple TCP link condition - round trip time (RTT) given both the noised RTT measurements and packet losses. The presented estimate is compared with the standard realized in TCP and the optimal filtering one given packet losses only.

Optimal control of Markov chains with constraints

A problem of optimal control of Markov chain with finite state space is considered. We consider a non-stationary finite horizon problem with constraints, given as a set of inequalities. Basing on recent results on existence of optimal solution we suggest to use the dual approach to optimization and thereby an approach to effective numerical algorithms. The approach is illustrated by numerical examples.

Minimax control of a process in a linear uncertain-stochastic system with incomplete data

Consideration is given to the control problem in a linear stochastic differential system where constant noise intensities in equations of state and observation are prescribed only accurate within the membership of some known sets. For control optimization, an integral root-mean-square performance criterion is used. The problem is solved by the transition to a dual one, which makes it possible to prove the existence of a saddle point of the criterion and obtain an explicit expression for the minimax control operator as functions of the solution to the dual problem. To solve the latter, an iteration algorithm is proposed; the convergence of the algorithm is proved and investigated by a model example.

Optimal Channel Choice for Lossy Data Flow Transmission

We consider the optimal control problem for the load of several communication channels defined by independent Markov jump processes. Implicit information on the state of a channel is available in the form of a flow of losses whose intensity is proportional to the controllable load of this channel. The optimized functionals take into account the total throughput of channels and energy costs for data transmission over a fixed interval of time. We obtain optimal filtering equations for joint estimation of channel states. We construct a locally optimal strategy that explicitly depends on the set of state estimates.

Optimal control problem regularization for the Markov process with finite number of states and constraints

The optimal control problem is considered for a system given by the Markov chain with integral constraints. It is shown that the solution to the optimal control problem on the set of all predictable controls satisfies Markov property. This optimal Markov control can be obtained as a solution of the corresponding dual problem (in case if the regularity condition holds) or (in other case) by means of proposed regularization method. The problems arising due to the system nonregularity along with the way to cope with those problems are illustrated by an example of optimal control problem for a single channel queueing system.

The Conditionally Minimax Nonlinear Filtering Method and Modern Approaches to State Estimation in Nonlinear Stochastic Systems

We consider, in chronological order, the main results that have defined the concept of conditionally minimax nonlinear filtering. This would let us to follow all the evolution stages of this universal method, from a particular application, through basic mathematical concepts, to an advanced theory able to solve a wide class of robust estimation problems in linear and nonlinear stochastic systems.

Modeling and Monitoring of RTP Link on the Receiver Side

The paper presents a new mathematical model of a link carrying by the Real Time Transport Protocol. The model attempts to meet the key features of the real link functioning like the frame delays, losses, bursting reception etc. The proposed approach is based on the Hidden Markov concept. The unobservable state is assumed to be a finite-dimensional Markov process. The observation is a non-Markovian multivariate point process that indicates heterogenous frames reception. The paper also contains the formulation and solution to the filtering problem of the hidden link state given the observable multivariate point process. Proposed link model validity and filtering algorithm performance are illustrated by processing of captured real video streams delivered via 3G mobile network.