Rajesh Bansal

Simultaneous design of measurement and control strategies for stochastic systems with feedback

Abstract We consider stochastic dynamic decision problems where at each step two consecutive decisions must be taken, one being what information bearing signal(s) to transmit and the other what control action(s) to exert. For finite-horizon problems involving first-order ARMA models with Gaussian statistics and a quadratic cost criterion, we show that the optimal measurement strategy consists of transmitting the innovation linearly at each stage, which in turn leads to optimality of a linear control law. We then extend this result to infinite-horizon models with discounted costs, showing optimality of linear designs. Subsequently, we show that these appealing results do not necessarily carry over to higher order ARMA models, for which we first characterize the best designs within the affine class, and then derive instances of the problem for which there exist non-linear designs that outperform the optimal designs within the affine class. The paper also includes some illustrative numerical examples on the different classes of problems considered.

Stochastic teams with nonclassical information revisited: When is an affine law optimal?

In this note we consider a parameterized family of two-stage stochastic control problems with nonclassical information patterns, which includes the well-known 1968 counterexample of Witsenhausen. We show that whenever the performance index does not contain a product term between the decision variables, the optimal solution is linear in the observation variables. The parameter space can be partitioned into two regions in one of which the optimal solution is linear, whereas in the other it is inherently nonlinear. Extensive computations using two-point piecewise constant policies and linear plus piecewise constant policies provide numerical evidence that nonlinear policies may indeed outperform linear policies when the product term is present.

Optimum design of measurement channels and control policies for linear-quadratic stochastic systems

Abstract In the design of optimal controllers for linear-quadratic stochastic systems, a standard assumption is that the measurement channels are fixed and linear, and the measurement noise is Gaussian. In this paper we relax the first part of this restriction and raise the issue of the derivation of optimum measurement structures as a part of the overall design. Toward this end, we take the measuement process as one given by a Wiener integral, and modify the cost function so that it now places some soft constraints on the measurement strategy. Using some results from information theory, we show that the scalar version (for both finite and infinite horizons) of this joint design problem admits an optimum, dictating linear designs for both the controller and the measurement strategy. For the vector version, however, it is possible for a nonlinear design to improve over the best linear one. In both cases, best linear designs involve the solutions of nonlinear (deterministic) optimal control problems.

Solutions to a class of linear-quadratic-Gaussian (LQG) stochastic team problems with nonclassical information

We consider a stochastic dynamic team problem with two controllers and nonclassical information, which can be viewed as the transmission of a garbled version of a Gaussian message over a number of noisy channels under a fidelity criterion. We show that the optimum solution (under a quadratic loss functional) consists of linearly transforming the garbled message to a certain (optimum) power level P* and then optimally decoding it by using a linear transformation at the receiving end. The power level P* is determined by the solution of a fifth order algebraic equation. The paper also discusses an extension of this result to the case when the channel noise is correlated with the input random variable, and shows that for the single channel case the optimum solution is again linear.

Communication games with partially soft power constraints

We consider a class of communication games which involves the transmission of a Gaussian random variable through a conditionally Gaussian memoryless channel in the presence of an intelligent jammer. The jammer is allowed to tap the channel and feed a correlated signal back into it. The transmitter-receiver pair is assumed to cooperate in minimizing some quadratic fidelity criterion while the jammer maximizes this same criterion. Security strategies which protect against irrational jammer behavior and which yield an upper bound on the cost are shown to exist for the transmitter-receiver pair over a class of fidelity criteria. Closed-form expressions for these strategies are provided in the paper, which are, in all cases but one, linear in the available information.

On problems exhibiting a dual role of control

Some of the simplest classes of linear-quadratic-Gaussian (LQG) problems involving the dual role of control are studied. Neither analytical tools nor numerical methods are known to exist for such problems. This dual role signifies that in addition to minimizing a quadratic cost, the control explicitly determines the quality of information transmitted to the next stage. It is shown that indirect techniques (information-theoretic bounds or the construction of related zero-sum games) can be used to obtain the optimal solution for some classes of such problems; these optimal solutions are linear and can be found by solving a related parameter optimization problem. For other classes of problems it is established that nonlinear strategies indeed outperform the optimal linear strategies.<<ETX>>

On decentralized problems with non-nested information patterns

Some of the issues related to information sharing that arise in a variety of situations are explored, particularly emphasizing some applications in the areas of automatic control and information sharing in organizations. A brief overview of information patterns is provided, and the notion of nestedness is developed. Prototype problems with nonnested information that arise in the area of automatic control are studied. Information sharing in organizations is then considered.<<ETX>>

Joint estimation and control for a class of stochastic dynamic teams

The authors formulate and solve an infinite-horizon stochastic optimization problem where both the control and the measurement strategies are to be designed simultaneously, under a quadratic performance index. The complete solution to the infinite-horizon problem is provided, the existence of optimal stationary policies is established, and an algorithm for the numerical computation of these policies is given. Thus linear stationary policies are overall optimal and can be obtained from the solution of an infinite-horizon nonlinear deterministic optimal control problem.<<ETX>>