Ashutosh Nayyar

Optimal Strategies for Communication and Remote Estimation With an Energy Harvesting Sensor

We consider a remote estimation problem with an energy harvesting sensor and a remote estimator. The sensor observes the state of a discrete-time source which may be a finite state Markov chain or a multidimensional linear Gaussian system. It harvests energy from its environment (say, for example, through a solar cell) and uses this energy for the purpose of communicating with the estimator. Due to randomness of the energy available for communication, the sensor may not be able to communicate all of the time. The sensor may also want to save its energy for future communications. The estimator relies on messages communicated by the sensor to produce real-time estimates of the source state. We consider the problem of finding a communication scheduling strategy for the sensor and an estimation strategy for the estimator that jointly minimizes the expected sum of communication and distortion costs over a finite time horizon. Our goal of joint optimization leads to a decentralized decision-making problem. By viewing the problem from the estimators perspective, we obtain a dynamic programming characterization for the decentralized decision-making problem that involves optimization over functions. Under some symmetry assumptions on the source statistics and the distortion metric, we show that an optimal communication strategy is described by easily computable thresholds and that the optimal estimate is a simple function of the most recently received sensor observation.

Decentralized Stochastic Control with Partial History Sharing: A Common Information Approach

A general model of decentralized stochastic control called partial history sharing information structure is presented. In this model, at each step the controllers share part of their observation and control history with each other. This general model subsumes several existing models of information sharing as special cases. Based on the information commonly known to all the controllers, the decentralized problem is reformulated as an equivalent centralized problem from the perspective of a coordinator. The coordinator knows the common information and selects prescriptions that map each controllers local information to its control actions. The optimal control problem at the coordinator is shown to be a partially observable Markov decision process (POMDP) which is solved using techniques from Markov decision theory. This approach provides 1) structural results for optimal strategies and 2) a dynamic program for obtaining optimal strategies for all controllers in the original decentralized problem. Thus, this approach unifies the various ad-hoc approaches taken in the literature. In addition, the structural results on optimal control strategies obtained by the proposed approach cannot be obtained by the existing generic approach (the person-by-person approach) for obtaining structural results in decentralized problems; and the dynamic program obtained by the proposed approach is simpler than that obtained by the existing generic approach (the designers approach) for obtaining dynamic programs in decentralized problems.

Optimal Control Strategies in Delayed Sharing Information Structures

The n-step delayed sharing information structure is investigated. This information structure comprises of K controllers that share their information with a delay of n time steps. This information structure is a link between the classical information structure, where information is shared perfectly between the controllers, and a non-classical information structure, where there is no “lateral” sharing of information among the controllers. Structural results for optimal control strategies for systems with such information structures are presented. A sequential methodology for finding the optimal strategies is also derived. The solution approach provides an insight for identifying structural results and sequential decomposition for general decentralized stochastic control problems.

A Wireless Soil Moisture Smart Sensor Web Using Physics-Based Optimal Control: Concept and Initial Demonstrations

This paper introduces a new concept for a smart wireless sensor web technology for optimal measurements of surface-to-depth profiles of soil moisture using in-situ sensors. The objective of the technology, supported by the NASA Earth Science Technology Office Advanced Information Systems Technology program, is to enable a guided and adaptive sampling strategy for the in-situ sensor network to meet the measurement validation objectives of spaceborne soil moisture sensors. A potential application for this technology is the validation of products from the Soil Moisture Active/Passive (SMAP) mission. Spatially, the total variability in soil-moisture fields comes from variability in processes on various scales. Temporally, variability is caused by external forcings, landscape heterogeneity, and antecedent conditions. Installing a dense in-situ network to sample the field continuously in time for all ranges of variability is impractical. However, a sparser but smarter network with an optimized measurement schedule can provide the validation estimates by operating in a guided fashion with guidance from its own sparse measurements. The feedback and control take place in the context of a dynamic physics-based hydrologic and sensor modeling system. The overall design of the smart sensor web-including the control architecture, physics-based hydrologic and sensor models, and actuation and communication hardware-is presented in this paper. We also present results illustrating sensor scheduling and estimation strategies as well as initial numerical and field demonstrations of the sensor web concept. It is shown that the coordinated operation of sensors through the control policy results in substantial savings in resource usage.

Aggregate flexibility of a collection of loadsπ

We consider a collection of flexible loads. Each load is modeled as requiring energy E on a service interval [a; d] at a maximum rate of m. The collection is serviced by available generation g(t) which must be allocated causally to the various tasks. Our objective is to characterize the aggregate flexibility offered by this collection. In the absence of rate limits, we offer necessary and sufficient conditions for the generation g(t) to service the loads under causal scheduling without surplus or deficit. Our results show that the flexibility in the collection can be modeled as electricity storage. The capacity Q(t) and maximum charge/discharge rate m(t) of the equivalent storage can be computed in real time. Ex ante, these parameters must be estimated based on arrival/departure statistics and charging needs. Thus, the collection is equivalent a stochastic time-varying electricity storage. We next consider the case with charging rate limits. Here, we offer bounds on the capacity and rate of the equivalent electricity storage. We offer synthetic examples to illustrate our results.

Identifying tractable decentralized control problems on the basis of information structure

Sequential decomposition of two general models of decentralized systems with non-classical information structures is presented. In model A, all agents have two observations at each step: a common observation that all agents observe and a private observation of their own. The control actions of each agent is based on all past common observations, the current private observation and the contents of its memory. At each step, each agent also updates the contents of its memory. A cost function, which depends on the state of the plant and the control actions of all agents, is given. The objective is to choose control and memory update functions for all agents to either minimize a total expected cost over a finite horizon or to minimize a discounted cost over an infinite horizon. In model B, the agents do not have any common observation, the rest is same as in model A. The key idea of our solution methodology is the following. From the point of view of a fictitious agent that observes all common observations, the system can be viewed as a centralized system with partial observations. This allows us to identify information states and obtain a sequential decomposition. When the system variables take values in finite sets, the optimality equations of the sequential decomposition are similar to those of partially observable Markov decision processes (POMDP) with finite state and action spaces. For such systems, we can use algorithms for POMDPs to compute optimal designs for models A and B.

Common Information Based Markov Perfect Equilibria for Stochastic Games With Asymmetric Information: Finite Games

A model of stochastic games where multiple controllers jointly control the evolution of the state of a dynamic system but have access to different information about the state and action processes is considered. The asymmetry of information among the controllers makes it difficult to compute or characterize Nash equilibria. Using the common information among the controllers, the game with asymmetric information is used to construct another game with symmetric information such that the equilibria of the new game can be transformed to equilibria of the original game. Further, under certain conditions, a Markov state is identified for the new symmetric information game and its Markov perfect equilibria are characterized. This characterization provides a backward induction algorithm to find Nash equilibria of the original game with asymmetric information in pure or behavioral strategies. Each step of this algorithm involves finding Bayesian Nash equilibria of a one-stage Bayesian game. The class of Nash equilibria of the original game that can be characterized in this backward manner are named common information based Markov perfect equilibria.

Measurement Scheduling for Soil Moisture Sensing: From Physical Models to Optimal Control

In this paper, we consider the problem of monitoring soil moisture evolution using a wireless network of in situ sensors. Continuously sampling moisture levels with these sensors incurs high-maintenance and energy consumption costs, which are particularly undesirable for wireless networks. Our main hypothesis is that a sparser set of measurements can meet the monitoring objectives in an energy-efficient manner. The underlying idea is that we can trade off some inaccuracy in estimating soil moisture evolution for a significant reduction in energy consumption. We investigate how to dynamically schedule the sensor measurements so as to balance this tradeoff. Unlike many prior studies on sensor scheduling that make generic assumptions on the statistics of the observed phenomenon, we obtain statistics of soil moisture evolution from a physical model. We formulate the optimal measurement scheduling and estimation problem as a partially observable Markov decision problem (POMDP). We then utilize special features of the problem to approximate the POMDP by a computationally simpler finite-state Markov decision problem (MDP). The result is a scalable, implementable technology that we have tested and validated numerically and in the field.

A dynamic transmitter-jammer game with asymmetric information

We consider a jamming attack on a transmitter-receiver pair, in which the transmitter wants to transmit the state of an i.i.d. Gaussian process across an unsecured communication channel to the receiver while minimizing its cost functional. The transmitter decides whether or not to transmit the current state of the random process. The jammer disrupts the transmission on the channel strategically in order to increase the total cost to the transmitter, but can do this only a limited number of times over the entire horizon. The jammer only detects whether or not a transmission is happening over the channel, but does not observe the state of the random process being transmitted. This leads to a dynamic zero-sum game with asymmetric information between the transmitter and the jammer. We prove that the saddle-point strategy of the transmitter is threshold-based and that under certain conditions, the jammer plays a mixed strategy.

Structural results and explicit solution for two-player LQG systems on a finite time horizon

It is well-known that linear dynamical systems with Gaussian noise and quadratic cost (LQG) satisfy a separation principle. Finding the optimal controller amounts to solving separate dual problems; one for control and one for estimation. For the discrete-time finite-horizon case, each problem is a simple forward or backward recursion. In this paper, we consider a generalization of the LQG problem with two controllers and a partially nested information structure. Each controller is responsible for one of two system inputs, but has access to different subsets of the available measurements. Our paper has three main contributions. First, we prove a fundamental structural result: sufficient statistics for the controllers can be expressed as conditional means of the global state. Second, we give explicit state-space formulae for the optimal controller. These formulae are reminiscent of the classical LQG solution with dual forward and backward recursions, but with the important difference that they are intricately coupled. Lastly, we show how these recursions can be solved efficiently, with computational complexity comparable to that of the centralized problem.