Alexander Zadorojniy

A Strongly Polynomial Algorithm for Controlled Queues

We consider the problem of computing optimal policies of finite-state finite-action Markov decision processes (MDPs). A reduction to a continuum of constrained MDPs (CMDPs) is presented such that the optimal policies for these CMDPs constitute a path in a graph defined over the deterministic policies. This path contains, in particular, an optimal policy of the original MDP. We present an algorithm based on this new approach that finds this path, and thus an optimal policy. In the general case, this path might be exponentially long in the number of states and actions. We prove that the length of this path is polynomial if the MDP satisfies a coupling property. Thus we obtain a strongly polynomial algorithm for MDP s that satisfies the coupling property. We prove that discrete time versions of controlled M/M/1 queues induce MDP s that satisfy the coupling property. The only previously known polynomial algorithm for controlled M/M/1 queues in the expected average cost model is based on linear programming (and is not known to be strongly polynomial). Our algorithm works both for the discounted and expected average cost models, and the running time does not depend on the discount factor.

Real-Time video streaming in multi-hop wireless static ad hoc networks

We deal with the problem of streaming multiple video streams between pairs of nodes in a multi-hop wireless ad hoc network. The nodes are static, know their locations, and are synchronized (via GPS). We introduce a new interference model that uses variable interference radiuses. We present an algorithm for computing a frequency assignment and a schedule whose goal is to maximize throughput over all the video streams. In addition, we developed a localized flow-control mechanism to stabilize the queue lengths. We simulated traffic scheduled by the algorithm using OMNET++/MixiM (i.e., physical SINR interference model with 802.11g) to test whether the computed throughput is achieved. The results of the simulation show that the computed solution is sinr-feasible and achieves predictable stable throughputs.

Relieving Pressure: Optimizing Water Distribution Pressure Management at Valley of the Moon Water District

Efficiently managing pressure in a water distribution network is an issue for water utilities worldwide. Utilities must maintain a delicate balance between lowering the pressure as much as possible to reduce water loss and electricity usage, while keeping pressure high enough to maintain the required level of service. In this paper, we describe how we created and deployed an advanced decision support solution to help the Valley of the Moon Water District VOMWD in Sonoma County, California improve its pressure management. Our solution enables VOMWD to efficiently manage water pressure in its network and better handle the pressure changes resulting from seasonal variations in demand. It provides a comprehensive view of the pressure status in the network and incorporates a novel optimization algorithm and problem formulation, which efficiently solve a nonconvex optimization problem and provide recommendations for demand and input pressure changes in the network. Following the deployment of our solution, VOMWD reduced the number of leaks and bursts by 16 percent compared to the previous year and by 19 percent compared to the average of the previous three years. Less quantifiable results included a reduction in pressure spikes and improvements in tank water levels and water turnover. Our solution has widespread applicability; therefore, we plan to use it to help water utilities worldwide significantly improve their pressure-management capabilities.

Strong polynomiality of the Gass-Saaty shadow-vertex pivoting rule for controlled random walks

We consider the subclass of linear programs that formulate Markov Decision Processes (mdps). We show that the Simplex algorithm with the Gass-Saaty shadow-vertex pivoting rule is strongly polynomial for a subclass of mdps, called controlled random walks (CRWs); the running time is O(|S|3⋅|U|2), where |S| denotes the number of states and |U| denotes the number of actions per state. This result improves the running time of Zadorojniy et al. (Mathematics of Operations Research 34(4):992–1007, 2009) algorithm by a factor of |S|. In particular, the number of iterations needed by the Simplex algorithm for CRWs is linear in the number of states and does not depend on the discount factor.

Emotion-awareness for intelligent vehicle assistants: a research agenda

EVA is describing a new class of emotion-aware autonomous systems delivering intelligent personal assistant functionalities. EVA requires a multi-disciplinary approach, combining a number of critical building blocks into a cybernetics systems/software architecture: emotion aware systems and algorithms, multimodal interaction design, cognitive modelling, decision making and recommender systems, emotion sensing as feedback for learning, and distributed (edge) computing delivering cognitive services.

IBM Cognitive Technology Helps Aqualia to Reduce Costs and Save Resources in Wastewater Treatment

This work addresses operational management optimization problems in wastewater treatment plants. We developed a novel technology that allows control of such plants, based on real-time sensor readings, with cloud computing at the front end and state-of-the-art operations research and data science algorithms at the back end. We used a constrained Markov decision process as the key optimization framework. We tested our technology in a one-year pilot at a plant in Lleida, Spain, operated by Aqualia, the world’s third-largest water company. The results showed a dramatic 13.5 percent general reduction in the plant’s electricity consumption, a 14 percent reduction in the amount of chemicals needed to remove phosphorus from the water, and a 17 percent reduction in sludge production. Moreover, results showed a significant improvement in total nitrogen removal, especially in low temperature conditions.