Markus Peters

A reinforcement learning approach to autonomous decision-making in smart electricity markets

The vision of a Smart Electric Grid relies critically on substantial advances in intelligent decentralized control mechanisms. We propose a novel class of autonomous broker agents for retail electricity trading that can operate in a wide range of Smart Electricity Markets, and that are capable of deriving long-term, profit-maximizing policies. Our brokers use Reinforcement Learning with function approximation, they can accommodate arbitrary economic signals from their environments, and they learn efficiently over the large state spaces resulting from these signals. We show how feature selection and regularization can be leveraged to automatically optimize brokers for particular market conditions, and demonstrate the performance of our design in extensive experiments using real-world energy market data.

A multiagent competitive gaming platform to address societal challenges

The shift toward sustainable electricity systems is one of the grand challenges of the 21st century. Decentralized production from renewable sources, electric mobility, and related advances are at odds with traditional power systems where central, large-scale generation of electricity follows inelastic consumer demand. Information systems innovations can enable new forms of dynamic electricity trading that leverage real-time consumption information and that use price signals to incentivize sustainable consumption behaviors. However, the best designs for these innovations, and the societal implications of different design choices, are largely unclear. We are addressing these challenges through the Power Trading Agent Competition (Power TAC), a competitive gaming platform on which numerous research groups now jointly devise, benchmark, and improve IS-based solutions to the sustainable electricity challenge. Based on the Power TAC communitys results, we give preliminary empirical evidence for the efficacy of competitive gaming platforms, and for the communitys contributions toward resolving the sustainable electricity challenge.

Competitive benchmarking: an IS research approach to address wicked problems with big data and analytics

Wicked problems like sustainable energy and financial market stability are societal challenges that arise from complex sociotechnical systems in which numerous social, economic, political, and technical factors interact. Understanding and mitigating these problems requires research methods that scale beyond the traditional areas of inquiry of information systems (IS) individuals, organizations, and markets and that deliver solutions in addition to insights. We describe an approach to address these challenges through competitive benchmarking (CB), a novel research method that helps interdisciplinary research communities tackle complex challenges of societal scale by using different types of data from a variety of sources such as usage data from customers, production patterns from producers, public policy and regulatory constraints, etc. for a given instantiation. Further, the CB platform generates data that can be used to improve operational strategies and judge the effectiveness of regulatory regimes and policies. We describe our experience applying CB to the sustainable energy challenge in the Power Trading Agent Competition (Power TAC) in which more than a dozen research groups from around the world jointly devise, benchmark, and improve IS-based solutions.

Autonomous Data-Driven decision-making in smart electricity markets

For the vision of a Smart Grid to materialize, substantial advances in intelligent decentralized control mechanisms are required. We propose a novel class of autonomous broker agents for retail electricity trading that can operate in a wide range of Smart Electricity Markets, and that are capable of deriving long-term, profit-maximizing policies. Our brokers use Reinforcement Learning with function approximation, they can accommodate arbitrary economic signals from their environments, and they learn efficiently over the large state spaces resulting from these signals. Our design is the first that can accommodate an offline training phase so as to automatically optimize the broker for particular market conditions. We demonstrate the performance of our design in a series of experiments using real-world energy market data, and find that it outperforms previous approaches by a significant margin.

A scalable preference model for autonomous decision-making

Emerging domains such as smart electric grids require decisions to be made autonomously, based on the observed behaviors of large numbers of connected consumers. Existing approaches either lack the flexibility to capture nuanced, individualized preference profiles, or scale poorly with the size of the dataset. We propose a preference model that combines flexible Bayesian nonparametric priors—providing state-of-the-art predictive power—with well-justified structural assumptions that allow a scalable implementation. The Gaussian process scalable preference model via Kronecker factorization (GaSPK) model provides accurate choice predictions and principled uncertainty estimates as input to decision-making tasks. In consumer choice settings where alternatives are described by few key attributes, inference in our model is highly efficient and scalable to tens of thousands of choices.