A reinforcement learning approach to autonomous decision-making in smart electricity markets

Authors:
Markus Peters;Wolfgang Ketter;Maytal Saar-Tsechansky;John Collins
Affiliations:
Rotterdam School of Management, Erasmus University, Rotterdam, The Netherlands;Rotterdam School of Management, Erasmus University, Rotterdam, The Netherlands;McCombs School of Business, University of Texas at Austin, Austin, USA;Dept. of Computer Science and Engineering, University of Minnesota, Minneapolis, USA
Venue:
Machine Learning
Year:
2013

Citing 20
Cited 0

Genetic algorithms: foundations and applications

Annals of Operations Research
Training with noise is equivalent to Tikhonov regularization

Neural Computation
Linear least-squares algorithms for temporal difference learning

Machine Learning - Special issue on reinforcement learning
Selection of relevant features and examples in machine learning

Artificial Intelligence - Special issue on relevance
Introduction to Reinforcement Learning

Introduction to Reinforcement Learning
Learning with Genetic Algorithms: An Overview

Machine Learning
An introduction to variable and feature selection

The Journal of Machine Learning Research
Least-squares policy iteration

The Journal of Machine Learning Research
Computational-Mechanism Design: A Call to Arms

IEEE Intelligent Systems
An analysis of linear models, linear value-function approximation, and feature selection for reinforcement learning

Proceedings of the 25th international conference on Machine learning
Regularization and feature selection in least-squares temporal difference learning

ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
Adaptive Auction Mechanism Design and the Incorporation of Prior Knowledge

INFORMS Journal on Computing
Reinforcement Learning and Dynamic Programming Using Function Approximators

Reinforcement Learning and Dynamic Programming Using Function Approximators
An adaptive Q-learning algorithm developed for agent-based computational modeling of electricity market

IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews
Research Commentary---Designing Smart Markets

Information Systems Research
Market power and efficiency in a computational electricity marketwith discriminatory double-auction pricing

IEEE Transactions on Evolutionary Computation
Strategy learning for autonomous agents in smart grid markets

IJCAI'11 Proceedings of the Twenty-Second international joint conference on Artificial Intelligence - Volume Volume Two
Autonomous data-driven decision-making in smart electricity markets

ECML PKDD'12 Proceedings of the 2012 European conference on Machine Learning and Knowledge Discovery in Databases - Volume Part II
Policy iteration based on a learned transition model

ECML PKDD'12 Proceedings of the 2012 European conference on Machine Learning and Knowledge Discovery in Databases - Volume Part II
Real-Time Tactical and Strategic Sales Management for Intelligent Agents Guided by Economic Regimes

Information Systems Research

Quantified Score

Hi-index	0.00

Visualization

Abstract

The vision of a Smart Electric Grid relies critically on substantial advances in intelligent decentralized control mechanisms. We propose a novel class of autonomous broker agents for retail electricity trading that can operate in a wide range of Smart Electricity Markets, and that are capable of deriving long-term, profit-maximizing policies. Our brokers use Reinforcement Learning with function approximation, they can accommodate arbitrary economic signals from their environments, and they learn efficiently over the large state spaces resulting from these signals. We show how feature selection and regularization can be leveraged to automatically optimize brokers for particular market conditions, and demonstrate the performance of our design in extensive experiments using real-world energy market data.