Eliciting additive reward functions for Markov decision processes

  • Authors:
  • Kevin Regan;Craig Boutilier

  • Affiliations:
  • Department of Computer Science, University of Toronto;Department of Computer Science, University of Toronto

  • Venue:
  • IJCAI'11 Proceedings of the Twenty-Second international joint conference on Artificial Intelligence - Volume Volume Three
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

Specifying the reward function of a Markov decision process (MDP) can be demanding, requiring human assessment of the precise quality of, and tradeoffs among, various states and actions. However, reward functions often possess considerable structure which can be leveraged to streamline their specification. We develop new, decision-theoretically sound heuristics for eliciting rewards for factored MDPs whose reward functions exhibit additive independence. Since we can often find good policies without complete reward specification, we also develop new (exact and approximate) algorithms for robust optimization of imprecise-reward MDPs with such additive reward. Our methods are evaluated in two domains: autonomic computing and assistive technology.