Self-Optimizing Memory Controllers: A Reinforcement Learning Approach

  • Authors:
  • Engin Ipek;Onur Mutlu;José F. Martínez;Rich Caruana

  • Affiliations:
  • -;-;-;-

  • Venue:
  • ISCA '08 Proceedings of the 35th Annual International Symposium on Computer Architecture
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

Efficiently utilizing off-chip DRAM bandwidth is a critical issuein designing cost-effective, high-performance chip multiprocessors(CMPs). Conventional memory controllers deliver relativelylow performance in part because they often employ fixed,rigid access scheduling policies designed for average-case applicationbehavior. As a result, they cannot learn and optimizethe long-term performance impact of their scheduling decisions,and cannot adapt their scheduling policies to dynamic workloadbehavior.We propose a new, self-optimizing memory controller designthat operates using the principles of reinforcement learning (RL)to overcome these limitations. Our RL-based memory controllerobserves the system state and estimates the long-term performanceimpact of each action it can take. In this way, the controllerlearns to optimize its scheduling policy on the fly to maximizelong-term performance. Our results show that an RL-basedmemory controller improves the performance of a set of parallelapplications run on a 4-core CMP by 19% on average (upto 33%), and it improves DRAM bandwidth utilization by 22%compared to a state-of-the-art controller.