Informing sequential clinical decision-making through reinforcement learning: an empirical study

  • Authors:

  • Susan M. Shortreed;Eric Laber;Daniel J. Lizotte;T. Scott Stroup;Joelle Pineau;Susan A. Murphy

  • Affiliations:

  • School of Computer Science, McGill University, Montreal, Canada H3A 2T5;Department of Statistics, University of Michigan, Ann Arbor, USA 48109;Department of Statistics, University of Michigan, Ann Arbor, USA 48109;NYS Psychiatric Institute, New York, USA 10032;School of Computer Science, McGill University, Montreal, Canada H3A 2T5;Department of Statistics, University of Michigan, Ann Arbor, USA 48109

  • Venue:
  • Machine Learning
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper highlights the role that reinforcement learning can play in the optimization of treatment policies for chronic illnesses. Before applying any off-the-shelf reinforcement learning methods in this setting, we must first tackle a number of challenges. We outline some of these challenges and present methods for overcoming them. First, we describe a multiple imputation approach to overcome the problem of missing data. Second, we discuss the use of function approximation in the context of a highly variable observation set. Finally, we discuss approaches to summarizing the evidence in the data for recommending a particular action and quantifying the uncertainty around the Q-function of the recommended policy. We present the results of applying these methods to real clinical trial data of patients with schizophrenia.