Markov chains with discontinuous drifts have differential inclusion limits

  • Authors:
  • Nicolas Gast;Bruno Gaujal

  • Affiliations:
  • Université Joseph Fourier, 38041 Grenoble, France and EPFL, IC-LCA2, BC 203 Bítiment BC, Station 14, 1015 Lausanne-EPFL, Switzerland;INRIA Grenoble - Rhône-Alpes, 655 avenue de l'Europe, 38 334 Saint Ismier Cedex, France

  • Venue:
  • Performance Evaluation
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

In this paper, we study deterministic limits of Markov processes having discontinuous drifts. While most results assume that the limiting dynamics is continuous, we show that these conditions are not necessary to prove convergence to a deterministic system. More precisely, we show that under mild assumptions, the stochastic system is a stochastic approximation algorithm with constant step size that converges to a differential inclusion. This differential inclusion is obtained by convexifying the rescaled drift of the Markov chain. This generic convergence result is used to compute stability conditions of stochastic systems, via their fluid limits. It is also used to analyze systems where discontinuous dynamics arise naturally, such as queuing systems with boundary conditions or with threshold control policies, via mean field approximations.