Average Continuous Control of Piecewise Deterministic Markov Processes
SIAM Journal on Control and Optimization
IEEE/ACM Transactions on Networking (TON)
| Hi-index | 0.00 |
The main goal of this paper is to establish some equivalence results on stability, recurrence, and ergodicity between a piecewise deterministic Markov process (PDMP) $\{X(t)\}$ and an embedded discrete-time Markov chain $\{\Theta_{n}\}$ generated by a Markov kernel $G$ that can be explicitly characterized in terms of the three local characteristics of the PDMP, leading to tractable criterion results. First we establish some important results characterizing $\{\Theta_{n}\}$ as a sampling of the PDMP $\{X(t)\}$ and deriving a connection between the probability of the first return time to a set for the discrete-time Markov chains generated by $G$ and the resolvent kernel $R$ of the PDMP. From these results we obtain equivalence results regarding irreducibility, existence of $\sigma$-finite invariant measures, and (positive) recurrence and (positive) Harris recurrence between $\{X(t)\}$ and $\{\Theta_{n}\}$, generalizing the results of [F. Dufour and O. L. V. Costa, SIAM J. Control Optim., 37 (1999), pp. 1483-1502] in several directions. Sufficient conditions in terms of a modified Foster-Lyapunov criterion are also presented to ensure positive Harris recurrence and ergodicity of the PDMP. We illustrate the use of these conditions by showing the ergodicity of a capacity expansion model.