On the Numerical Analysis of Inhomogeneous Continuous-Time Markov Chains

Authors:
M. Arns;P. Buchholz;A. Panchenko
Affiliations:
Informatik IV, Technische Universität Dortmund, D-44221 Dortmund, Germany;Informatik IV, Technische Universität Dortmund, D-44221 Dortmund, Germany;Informatik IV, Technische Universität Dortmund, D-44221 Dortmund, Germany
Venue:
INFORMS Journal on Computing
Year:
2010

Citing 10
Cited 2

Uniformization and hybrid simulation/analytic models of renewal processes

Operations Research
Computing Poisson probabilities

Communications of the ACM
Numerical transient analysis of Markov models

Computers and Operations Research
Nonparametric estimation of the Cumulative intensity function for a nonhomogeneous Poisson process

Management Science
Computing bounds on steady state availability of repairable computer systems

Journal of the ACM (JACM)
Probability, stochastic processes, and queueing theory: the mathematics of computer performance modeling

Probability, stochastic processes, and queueing theory: the mathematics of computer performance modeling
Simulation Modeling and Analysis

Simulation Modeling and Analysis
Numerical Solution of Non-Homogeneous Markov Processes through Uniformization

Proceedings of the 12th European Simulation Multiconference on Simulation - Past, Present and Future
Continuous System Simulation

Continuous System Simulation
Computing Battery Lifetime Distributions

DSN '07 Proceedings of the 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks

Numerical analysis of continuous time Markov decision processes over finite horizons

Computers and Operations Research
Product Form Approximation of Transient Probabilities in Stochastic Reaction Networks

Electronic Notes in Theoretical Computer Science (ENTCS)

Quantified Score

Hi-index	0.00

Visualization

Abstract

Inhomogeneous continuous-time Markov chains play an important role in different application areas. In contrast to homogeneous continuous-time Markov chains, where a large number of numerical analysis techniques are available and have been compared, few results about the performance of numerical techniques in the inhomogeneous case are known. This paper presents a new variant of the uniformization technique, the most efficient approach for homogeneous Markov chains. The new uniformization technique allows for the stable computation of strict bounds for the transient distribution of inhomogeneous continuous-time Markov chains, which is not possible with other numerical techniques that provide only an approximation of the distribution and asymptotic bounds. Furthermore, another variant of uniformization is presented that computes an approximation of the transient distribution and is shown to outperform standard differential equation solvers if transition rates change slowly.