Second-order Markov reward models driven by QBD processes

Authors:
Nigel G. Bean;MałGorzata M. O'Reilly;Yong Ren
Affiliations:
Applied Mathematics, University of Adelaide, SA 5005, Australia;School of Mathematics, University of Tasmania, GPO Box 252C-37, Hobart, Tasmania 7001, Australia;School of Mathematics, University of Tasmania, GPO Box 252C-37, Hobart, Tasmania 7001, Australia
Venue:
Performance Evaluation
Year:
2012

Citing 8
Cited 0

Evaluation of Performability for Degradable Computer Systems

IEEE Transactions on Computers
Performability Analysis Using Semi-Markov Reward Processes

IEEE Transactions on Computers
Reward model solution methods with impulse and rate rewards: an algorithm and numerical results

Performance Evaluation
Performance analysis of Markov regenerative reward models

Performance Evaluation
The Markov modulated regulated Brownian motion: A second-order fluid flow model of a finite buffer

Queueing Systems: Theory and Applications
Analysis of Second-Order Markov Reward Models

DSN '04 Proceedings of the 2004 International Conference on Dependable Systems and Networks
Constructive Computation in Stochastic Models with Applications: The RG-Factorizations

Constructive Computation in Stochastic Models with Applications: The RG-Factorizations
On modelling the performance and reliability of multimode computer systems

Journal of Systems and Software

Quantified Score

Hi-index	0.00

Visualization

Abstract

Second-order reward models are an important class of models for evaluating the performance of real-life systems in which the reward measure fluctuates according to some underlying noise. These models consist of a Markov chain driving the evolution of the system, and a continuous reward variable representing its performance. Thus far, only models with a finite number of states have been studied. We consider second-order reward models driven by Quasi-birth-and-death processes, a class of block-structured Markov chains with infinitely many states. We derive the expressions for the Laplace-Stieltjes transforms of the accumulated reward and demonstrate how they can be efficiently evaluated. We use our results to analyse a simple example and, in doing so, show that the second-order feature can make a significant difference to the accumulated reward. The inclusion of the second-order feature also creates new difficulties which require the development of new conditions in the analysis.