Computing the Stationary Distribution of a Finite Markov Chain Through Stochastic Factorization

• Authors: André M. S. Barreto;Marcelo D. Fragoso
• Venue: SIAM Journal on Matrix Analysis and Applications
• Year: 2011

This work presents an approach for reducing the number of arithmetic operations involved in the computation of a stationary distribution for a finite Markov chain. The proposed method relies on a particular decomposition of a transition-probability matrix called stochastic factorization. The idea is simple: when a transition matrix is represented as the product of two stochastic matrices, one can swap the factors of the multiplication to obtain another transition matrix, potentially much smaller than the original. We show in the paper that the stationary distributions of both Markov chains are related through a linear transformation, which opens up the possibility of using the smaller chain to compute the stationary distribution of the original model. In order to support the application of stochastic factorization, we prove that the model derived from it retains all the properties of the original chain which are relevant to the stationary distribution computation. Specifically, we show that (i) for each recurrent class in the original Markov chain there is a corresponding class in the derived model with the same period and, given some simple assumptions about the factorization, (ii) the original chain is irreducible if and only if the derived chain is irreducible and (iii) the original chain is regular if and only if the derived chain is regular. The paper also addresses some issues associated with the application of the proposed approach in practice and briefly discusses how stochastic factorization can be used to reduce the number of operations needed to compute the fundamental matrix of an absorbing Markov chain.