Polynomial-Time verification of PCTL properties of MDPs with convex uncertainties

Authors:
Alberto Puggelli;Wenchao Li;Alberto L. Sangiovanni-Vincentelli;Sanjit A. Seshia
Affiliations:
Department of Electrical Engineering and Computer Sciences, University of California, Berkeley;Department of Electrical Engineering and Computer Sciences, University of California, Berkeley;Department of Electrical Engineering and Computer Sciences, University of California, Berkeley;Department of Electrical Engineering and Computer Sciences, University of California, Berkeley
Venue:
CAV'13 Proceedings of the 25th international conference on Computer Aided Verification
Year:
2013

Citing 18
Cited 0

Fast randomized consensus using shared memory

Journal of Algorithms
The complexity of probabilistic verification

Journal of the ACM (JACM)
Markov Decision Processes: Discrete Stochastic Dynamic Programming

Markov Decision Processes: Discrete Stochastic Dynamic Programming
Model Checking of Probabalistic and Nondeterministic Systems

Proceedings of the 15th Conference on Foundations of Software Technology and Theoretical Computer Science
Automated Verification of a Randomized Distributed Consensus Protocol Using Cadence SMV and PRISM

CAV '01 Proceedings of the 13th International Conference on Computer Aided Verification
Design and Synthesis of Synchronization Skeletons Using Branching-Time Temporal Logic

Logic of Programs, Workshop
Convex Optimization

Convex Optimization
Performance analysis of probabilistic timed automata using digital clocks

Formal Methods in System Design
Robust Control of Markov Decision Processes with Uncertain Transition Matrices

Operations Research
Quantitative verification: models techniques and tools

Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering
Automatic verification of probabilistic concurrent finite state programs

SFCS '85 Proceedings of the 26th Annual Symposium on Foundations of Computer Science
Probabilistic Model Checking of Biological Systems with Uncertain Kinetic Rates

RP '09 Proceedings of the 3rd International Workshop on Reachability Problems
Model-checking ω-regular properties of interval Markov chains

FOSSACS'08/ETAPS'08 Proceedings of the Theory and practice of software, 11th international conference on Foundations of software science and computational structures
Parameter identification for Markov models of biochemical reactions

CAV'11 Proceedings of the 23rd international conference on Computer aided verification
PRISM 4.0: verification of probabilistic real-time systems

CAV'11 Proceedings of the 23rd international conference on Computer aided verification
Model-Checking markov chains in the presence of uncertainties

TACAS'06 Proceedings of the 12th international conference on Tools and Algorithms for the Construction and Analysis of Systems
Robust PCTL model checking

Proceedings of the 15th ACM international conference on Hybrid Systems: Computation and Control
Robust solutions of uncertain linear programs

Operations Research Letters

Quantified Score

Hi-index	0.00

Visualization

Abstract

We address the problem of verifying Probabilistic Computation Tree Logic (PCTL) properties of Markov Decision Processes (MDPs) whose state transition probabilities are only known to lie within uncertainty sets. We first introduce the model of Convex-MDPs (CMDPs), i.e., MDPs with convex uncertainty sets. CMDPs generalize Interval-MDPs (IMDPs) by allowing also more expressive (convex) descriptions of uncertainty. Using results on strong duality for convex programs, we then present a PCTL verification algorithm for CMDPs, and prove that it runs in time polynomial in the size of a CMDP for a rich subclass of convex uncertainty models. This result allows us to lower the previously known algorithmic complexity upper bound for IMDPs from co-NP to PTIME. We demonstrate the practical effectiveness of the proposed approach by verifying a consensus protocol and a dynamic configuration protocol for IPv4 addresses.