Can theories be tested?: a cryptographic treatment of forecast testing

Authors:
Kai-Min Chung;Edward Lui;Rafael Pass
Affiliations:
Cornell University, Ithaca, NY, USA;Cornell University, Ithaca, NY, USA;Cornell University, Ithaca, NY, USA
Venue:
Proceedings of the 4th conference on Innovations in Theoretical Computer Science
Year:
2013

Citing 8
Cited 0

Arthur-Merlin games: a randomized proof system, and a hierarchy of complexity class

Journal of Computer and System Sciences - 17th Annual ACM Symposium in the Theory of Computing, May 6-8, 1985
The knowledge complexity of interactive proof systems

SIAM Journal on Computing
Hardness vs randomness

Journal of Computer and System Sciences
P = BPP if E requires exponential circuits: derandomizing the XOR lemma

STOC '97 Proceedings of the twenty-ninth annual ACM symposium on Theory of computing
Zero Knowledge Proofs of Knowledge in Two Rounds

CRYPTO '89 Proceedings of the 9th Annual International Cryptology Conference on Advances in Cryptology
On Defining Proofs of Knowledge

CRYPTO '92 Proceedings of the 12th Annual International Cryptology Conference on Advances in Cryptology
Some Comparisons among Quadratic, Spherical, and Logarithmic Scoring Rules

Decision Analysis
Strategic Manipulation of Empirical Tests

Mathematics of Operations Research

Quantified Score

Hi-index	0.00

Visualization

Abstract

How do we test if a weather forecaster actually knows something about whether it will rain or not? Intuitively, a "good" forecast test should be complete---namely, a forecaster knowing the distribution of Nature should be able to pass the test with high probability, and sound---an uninformed forecaster should only be able to pass the test with small probability. We provide a comprehensive cryptographic study of the feasibility of complete and sound forecast testing, introducing various notions of both completeness and soundness, inspired by the literature on interactive proofs. Our main technical result is an incompleteness theorem for our most basic notion of computationally sound and complete forecast testing: If Nature is implemented by a polynomial-time algorithm, then every complete polynomial-time test can be passed by a completely uninformed polynomial-time forecaster (i.e., a computationally-bounded "charlatan") with high probability. We additionally study alternative notions of soundness and completeness and present both positive and negative results for these notions.