Decision Analysis
A Kullback-Leibler View of Linear and Log-Linear Pools
Decision Analysis
Decision Analysis
Decision Analysis
Decision Analysis
Scoring Rules and Decision Analysis Education
Decision Analysis
Tailored Scoring Rules for Probabilities
Decision Analysis
Task routing for prediction tasks
Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems - Volume 2
Can theories be tested?: a cryptographic treatment of forecast testing
Proceedings of the 4th conference on Innovations in Theoretical Computer Science
Approaching utopia: strong truthfulness and externality-resistant mechanisms
Proceedings of the 4th conference on Innovations in Theoretical Computer Science
The construction of causal networks to estimate coral bleaching intensity
Environmental Modelling & Software
Eliciting high quality feedback from crowdsourced tree networks using continuous scoring rules
Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems
Decreasing Marginal Value of Information Under Symmetric Loss
Decision Analysis
Hi-index | 0.02 |
Strictly proper scoring rules continue to play an important role in probability assessment. Although many such rules have been developed, relatively little guidance exists as to which rule is the most appropriate. In this paper, we discuss two important properties of quadratic, spherical, and logarithmic scoring rules. From an ex post perspective, we compare their rank order properties and conclude that both quadratic and spherical scoring perform poorly in this regard, relative to logarithmic. Second, from an ex ante perspective, we demonstrate that in many situations, logarithmic scoring is the method least affected by a nonlinear utility function. These results suggest that logarithmic scoring is superior when rank order results are important and/or when the assessor has a nonlinear utility function. In addition to these results, and perhaps more important, we demonstrate that nonlinear utility induces relatively little deviation from the optimal assessment under an assumption of risk neutrality. These results provide both comfort and guidance to those who would like to use scoring rules as part of the assessment process.