Scoring Rules and Decision Analysis Education

Authors:
J. Eric Bickel
Affiliations:
Operations Research and Industrial Engineering, The University of Texas at Austin, Austin, Texas 78712
Venue:
Decision Analysis
Year:
2010

Citing 4
Cited 7

Elements of information theory

Elements of information theory
Some Comparisons among Quadratic, Spherical, and Logarithmic Scoring Rules

Decision Analysis
Probability Elicitation, Scoring Rules, and Competition Among Forecasters

Management Science
Sensitivity to Distance and Baseline Distributions in Forecast Evaluation

Management Science

From the Editors...

Decision Analysis
From the Editors---Deterrence, Multiattribute Utility, and Probability and Bayes' Updating

Decision Analysis
From the Editor---Multiattribute and Intertemporal Preferences, Probability, and Stochastic Processes: Models and Assessment

Decision Analysis
From the Editors---Probability Scoring Rules, Ambiguity, Multiattribute Terrorist Utility, and Sensitivity Analysis

Decision Analysis
From the Editors---Copulas, Group Preferences, Multilevel Defenders, Sharing Rewards, and Communicating Analytics

Decision Analysis
From the Editors---Brainstorming, Multiplicative Utilities, Partial Information on Probabilities or Outcomes, and Regulatory Focus

Decision Analysis
From the Editors: Probability Approximations, Anti-Terrorism Strategy, and Bull's-Eye Display for Performance Feedback

Decision Analysis

Quantified Score

Hi-index	0.00

Visualization

Abstract

Experiential learning is perhaps the most effective way to teach. One example is the scoring procedure used for exams in some decision analysis programs. Under this grading scheme, students take a multiple-choice exam, but rather than simply marking which answer they think is correct, they must assign a probability to each possible answer. The exam is then scored with a special scoring rule, under which students' best strategy is to avoid guessing and instead assign their true beliefs. Such a scoring function is known as a strictly proper scoring rule. In this paper, we discuss several different scoring rules and demonstrate how their use in testing situations provides insights for both students and instructors.