A method for evaluating elicitation schemes for probabilisticmodels

  • Authors:
  • H. Wang;D. Dash;M. J. Druzdzel

  • Affiliations:
  • Decision Syst. Lab., Pittsburgh Univ., PA;-;-

  • Venue:
  • IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
  • Year:
  • 2002

Quantified Score

Hi-index 0.00

Visualization

Abstract

We present an objective approach for evaluating probability and structure elicitation methods in probabilistic models. The main idea is to use the model derived from the experts' experience rather than the true model as the standard to compare the elicited model. We describe a general procedure by which it is possible to capture the data corresponding to the expert's beliefs, and we present a simple experiment in which we utilize this technique to compare three methods for eliciting discrete probabilities: 1) direct numerical assessment, 2) the probability wheel, and 3) the scaled probability bar. We show that for our domain, the scaled probability bar is the most effective tool for probability elicitation