Practical statistical analyses of simulation output data

  • Authors:
  • Averill M. Law

  • Affiliations:
  • -

  • Venue:
  • WSC '84 Proceedings of the 16th conference on Winter simulation
  • Year:
  • 1984

Quantified Score

Hi-index 0.00

Visualization

Abstract

It has been our observation that in many simulation studies a large amount of time and money is spent on model development and programming, but little effort is made to analyze the simulation output data in an appropriate manner. As a matter of fact, a common mode of operation is to make a single simulation run of somewhat arbitrary length and then treat the resulting simulation estimates as being the “true” answers for the model. Since these estimates are observations of random variables which may have large variances, these estimates may, in a particular simulation run, differ greatly from the corresponding true answers. The net effect is, of course, that there may be a significant probability of making erroneous inferences about the system under study. One reason for the historical lack of definitive output data analyses is that simulation output data are rarely, if ever, independent. Thus, classical statistical analyses based on independent identically distributed observations are not directly applicable. At the present time, there are still several output analysis problems for which there is no completely accepted solution, and the solutions that do exist are often complicated to apply. Another impediment to getting accurate estimates of a model's true parameters or characteristics is the computer cost associated with collecting the necessary amount of simulation output data. Indeed, there are situations where an appropriate statistical procedure is available, but the cost of collecting the amount of data dictated by the procedure is prohibitive. We expect this latter problem to become less important as the cost of computer time continues to drop.