An Improved Batch Means Procedure for Simulation Output Analysis
Management Science
Output analysis: on choosing a single criterion for confidence-interval procedures
Proceedings of the 34th conference on Winter simulation: exploring new frontiers
Simulation Modeling and Analysis (McGraw-Hill Series in Industrial Engineering and Management)
Simulation Modeling and Analysis (McGraw-Hill Series in Industrial Engineering and Management)
Graphical methods for investigating the finite-sample properties of confidence regions
Computational Statistics & Data Analysis
Analysis of sequential stopping rules
Winter Simulation Conference
| Hi-index | 0.00 |
Absolute precision stopping rules are often used to determine the length of sequential experiments to estimate confidence intervals for simulated performance measures. Much is known about the asymptotic behavior of such procedures. In this paper, we introduce coverage contours to quantify the trade-offs in interval coverage, stopping times, and precision for finite-sample experiments using absolute precision rules. We use these contours to evaluate the coverage of a basic absolute precision stopping rule, and we show that this rule will lead to a bias in coverage even if all of the assumptions supporting the procedure are true. We define optimal stopping rules that deliver nominal coverage with the smallest expected number of observations. Contrary to previous asymptotic results that suggest decreasing the precision of the rule to approach nominal coverage in the limit, we find that it is optimal to increase the confidence coefficient used in the stopping rule, thus obtaining nominal coverage in a finite-sample experiment. If the simulation data are independent and identically normally distributed, we can calculate coverage contours analytically and find a stopping rule that is insensitive to the variance of the data while delivering at least nominal coverage for any precision value.