How to assess the acceptability and credibility of simulation results
WSC '89 Proceedings of the 21st conference on Winter simulation
Validation and verification of simulation models
WSC '04 Proceedings of the 36th conference on Winter simulation
Two new subjective validation methods using data displays
WSC '05 Proceedings of the 37th conference on Winter simulation
A transmission line fault locator based on Elman recurrent networks
Applied Soft Computing
Evaluating Sampling Based Hotspot Detection
ARCS '09 Proceedings of the 22nd International Conference on Architecture of Computing Systems
Structural and Multidisciplinary Optimization
| Hi-index | 0.00 |
Metamodels are used to provide simpler prediction means than the complex simulation models they approximate. Accuracy of a metamodel is one fundamental criterion that is used as the basis for accepting or rejecting a metamodel. Average-based metrics such as root-mean-square error RMSE and R-square are often used. Like all other average-based statistics, these measures are sensitive to sample sizes unless the number of test points in these samples is adequate. We introduce in this paper a new metric that can be used to measure metamodels fit quality, called metamodel acceptability score MAS. The proposed metric gives readily interpretable meaning to metamodels acceptability. Furthermore, initial studies show that MAS is less sensitive to test sample sizes compared to average-based validation measures.