Opening the `black box' of simulations: increased transparency and effective communication through the systematic design of experiments

Authors:
Iris Lorscheid;Bernd-Oliver Heine;Matthias Meyer
Affiliations:
Institute of Management Control and Accounting, Hamburg University of Technology, Hamburg, Germany;Institute of Management Control and Accounting, Hamburg University of Technology, Hamburg, Germany;Institute of Management Control and Accounting, Hamburg University of Technology, Hamburg, Germany
Venue:
Computational & Mathematical Organization Theory
Year:
2012

Citing 14
Cited 3

Experimental design and analysis: an overview of newer, advanced screening methods for the initial phase in an experimental design

Proceedings of the 33nd conference on Winter simulation
Relative Performance of Incentive Mechanisms: Computational Modeling and Simulation of Delegated Investment Decisions

Management Science
Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models

Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models
Experimental design for simulation: experimental design for simulation

Proceedings of the 35th conference on Winter simulation: driving innovation
Simulation for the Social Scientist

Simulation for the Social Scientist
Designing simulation experiments

WSC '04 Proceedings of the 36th conference on Winter simulation
Design and Analysis of Experiments

Design and Analysis of Experiments
Managing Business Complexity: Discovering Strategic Solutions with Agent-Based Modeling and Simulation

Managing Business Complexity: Discovering Strategic Solutions with Agent-Based Modeling and Simulation
Simulation Modeling and Analysis (McGraw-Hill Series in Industrial Engineering and Management)

Simulation Modeling and Analysis (McGraw-Hill Series in Industrial Engineering and Management)
Work smarter, not harder: guidelines for designing simulation experiments

Proceedings of the 38th conference on Winter simulation
State-of-the-Art Review: A User's Guide to the Brave New World of Designing Simulation Experiments

INFORMS Journal on Computing
Design and Analysis of Simulation Experiments

Design and Analysis of Simulation Experiments
Efficient experimental design tools for exploring large simulation models

Computational & Mathematical Organization Theory
A Plea for (Good) Simulations: Nudging Economics Toward an Experimental Science

Simulation and Gaming

Epistemological perspectives on simulation: overview and introduction

Computational & Mathematical Organization Theory
Effectively combining experimental economics and multi-agent simulation: suggestions for a procedural integration with an example from prediction markets research

Computational & Mathematical Organization Theory
Metamodels for role-driven agent-based modelling

Computational & Mathematical Organization Theory

Quantified Score

Hi-index	0.00

Visualization

Abstract

Many still view simulation models as a black box. This paper argues that perceptions could change if the systematic design of experiments (DOE) for simulation research was fully realized. DOE can increase (1) the transparency of simulation model behavior and (2) the effectiveness of reporting simulation results. Based on DOE principles, we develop a systematic procedure to guide the analysis of simulation models as well as concrete templates for sharing the results. A simulation model investigating the performance of learning algorithms in an economic mechanism design context illustrates our suggestions. Overall, the proposed systematic procedure for applying DOE principles complements current initiatives for a more standardized simulation research process.