Amortized efficiency of list update and paging rules
Communications of the ACM
Amortized analyses of self-organizing sequential search heuristics
Communications of the ACM - Lecture notes in computer science Vol. 174
A guide to simulation (2nd ed.)
A guide to simulation (2nd ed.)
More programming pearls: confessions of a coder
More programming pearls: confessions of a coder
Programming pearls
Steady-state simulation of queueing processes: survey of problems and solutions
ACM Computing Surveys (CSUR)
Bounded space on-line bin packing: best is better than first
SODA '91 Proceedings of the second annual ACM-SIAM symposium on Discrete algorithms
The art of computer programming, volume 1 (3rd ed.): fundamental algorithms
The art of computer programming, volume 1 (3rd ed.): fundamental algorithms
The art of computer programming, volume 2 (3rd ed.): seminumerical algorithms
The art of computer programming, volume 2 (3rd ed.): seminumerical algorithms
The art of computer programming, volume 3: (2nd ed.) sorting and searching
The art of computer programming, volume 3: (2nd ed.) sorting and searching
WSC '86 Proceedings of the 18th conference on Winter simulation
Experiments on traveling salesman heuristics
SODA '90 Proceedings of the first annual ACM-SIAM symposium on Discrete algorithms
An empirical study of insertion and deletion in binary search trees
Communications of the ACM
Simulations of dynamic sequential search algorithms
Communications of the ACM
On self-organizing sequential search heuristics
Communications of the ACM
Simulation Modeling and Analysis
Simulation Modeling and Analysis
Computing minimum spanning trees efficiently
ACM '72 Proceedings of the ACM annual conference - Volume 1
Computers and Intractability: A Guide to the Theory of NP-Completeness
Computers and Intractability: A Guide to the Theory of NP-Completeness
Some unexpected expected behavior results for bin packing
STOC '84 Proceedings of the sixteenth annual ACM symposium on Theory of computing
Deletion in binary storage trees.
Deletion in binary storage trees.
Heuristics that dynamically alter data structures to reduce their access time.
Heuristics that dynamically alter data structures to reduce their access time.
Experimental analysis of algorithms
Experimental analysis of algorithms
Partition testing, stratified sampling, and cluster analysis
SIGSOFT '93 Proceedings of the 1st ACM SIGSOFT symposium on Foundations of software engineering
Average-case analysis of off-line and on-line knapsack problems
Proceedings of the sixth annual ACM-SIAM symposium on Discrete algorithms
Performance study of phylogenetic methods: (unweighted) quartet methods and neighbor-joining
SODA '01 Proceedings of the twelfth annual ACM-SIAM symposium on Discrete algorithms
Dijkstra's algorithm on-line: an empirical case study from public railroad transport
Journal of Experimental Algorithmics (JEA)
The Performance of Phylogenetic Methods on Trees of Bounded Diameter
WABI '01 Proceedings of the First International Workshop on Algorithms in Bioinformatics
Estimating the Deviation from a Molecular Clock
WABI '02 Proceedings of the Second International Workshop on Algorithms in Bioinformatics
On the Differences between ``Practical'' and ``Applied''
WAE '00 Proceedings of the 4th International Workshop on Algorithm Engineering
Using finite experiments to study asymptotic performance
Experimental algorithmics
Reconstructing optimal phylogenetic trees: a challenge in experimental algorithmics
Experimental algorithmics
APPLYING VARIANCE REDUCTION IDEAS IN QUEUING SIMULATIONS
Probability in the Engineering and Informational Sciences
Performance study of phylogenetic methods: (unweighted) quartet methods and neighbor-joining
Journal of Algorithms - Special issue: Twelfth annual ACM-SIAM symposium on discrete algorithms
ACM Transactions on Internet Technology (TOIT)
Proceedings of the 40th Conference on Winter Simulation
Proceedings of the 40th Conference on Winter Simulation
Data mining for simulation algorithm selection
Proceedings of the 2nd International Conference on Simulation Tools and Techniques
An Efficient and Adaptive Mechanism for Parallel Simulation Replication
PADS '09 Proceedings of the 2009 ACM/IEEE/SCS 23rd Workshop on Principles of Advanced and Distributed Simulation
Algorithm engineering: bridging the gap between algorithm theory and practice
Algorithm engineering: bridging the gap between algorithm theory and practice
Automating the runtime performance evaluation of simulation algorithms
Winter Simulation Conference
Hi-index | 0.00 |
Although experimental studies have been widely applied to the investigation of algorithm performance, very little attention has been given to experimental method in this area. This is unfortunate, since much can be done to improve the quality of the data obtained; often, much improvement may be needed for the data to be useful. This paper gives a tutorial discussion of two aspects of good experimental technique: the use of variance reduction techniques and simulation speedups in algorithm studies.In an illustrative study, application of variance reduction techniques produces a decrease in variance by a factor 1000 in one case, giving a dramatic improvement in the precision of experimental results. Furthermore, the complexity of the simulation program is improved from &THgr;mn/Hn) to &THgr;(m + n log n) (where m is typically much larger than n), giving a much faster simulation program and therefore more data per unit of computation time. The general application of variance reduction techniques is also discussed for a variety of algorithm problem domains.