The new S language: a programming environment for data analysis and graphics
The new S language: a programming environment for data analysis and graphics
Empirical study of the expected running time for shellsort
The Computer Journal - Special issue on term rewriting
Introduction to parallel algorithms and architectures: array, trees, hypercubes
Introduction to parallel algorithms and architectures: array, trees, hypercubes
Analyzing algorithms by simulation: variance reduction techniques and simulation speedups
ACM Computing Surveys (CSUR)
Using scientific experiments in early computer science laboratories
SIGCSE '92 Proceedings of the twenty-third SIGCSE technical symposium on Computer science education
Introduction to parallel computing: design and analysis of algorithms
Introduction to parallel computing: design and analysis of algorithms
Academic careers for experimental computer scientists and engineers
Communications of the ACM
Science and Substance: A Challenge to Software Engineers
IEEE Software
Balanced allocations (extended abstract)
STOC '94 Proceedings of the twenty-sixth annual ACM symposium on Theory of computing
Empirical methods for artificial intelligence
Empirical methods for artificial intelligence
Experimental evaluation in computer science: a quantitative study
Journal of Systems and Software
Mersenne twister: a 623-dimensionally equidistributed uniform pseudo-random number generator
ACM Transactions on Modeling and Computer Simulation (TOMACS) - Special issue on uniform random number generation
The art of computer programming, volume 3: (2nd ed.) sorting and searching
The art of computer programming, volume 3: (2nd ed.) sorting and searching
Balanced allocations: the heavily loaded case
STOC '00 Proceedings of the thirty-second annual ACM symposium on Theory of computing
How to present a paper on experimental work with algorithms
ACM SIGACT News
Fast concurrent access to parallel disks
SODA '00 Proceedings of the eleventh annual ACM-SIAM symposium on Discrete algorithms
ACM president's letter: performance analysis: experimental computer science as its best
Communications of the ACM
ACM President's Letter: What is experimental computer science?
Communications of the ACM
A high-speed sorting procedure
Communications of the ACM
Analysis of Shellsort and Related Algorithms
ESA '96 Proceedings of the Fourth Annual European Symposium on Algorithms
Towards Optimal Locality in Mesh-Indexings
FCT '97 Proceedings of the 11th International Symposium on Fundamentals of Computation Theory
How Asymmetry Helps Load Balancing
FOCS '99 Proceedings of the 40th Annual Symposium on Foundations of Computer Science
Some unexpected expected behavior results for bin packing
STOC '84 Proceedings of the sixteenth annual ACM symposium on Theory of computing
Domain-independent scientific function finding
Domain-independent scientific function finding
Randomized Shellsort: a simple oblivious sorting algorithm
SODA '10 Proceedings of the twenty-first annual ACM-SIAM symposium on Discrete Algorithms
Algorithm engineering: bridging the gap between algorithm theory and practice
Algorithm engineering: bridging the gap between algorithm theory and practice
Randomized Shellsort: A Simple Data-Oblivious Sorting Algorithm
Journal of the ACM (JACM)
Proceedings of the 33rd ACM SIGPLAN conference on Programming Language Design and Implementation
Proceedings of the 33rd ACM SIGPLAN conference on Programming Language Design and Implementation
Estimating the Empirical Cost Function of Routines with Dynamic Workloads
Proceedings of Annual IEEE/ACM International Symposium on Code Generation and Optimization
Hi-index | 0.00 |
In the analysis of algorithms we are interested in obtaining closed form expressions for algorithmic complexity, or at least asymptotic expressions in O(ċ)-notation. It is often possible to use experimental results to make significant progress towards this goal, although there are fundamental reasons why we cannot guarantee to obtain such expressions from experiments alone. This paper investigates two approaches relating to problems of developing theoretical analyses based on experimental data.We first consider the scientific method, which views experimentation as part of a cycle alternating with theoretical analysis. This approach has been very successful in the natural sciences. Besides supplying preliminary ideas for theoretical analysis, experiments can test falsifiable hypotheses obtained by incomplete theoretical analysis. Asymptotic behavior can also sometimes be deduced from stronger hypotheses which have been induced from experiments. As long as complete mathematical analyses remains elusive, well tested hypotheses may have to take their place. Several examples are given where average complexity can be tested experimentally so that support for hypotheses is quite strong.A second question is how to approach systematically the problem of inferring asymptotic bounds from experimental data. Five heuristic rules for "empirical curve bounding" are presented, ogether with analytical results guaranteeing correctness for certain families of functions. Experimental evaluations of the correctness and tightness of bounds obtained by the rules for several constructed functions and real datasets are described.