ACE: an automatic complexity evaluator
ACM Transactions on Programming Languages and Systems (TOPLAS)
Determining average program execution times and their variance
PLDI '89 Proceedings of the ACM SIGPLAN 1989 Conference on Programming language design and implementation
FPCA '89 Proceedings of the fourth international conference on Functional programming languages and computer architecture
Optimally profiling and tracing programs
ACM Transactions on Programming Languages and Systems (TOPLAS)
High-level optimization via automated statistical modeling
PPOPP '95 Proceedings of the fifth ACM SIGPLAN symposium on Principles and practice of parallel programming
Proceedings of the seventh annual ACM symposium on Parallel algorithms and architectures
Communications of the ACM
Gprof: A call graph execution profiler
SIGPLAN '82 Proceedings of the 1982 SIGPLAN symposium on Compiler construction
An Information Exploration Tool for Performance Analysis of Java Programs
TOOLS '01 Proceedings of the Technology of Object-Oriented Languages and Systems
Predicting the Running Times of Parallel Programs by Simulation
IPPS '98 Proceedings of the 12th. International Parallel Processing Symposium on International Parallel Processing Symposium
Knowledge based automatic scalability analysis and extrapolation for MPI programs
Euro-Par'05 Proceedings of the 11th international Euro-Par conference on Parallel Processing
Banshee: a scalable constraint-based analysis toolkit
SAS'05 Proceedings of the 12th international conference on Static Analysis
A framework for measurement based performance modeling
WOSP '08 Proceedings of the 7th international workshop on Software and performance
Automatic Communication Performance Debugging in PGAS Languages
Languages and Compilers for Parallel Computing
SPEED: precise and efficient static estimation of program computational complexity
Proceedings of the 36th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages
Empirical hardness models: Methodology and a case study on combinatorial auctions
Journal of the ACM (JACM)
Control-flow refinement and progress invariants for bound analysis
Proceedings of the 2009 ACM SIGPLAN conference on Programming language design and implementation
The road not taken: Estimating path execution frequency statically
ICSE '09 Proceedings of the 31st International Conference on Software Engineering
WISE: Automated test generation for worst-case complexity
ICSE '09 Proceedings of the 31st International Conference on Software Engineering
The Complexity of Andersen's Analysis in Practice
SAS '09 Proceedings of the 16th International Symposium on Static Analysis
Finding low-utility data structures
PLDI '10 Proceedings of the 2010 ACM SIGPLAN conference on Programming language design and implementation
The reachability-bound problem
PLDI '10 Proceedings of the 2010 ACM SIGPLAN conference on Programming language design and implementation
Finding latent performance bugs in systems implementations
Proceedings of the eighteenth ACM SIGSOFT international symposium on Foundations of software engineering
Optimizing data partitioning for data-parallel computing
HotOS'13 Proceedings of the 13th USENIX conference on Hot topics in operating systems
Bound analysis of imperative programs with the size-change abstraction
SAS'11 Proceedings of the 18th international conference on Static analysis
Oracle scheduling: controlling granularity in implicitly parallel languages
Proceedings of the 2011 ACM international conference on Object oriented programming systems languages and applications
Automatic generation of load tests
ASE '11 Proceedings of the 2011 26th IEEE/ACM International Conference on Automated Software Engineering
Proceedings of the 33rd ACM SIGPLAN conference on Programming Language Design and Implementation
Proceedings of the 33rd ACM SIGPLAN conference on Programming Language Design and Implementation
Context-sensitive delta inference for identifying workload-dependent performance bottlenecks
Proceedings of the 2013 International Symposium on Software Testing and Analysis
Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering
Mantis: automatic performance prediction for smartphone applications
USENIX ATC'13 Proceedings of the 2013 USENIX conference on Annual Technical Conference
A runtime approach for estimating resource usage
Proceedings of the Fourth Symposium on Information and Communication Technology
Estimating the Empirical Cost Function of Routines with Dynamic Workloads
Proceedings of Annual IEEE/ACM International Symposium on Code Generation and Optimization
Predicting execution time of machine learning tasks for scheduling
International Journal of Hybrid Intelligent Systems
EuroVis '13 Proceedings of the 15th Eurographics Conference on Visualization
| Hi-index | 0.01 |
The standard language for describing the asymptotic behavior of algorithms is theoretical computational complexity. We propose a method for describing the asymptotic behavior of programs in practice by measuring their empirical computational complexity. Our method involves running a program on workloads spanning several orders of magnitude in size, measuring their performance, and fitting these observations to a model that predicts performance as a function of workload size. Comparing these models to the programmer's expectations or to theoretical asymptotic bounds can reveal performance bugs or confirm that a program's performance scales as expected. Grouping and ranking program locations based on these models focuses attention on scalability-critical code. We describe our tool, the Trend Profiler (trend-prof), for constructing models of empirical computational complexity that predict how many times each basic block in a program runs as a linear (y = a + bx) or a powerlaw (y = axb) function of user-specified features of the program's workloads. We ran trend-prof on several large programs and report cases where a program scaled as expected, beat its worst-case theoretical complexity bound, or had a performance bug.