Composing high-performance memory allocators
Proceedings of the ACM SIGPLAN 2001 conference on Programming language design and implementation
Dynamic Storage Allocation: A Survey and Critical Review
IWMM '95 Proceedings of the International Workshop on Memory Management
Variability in Architectural Simulations of Multi-Threaded Workloads
HPCA '03 Proceedings of the 9th International Symposium on High-Performance Computer Architecture
LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation
Proceedings of the international symposium on Code generation and optimization: feedback-directed and runtime optimization
TLSF: A New Dynamic Memory Allocator for Real-Time Systems
ECRTS '04 Proceedings of the 16th Euromicro Conference on Real-Time Systems
Code placement for improving dynamic branch prediction accuracy
Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation
DieHard: probabilistic memory safety for unsafe languages
Proceedings of the 2006 ACM SIGPLAN conference on Programming language design and implementation
Improving address space randomization with a dynamic offset randomization technique
Proceedings of the 2006 ACM symposium on Applied computing
Address Space Layout Permutation (ASLP): Towards Fine-Grained Randomization of Commodity Software
ACSAC '06 Proceedings of the 22nd Annual Computer Security Applications Conference
Using Machine Learning to Guide Architecture Simulation
The Journal of Machine Learning Research
Address obfuscation: an efficient approach to combat a board range of memory error exploits
SSYM'03 Proceedings of the 12th conference on USENIX Security Symposium - Volume 12
Efficient techniques for comprehensive protection from memory error exploits
SSYM'05 Proceedings of the 14th conference on USENIX Security Symposium - Volume 14
Statistically rigorous java performance evaluation
Proceedings of the 22nd annual ACM SIGPLAN conference on Object-oriented programming systems and applications
Exterminator: Automatically correcting memory errors with high probability
Communications of the ACM - Surviving the data deluge
Reducing Performance Evaluation Sensitivity and Variability by Input Shaking
MASCOTS '07 Proceedings of the 2007 15th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems
Producing wrong data without doing anything obviously wrong!
Proceedings of the 14th international conference on Architectural support for programming languages and operating systems
Proceedings of the 17th ACM conference on Computer and communications security
Workshop on experimental evaluation of software and systems in computer science (Evaluate 2010)
Proceedings of the ACM international conference companion on Object oriented programming systems languages and applications companion
Instability in parallel job scheduling simulation: the role of workload flurries
IPDPS'06 Proceedings of the 20th international conference on Parallel and distributed processing
SP 800-22 Rev. 1a. A Statistical Test Suite for Random and Pseudorandom Number Generators for Cryptographic Applications
A proper performance evaluation system that summarizes code placement effects
Proceedings of the 11th ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering
Rigorous benchmarking in reasonable time
Proceedings of the 2013 international symposium on memory management
Probabilistic timing analysis on conventional cache designs
Proceedings of the Conference on Design, Automation and Test in Europe
Towards software performance engineering for multicore and manycore systems
ACM SIGMETRICS Performance Evaluation Review
Hi-index | 0.00 |
Researchers and software developers require effective performance evaluation. Researchers must evaluate optimizations or measure overhead. Software developers use automatic performance regression tests to discover when changes improve or degrade performance. The standard methodology is to compare execution times before and after applying changes. Unfortunately, modern architectural features make this approach unsound. Statistically sound evaluation requires multiple samples to test whether one can or cannot (with high confidence) reject the null hypothesis that results are the same before and after. However, caches and branch predictors make performance dependent on machine-specific parameters and the exact layout of code, stack frames, and heap objects. A single binary constitutes just one sample from the space of program layouts, regardless of the number of runs. Since compiler optimizations and code changes also alter layout, it is currently impossible to distinguish the impact of an optimization from that of its layout effects. This paper presents Stabilizer, a system that enables the use of the powerful statistical techniques required for sound performance evaluation on modern architectures. Stabilizer forces executions to sample the space of memory configurations by repeatedly re-randomizing layouts of code, stack, and heap objects at runtime. Stabilizer thus makes it possible to control for layout effects. Re-randomization also ensures that layout effects follow a Gaussian distribution, enabling the use of statistical tests like ANOVA. We demonstrate Stabilizer's efficiency (