Proprietary code to non-proprietary benchmarks: synthesis techniques for scalable benchmarks

• Authors: Lizy Kurian John
• Venue: Proceedings of the 2nd ACM/SPEC International Conference on Performance engineering
• Year: 2011

Real world applications constitute intellectual property and simultaneous design of hardware and software is made very difficult due to the need for disclosing proprietary software to hardware designers. Consider a smart phone for which applications are developed by various third parties or a military system, where the classified applications are developed in-house at the military while hardware is procured from standard vendors. Design of hardware that gives good performance and low power can be done if hardware designers had access to the software, so they can understand the features of the software and tune various hardware features to the software characteristics. While non-disclosure agreements and legal arrangements can be used to partly solve the problem, it will be much more convenient to have a mechanism to create proxies of proprietary benchmarks that have the performance (and power) characteristics of the source, but not the functionality. In our past research, we created a benchmark synthesis process for early design exploration. The benchmark synthesis process consists of constructing a proxy workload that possesses approximately the same performance and power characteristics as the original workload [1-3]. The synthesis comprises of two steps: (1) profiling the real-world proprietary workload to measure its inherent behavior characteristics, and (2) modeling the measured workload attributes into a synthetic benchmark program. The set of workload characteristics can be thought of as a signature that uniquely describes the workload's inherent behavior, independent of the microarchitecture. The cloned code in fact has no functionality and cannot be reverse engineered to create the original code or algorithms. The cloned software can be freely released to hardware developers so that they can optimize the hardware they deliver to their clients to yield improved performance. This talk will describe the benchmark synthesis process as well as the status of the research. The process of constructing synthetic proxies that approximately resemble the original proprietary application will be explained. Synthesis for multicore and multithreaded processors will be described.