A scalability benchmark suite for Erlang/OTP

Authors:
Stavros Aronis;Nikolaos Papaspyrou;Katerina Roukounaki;Konstantinos Sagonas;Yiannis Tsiouris;Ioannis E. Venetis
Affiliations:
Uppsala University, Uppsala, Sweden;National Technical University of Athens, Athens, Greece;National Technical University of Athens, Athens, Greece;Uppsala University, Uppsala, Sweden & National Technical University of Athens, Athens, Greece;National Technical University of Athens, Athens, Greece;National Technical University of Athens, Athens, Greece
Venue:
Proceedings of the eleventh ACM SIGPLAN workshop on Erlang workshop
Year:
2012

Citing 6
Cited 5

Characteristics of scalability and their impact on performance

Proceedings of the 2nd international workshop on Software and performance
The nofib Benchmark Suite of Haskell Programs

Proceedings of the 1992 Glasgow Workshop on Functional Programming
The DaCapo benchmarks: java benchmarking development and analysis

Proceedings of the 21st annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications
Scalaris: reliable transactional p2p key/value store

Proceedings of the 7th ACM SIGPLAN workshop on ERLANG
Benchmarking modern multiprocessors

Benchmarking modern multiprocessors
A structured overlay for multi-dimensional range queries

Euro-Par'07 Proceedings of the 13th international Euro-Par conference on Parallel Processing

Using many-core coprocessor to boost up Erlang VM

Proceedings of the twelfth ACM SIGPLAN workshop on Erlang
On the scalability of the Erlang term storage

Proceedings of the twelfth ACM SIGPLAN workshop on Erlang
Multicore profiling for Erlang programs using percept2

Proceedings of the twelfth ACM SIGPLAN workshop on Erlang
Actor scheduling for multicore hierarchical memory platforms

Proceedings of the twelfth ACM SIGPLAN workshop on Erlang
Improving the performance of actor model runtime environments on multicore and manycore platforms

Proceedings of the 2013 workshop on Programming based on actors, agents, and decentralized control

Quantified Score

Hi-index	0.00

Visualization

Abstract

Programming language implementers rely heavily on benchmarking for measuring and understanding performance of algorithms, architectural designs, and trade-offs between alternative implementations of compilers, runtime systems, and virtual machine components. Given this fact, it seems a bit ironic that it is often more difficult to come up with a good benchmark suite than a good implementation of a programming language. This paper presents the main aspects of the design and the current status of bencherl, a publicly available scalability benchmark suite for applications written in Erlang. In contrast to other benchmark suites, which are usually designed to report a particular performance point, our benchmark suite aims to assess scalability, i.e., help developers to study a set of performance points that show how an application's performance changes when additional resources (e.g., CPU cores, schedulers, etc.) are added. We describe the scalability dimensions that the suite aims to examine and present its infrastructure and current set of benchmarks. We also report some limited set of performance results in order to show the capabilities of our suite.