Quickly generating billion-record synthetic databases
SIGMOD '94 Proceedings of the 1994 ACM SIGMOD international conference on Management of data
Benchmark Handbook: For Database and Transaction Processing Systems
Benchmark Handbook: For Database and Transaction Processing Systems
The Case for Application-Specific Benchmarking
HOTOS '99 Proceedings of the The Seventh Workshop on Hot Topics in Operating Systems
An integrated experimental environment for distributed systems and networks
OSDI '02 Proceedings of the 5th symposium on Operating systems design and implementationCopyright restrictions prevent ACM from being able to make the PDFs for this conference available for downloading
C-store: a column-oriented DBMS
VLDB '05 Proceedings of the 31st international conference on Very large data bases
Bigtable: a distributed storage system for structured data
OSDI '06 Proceedings of the 7th USENIX Symposium on Operating Systems Design and Implementation - Volume 7
XMark: a benchmark for XML data management
VLDB '02 Proceedings of the 28th international conference on Very Large Data Bases
Dynamo: amazon's highly available key-value store
Proceedings of twenty-first ACM SIGOPS symposium on Operating systems principles
Linear road: a stream data management benchmark
VLDB '04 Proceedings of the Thirtieth international conference on Very large data bases - Volume 30
Cutting corners: workbench automation for server benchmarking
ATC'08 USENIX 2008 Annual Technical Conference on Annual Technical Conference
Rose: compressed, log-structured replication
Proceedings of the VLDB Endowment
PNUTS: Yahoo!'s hosted data serving platform
Proceedings of the VLDB Endowment
A comparison of approaches to large-scale data analysis
Proceedings of the 2009 ACM SIGMOD International Conference on Management of data
Parallel bulk insertion for large-scale analytics applications
Proceedings of the 4th International Workshop on Large Scale Distributed Systems and Middleware
Benchmarking cloud-based data management systems
CloudDB '10 Proceedings of the second international workshop on Cloud data management
CloudCmp: comparing public cloud providers
IMC '10 Proceedings of the 10th ACM SIGCOMM conference on Internet measurement
Schism: a workload-driven approach to database replication and partitioning
Proceedings of the VLDB Endowment
What consistency does your key-value store actually provide?
HotDep'10 Proceedings of the Sixth international conference on Hot topics in system dependability
Communications of the ACM
A data generator for cloud-scale benchmarking
TPCTC'10 Proceedings of the Second TPC technology conference on Performance evaluation, measurement and characterization of complex systems
TPCTC'10 Proceedings of the Second TPC technology conference on Performance evaluation, measurement and characterization of complex systems
Consistent and durable data structures for non-volatile byte-addressable memory
FAST'11 Proceedings of the 9th USENIX conference on File and stroage technologies
The SCADS director: scaling a distributed storage system under stringent performance requirements
FAST'11 Proceedings of the 9th USENIX conference on File and stroage technologies
FATE and DESTINI: a framework for cloud recovery testing
Proceedings of the 8th USENIX conference on Networked systems design and implementation
Scalable SQL and NoSQL data stores
ACM SIGMOD Record
Parallel data generation for performance analysis of large, complex RDBMS
Proceedings of the Fourth International Workshop on Testing Database Systems
Zephyr: live migration in shared nothing databases for elastic cloud platforms
Proceedings of the 2011 ACM SIGMOD International Conference on Management of data
A batch of PNUTS: experiences connecting cloud batch and serving systems
Proceedings of the 2011 ACM SIGMOD International Conference on Management of data
Analyzing consistency properties for fun and profit
Proceedings of the 30th annual ACM SIGACT-SIGOPS symposium on Principles of distributed computing
Proceedings of the VLDB Endowment
An efficient multi-tier tablet server storage architecture
Proceedings of the 2nd ACM Symposium on Cloud Computing
Modeling and synthesizing task placement constraints in Google compute clusters
Proceedings of the 2nd ACM Symposium on Cloud Computing
YCSB++: benchmarking and performance debugging advanced features in scalable table stores
Proceedings of the 2nd ACM Symposium on Cloud Computing
ActiveSLA: a profit-oriented admission control framework for database-as-a-service providers
Proceedings of the 2nd ACM Symposium on Cloud Computing
SILT: a memory-efficient, high-performance key-value store
SOSP '11 Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles
PREFAIL: a programmable tool for multiple-failure injection
Proceedings of the 2011 ACM international conference on Object oriented programming systems languages and applications
On the elasticity of NoSQL databases over cloud management platforms
Proceedings of the 20th ACM international conference on Information and knowledge management
Proceedings of the 2nd ACM SIGSPATIAL International Workshop on GeoStreaming
Scalable queries for large datasets using cloud computing: a case study
Proceedings of the 15th Symposium on International Database Engineering & Applications
Eventual consistency: How soon is eventual? An evaluation of Amazon S3's consistency behavior
Proceedings of the 6th Workshop on Middleware for Service Oriented Computing
Enhancing application robustness in cloud data centers
Proceedings of the 2011 Conference of the Center for Advanced Studies on Collaborative Research
Clearing the clouds: a study of emerging scale-out workloads on modern hardware
ASPLOS XVII Proceedings of the seventeenth international conference on Architectural Support for Programming Languages and Operating Systems
DejaVu: accelerating resource allocation in virtualized environments
ASPLOS XVII Proceedings of the seventeenth international conference on Architectural Support for Programming Languages and Operating Systems
A critique of snapshot isolation
Proceedings of the 7th ACM european conference on Computer Systems
LazyBase: trading freshness for performance in a scalable database
Proceedings of the 7th ACM european conference on Computer Systems
Cache craftiness for fast multicore key-value storage
Proceedings of the 7th ACM european conference on Computer Systems
How a consumer can measure elasticity for cloud platforms
ICPE '12 Proceedings of the 3rd ACM/SPEC International Conference on Performance Engineering
Efficient update data generation for DBMS benchmarks
ICPE '12 Proceedings of the 3rd ACM/SPEC International Conference on Performance Engineering
Performance Evaluation of Range Queries in Key Value Stores
Journal of Grid Computing
Serving large-scale batch computed data with project Voldemort
FAST'12 Proceedings of the 10th USENIX conference on File and Storage Technologies
bLSM: a general purpose log structured merge tree
SIGMOD '12 Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data
TIRAMOLA: elastic nosql provisioning through a cloud management platform
SIGMOD '12 Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data
Walnut: a unified cloud object store
SIGMOD '12 Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data
Structured comparative analysis of systems logs to diagnose performance problems
NSDI'12 Proceedings of the 9th USENIX conference on Networked Systems Design and Implementation
Automatic scaling of selective SPARQL joins using the TIRAMOLA system
SWIM '12 Proceedings of the 4th International Workshop on Semantic Web Information Management
"Cut me some slack": latency-aware live migration for databases
Proceedings of the 15th International Conference on Extending Database Technology
Group-Scope query and its access method
APWeb'12 Proceedings of the 14th Asia-Pacific international conference on Web Technologies and Applications
Gumball: a race condition prevention technique for cache augmented SQL database management systems
DBSocial '12 Proceedings of the 2nd ACM SIGMOD Workshop on Databases and Social Networks
Metrics and techniques for quantifying performance isolation in cloud environments
Proceedings of the 8th international ACM SIGSOFT conference on Quality of Software Architectures
Executing Data-Intensive Workloads in a Cloud
CCGRID '12 Proceedings of the 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012)
Minuet: a scalable distributed multiversion B-tree
Proceedings of the VLDB Endowment
LogBase: a scalable log-structured database system in the cloud
Proceedings of the VLDB Endowment
HyperDex: a distributed, searchable key-value store
Proceedings of the ACM SIGCOMM 2012 conference on Applications, technologies, architectures, and protocols for computer communication
Exploiting hardware heterogeneity within the same instance type of Amazon EC2
HotCloud'12 Proceedings of the 4th USENIX conference on Hot Topics in Cloud Ccomputing
Sweet storage SLOs with Frosting
HotCloud'12 Proceedings of the 4th USENIX conference on Hot Topics in Cloud Ccomputing
A parallel page cache: IOPS and caching for multicore systems
HotStorage'12 Proceedings of the 4th USENIX conference on Hot Topics in Storage and File Systems
Poor man's social network: consistently trade freshness for scalability
WebApps'12 Proceedings of the 3rd USENIX conference on Web Application Development
Serializability, not serial: concurrency control and availability in multi-datacenter datastores
Proceedings of the VLDB Endowment
An architecture framework for application-managed scaling of cloud-hosted relational databases
Proceedings of the WICSA/ECSA 2012 Companion Volume
When free is not really free: what does it cost to run a database workload in the cloud?
TPCTC'11 Proceedings of the Third TPC Technology conference on Topics in Performance Evaluation, Measurement and Characterization
Can the elephants handle the NoSQL onslaught?
Proceedings of the VLDB Endowment
Solving big data challenges for enterprise application performance management
Proceedings of the VLDB Endowment
MyCassandra: a cloud storage supporting both read heavy and write heavy workloads
Proceedings of the 5th Annual International Systems and Storage Conference
Methodologies for generating HTTP streaming video workloads to evaluate web server performance
Proceedings of the 5th Annual International Systems and Storage Conference
HyperDex: a distributed, searchable key-value store
ACM SIGCOMM Computer Communication Review - Special october issue SIGCOMM '12
Quantifying the Mismatch between Emerging Scale-Out Applications and Modern Processors
ACM Transactions on Computer Systems (TOCS)
Toward a principled framework for benchmarking consistency
HotDep'12 Proceedings of the Eighth USENIX conference on Hot Topics in System Dependability
Performance isolation and fairness for multi-tenant cloud storage
OSDI'12 Proceedings of the 10th USENIX conference on Operating Systems Design and Implementation
High throughput computing over peer-to-peer networks
Future Generation Computer Systems
High performance RDMA-based design of HDFS over InfiniBand
SC '12 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
The Yahoo!: cloud datastore load balancer
Proceedings of the fourth international workshop on Cloud data management
Towards benchmarking stream data warehouses
Proceedings of the fifteenth international workshop on Data warehousing and OLAP
Chronos: predictable low latency for data center applications
Proceedings of the Third ACM Symposium on Cloud Computing
Optimizing data migration for cloud-based key-value stores
Proceedings of the 21st ACM international conference on Information and knowledge management
Quality-of-service for consistency of data geo-replication in cloud computing
Euro-Par'12 Proceedings of the 18th international conference on Parallel Processing
Estimating resource costs of data-intensive workloads in public clouds
Proceedings of the 10th International Workshop on Middleware for Grids, Clouds and e-Science
Managing service performance in NoSQL distributed storage systems
Proceedings of the 7th Workshop on Middleware for Next Generation Internet Computing
A Distributed Cache for Hadoop Distributed File System in Real-Time Cloud Services
GRID '12 Proceedings of the 2012 ACM/IEEE 13th International Conference on Grid Computing
On a Catalogue of Metrics for Evaluating Commercial Cloud Services
GRID '12 Proceedings of the 2012 ACM/IEEE 13th International Conference on Grid Computing
Actors and publish/subscribe: an efficient approach to scalable distribution in data centers
Proceedings of the 2012 ACM conference on CoNEXT student workshop
Towards Elastic Multi-Tenant Database Replication with Quality of Service
UCC '12 Proceedings of the 2012 IEEE/ACM Fifth International Conference on Utility and Cloud Computing
Hardware-in-the-loop simulation for automated benchmarking of cloud infrastructures
Proceedings of the Winter Simulation Conference
A RAMCloud Storage System based on HDFS: Architecture, implementation and evaluation
Journal of Systems and Software
Fairness and isolation in multi-tenant storage as optimization decomposition
ACM SIGOPS Operating Systems Review
A framework for ranking of cloud computing services
Future Generation Computer Systems
SCORe: a scalable one-copy serializable partial replication protocol
Proceedings of the 13th International Middleware Conference
Evaluating parameter sweep workflows in high performance computing
Proceedings of the 1st ACM SIGMOD Workshop on Scalable Workflow Execution Engines and Technologies
ElasTraS: An elastic, scalable, and self-managing transactional database for the cloud
ACM Transactions on Database Systems (TODS)
Rethinking DRAM Power Modes for Energy Proportionality
MICRO-45 Proceedings of the 2012 45th Annual IEEE/ACM International Symposium on Microarchitecture
Evaluating Cassandra as a manager of large file sets
Proceedings of the 3rd International Workshop on Cloud Data and Platforms
Position paper: cloud system deployment and performance evaluation tools for distributed databases
Proceedings of the 2013 international workshop on Hot topics in cloud services
BigBench: towards an industry standard benchmark for big data analytics
Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data
DBalancer: distributed load balancing for NoSQL data-stores
Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data
COCCUS: self-configured cost-based query services in the cloud
Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data
Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data
LinkBench: a database benchmark based on the Facebook social graph
Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data
Characterizing tenant behavior for placement and crisis mitigation in multitenant DBMSs
Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data
DBMS metrology: measuring query time
Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data
ChainReaction: a causal+ consistent datastore based on chain replication
Proceedings of the 8th ACM European Conference on Computer Systems
MeT: workload aware elasticity for NoSQL
Proceedings of the 8th ACM European Conference on Computer Systems
Performance evaluation of a MongoDB and hadoop platform for scientific data analysis
Proceedings of the 4th ACM workshop on Scientific cloud computing
Rapid development of data generators using meta generators in PDGF
Proceedings of the Sixth International Workshop on Testing Database Systems
COSBench: cloud object storage benchmark
Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering
A provenance model for key-value systems
TaPP'13 Proceedings of the 5th USENIX conference on Theory and Practice of Provenance
MemC3: compact and concurrent MemCache with dumber caching and smarter hashing
nsdi'13 Proceedings of the 10th USENIX conference on Networked Systems Design and Implementation
A provenance model for key-value systems
Proceedings of the 5th USENIX Workshop on the Theory and Practice of Provenance
On benchmarking online social media analytical queries
First International Workshop on Graph Data Management Experiences and Systems
Mercury: bringing efficiency to key-value stores
Proceedings of the 6th International Systems and Storage Conference
Virtualizing power distribution in datacenters
Proceedings of the 40th Annual International Symposium on Computer Architecture
Bubble-flux: precise online QoS management for increased utilization in warehouse scale computers
Proceedings of the 40th Annual International Symposium on Computer Architecture
On estimating actuation delays in elastic computing systems
Proceedings of the 8th International Symposium on Software Engineering for Adaptive and Self-Managing Systems
DAX: a widely distributed multitenant storage service for DBMS hosting
Proceedings of the VLDB Endowment
NoSQL databases: MongoDB vs cassandra
Proceedings of the International C* Conference on Computer Science and Software Engineering
A flexible elastic control plane for private clouds
Proceedings of the 2013 ACM Cloud and Autonomic Computing Conference
ElastMan: elasticity manager for elastic key-value stores in the cloud
Proceedings of the 2013 ACM Cloud and Autonomic Computing Conference
The data partition strategy based on hybrid range consistent hash in NoSQL database
Proceedings of the Fifth International Conference on Internet Multimedia Computing and Service
Toward millions of file system IOPS on low-cost, commodity hardware
SC '13 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
Archiving the relaxed consistency web
Proceedings of the 22nd ACM international conference on Conference on information & knowledge management
Expedited rating of data stores using agile data loading techniques
Proceedings of the 22nd ACM international conference on Conference on information & knowledge management
A comparison of two physical data designs for interactive social networking actions
Proceedings of the 22nd ACM international conference on Conference on information & knowledge management
Data-Intensive Cloud Computing: Requirements, Expectations, Challenges, and Solutions
Journal of Grid Computing
Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles
ACM SIGOPS 24th Symposium on Operating Systems Principles
LibRe: a consistency protocol for modern storage systems
Proceedings of the 6th ACM India Computing Convention
Speedy transactions in multicore in-memory databases
Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles
Consistency-based service level agreements for cloud storage
Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles
Tango: distributed data structures over a shared log
Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles
Limplock: understanding the impact of limpware on scale-out cloud systems
Proceedings of the 4th annual Symposium on Cloud Computing
Understanding and mitigating the impact of load imbalance in the memory caching tier
Proceedings of the 4th annual Symposium on Cloud Computing
Multi-tenancy performance benchmark for web application platforms
ICWE'13 Proceedings of the 13th international conference on Web Engineering
Fault tolerant transaction architectures
ACM SIGACT News
Towards the next generation intelligent BPM: in the era of big data
BPM'13 Proceedings of the 11th international conference on Business Process Management
Generating request streams on Big Data using clustered renewal processes
Performance Evaluation
UpSizeR: Synthetically scaling an empirical relational database
Information Systems
Using one-sided RDMA reads to build a fast, CPU-efficient key-value store
USENIX ATC'13 Proceedings of the 2013 USENIX conference on Annual Technical Conference
On the efficiency of durable state machine replication
USENIX ATC'13 Proceedings of the 2013 USENIX conference on Annual Technical Conference
USENIX ATC'13 Proceedings of the 2013 USENIX conference on Annual Technical Conference
Network support for resource disaggregation in next-generation datacenters
Proceedings of the Twelfth ACM Workshop on Hot Topics in Networks
Scalable transactions across heterogeneous NoSQL key-value data stores
Proceedings of the VLDB Endowment
Low-latency multi-datacenter databases using replicated commit
Proceedings of the VLDB Endowment
GMonE: A complete approach to cloud monitoring
Future Generation Computer Systems
Anti-caching: a new approach to database management system architecture
Proceedings of the VLDB Endowment
Proceedings of the 8th International Conference on Ubiquitous Information Management and Communication
Write policies for host-side flash caches
FAST'13 Proceedings of the 11th USENIX conference on File and Storage Technologies
Log-structured memory for DRAM-based storage
FAST'14 Proceedings of the 12th USENIX conference on File and Storage Technologies
Toward a scale-out data-management middleware for low-latency enterprise computing
IBM Journal of Research and Development
NSDI'14 Proceedings of the 11th USENIX Conference on Networked Systems Design and Implementation
MICA: a holistic approach to fast in-memory key-value storage
NSDI'14 Proceedings of the 11th USENIX Conference on Networked Systems Design and Implementation
Hi-index | 0.02 |
While the use of MapReduce systems (such as Hadoop) for large scale data analysis has been widely recognized and studied, we have recently seen an explosion in the number of systems developed for cloud data serving. These newer systems address "cloud OLTP" applications, though they typically do not support ACID transactions. Examples of systems proposed for cloud serving use include BigTable, PNUTS, Cassandra, HBase, Azure, CouchDB, SimpleDB, Voldemort, and many others. Further, they are being applied to a diverse range of applications that differ considerably from traditional (e.g., TPC-C like) serving workloads. The number of emerging cloud serving systems and the wide range of proposed applications, coupled with a lack of apples-to-apples performance comparisons, makes it difficult to understand the tradeoffs between systems and the workloads for which they are suited. We present the "Yahoo! Cloud Serving Benchmark" (YCSB) framework, with the goal of facilitating performance comparisons of the new generation of cloud data serving systems. We define a core set of benchmarks and report results for four widely used systems: Cassandra, HBase, Yahoo!'s PNUTS, and a simple sharded MySQL implementation. We also hope to foster the development of additional cloud benchmark suites that represent other classes of applications by making our benchmark tool available via open source. In this regard, a key feature of the YCSB framework/tool is that it is extensible--it supports easy definition of new workloads, in addition to making it easy to benchmark new systems.