Riding the elephant: managing ensembles with hadoop
Proceedings of the 2011 ACM international workshop on Many task computing on grids and supercomputers
Cutting MapReduce cost with spot market
HotCloud'11 Proceedings of the 3rd USENIX conference on Hot topics in cloud computing
The seven deadly sins of cloud computing research
HotCloud'12 Proceedings of the 4th USENIX conference on Hot Topics in Cloud Ccomputing
Scalable parallel computing on clouds using Twister4Azure iterative MapReduce
Future Generation Computer Systems
MRBS: towards dependability benchmarking for hadoop mapreduce
Euro-Par'12 Proceedings of the 18th international conference on Parallel processing workshops
An improved partitioning mechanism for optimizing massive data analysis using MapReduce
The Journal of Supercomputing
| Hi-index | 0.00 |
Like a traditional Operating System (OS), a cloud OS is responsible for managing the low level cloud resources and presenting a high level interface to the application programmers in order to hide the infrastructure details. However, unlike a traditional OS, a cloud OS has to manage these resources at scale. If a cloud OS has already taken on the complexity to make its services scalable, we should be able to greatly simplify a large-scale system design and implementation if we build on top of it. Unfortunately, a cloud's scale comes at a price. For example, Amazon cloud not only relies on horizontal scaling, but it also adopts a weaker consistency model called eventual consistency. We describe Cloud MapReduce (CMR), which implements the MapReduce programming model on top of the Amazon cloud OS. CMR is a demonstration that it is possible to overcome the cloud limitations and simplify system design and implementation by building on top of a cloud OS. We describe how we overcome the limitations presented by horizontal scaling and the weaker consistency guarantee. Our experimental results show that CMR runs faster than Hadoop, another implementation of MapReduce, and that CMR is a practical system. We believe that the techniques we used are general enough that they can be used to build other systems on top of a cloud OS.