Massively-parallel stream processing under QoS constraints with Nephele
Proceedings of the 21st international symposium on High-Performance Parallel and Distributed Computing
Proceedings of the 2012 Joint EDBT/ICDT Workshops
AROMA: automated resource allocation and configuration of mapreduce environment in the cloud
Proceedings of the 9th international conference on Autonomic computing
Optimal resource provisioning for cloud computing environment
The Journal of Supercomputing
An architecture for enterprise PC cloud
International Journal of Computational Science and Engineering
Proceedings of the first ACM workshop on Optimization techniques for resources management in clouds
Resource allocation in cloud computing: model and algorithm
International Journal of Web and Grid Services
Adaptive Online Compression in Clouds--Making Informed Decisions in Virtual Machine Environments
Journal of Grid Computing
Managing large numbers of business processes with cloud workflow systems
AusPDC '12 Proceedings of the Tenth Australasian Symposium on Parallel and Distributed Computing - Volume 127
Scheduling jobs in the cloud using on-demand and reserved instances
Euro-Par'13 Proceedings of the 19th international conference on Parallel Processing
Euro-Par'13 Proceedings of the 19th international conference on Parallel Processing
Computers and Operations Research
Energy-saving self-configuring networked data centers
Computer Networks: The International Journal of Computer and Telecommunications Networking
ACM Transactions on Autonomous and Adaptive Systems (TAAS)
Multi-Layer Resource Management in Cloud Computing
Journal of Network and Systems Management
Nephele streaming: stream processing under QoS constraints at scale
Cluster Computing
Hi-index | 0.00 |
In recent years ad hoc parallel data processing has emerged to be one of the killer applications for Infrastructure-as-a-Service (IaaS) clouds. Major Cloud computing companies have started to integrate frameworks for parallel data processing in their product portfolio, making it easy for customers to access these services and to deploy their programs. However, the processing frameworks which are currently used have been designed for static, homogeneous cluster setups and disregard the particular nature of a cloud. Consequently, the allocated compute resources may be inadequate for big parts of the submitted job and unnecessarily increase processing time and cost. In this paper, we discuss the opportunities and challenges for efficient parallel data processing in clouds and present our research project Nephele. Nephele is the first data processing framework to explicitly exploit the dynamic resource allocation offered by today's IaaS clouds for both, task scheduling and execution. Particular tasks of a processing job can be assigned to different types of virtual machines which are automatically instantiated and terminated during the job execution. Based on this new framework, we perform extended evaluations of MapReduce-inspired processing jobs on an IaaS cloud system and compare the results to the popular data processing framework Hadoop.