Combinatorial algorithms for integrated circuit layout
Combinatorial algorithms for integrated circuit layout
Computers and Intractability: A Guide to the Theory of NP-Completeness
Computers and Intractability: A Guide to the Theory of NP-Completeness
A hypergraph-partitioning approach for coarse-grain decomposition
Proceedings of the 2001 ACM/IEEE conference on Supercomputing
Distributed and Parallel Databases
A linear-time heuristic for improving network partitions
DAC '82 Proceedings of the 19th Design Automation Conference
Scientific workflow management and the Kepler system: Research Articles
Concurrency and Computation: Practice & Experience - Workflow in Grid Systems
Scheduling Data-IntensiveWorkflows onto Storage-Constrained Distributed Resources
CCGRID '07 Proceedings of the Seventh IEEE International Symposium on Cluster Computing and the Grid
netWorker - Cloud computing: PC functions move onto the web
Multi-level direct K-way hypergraph partitioning with multiple constraints and fixed vertices
Journal of Parallel and Distributed Computing
On Building Scientific Workflow Systems for Data Management in the Cloud
ESCIENCE '08 Proceedings of the 2008 Fourth IEEE International Conference on eScience
On the Use of Cloud Computing for Scientific Workflows
ESCIENCE '08 Proceedings of the 2008 Fourth IEEE International Conference on eScience
A repartitioning hypergraph model for dynamic load balancing
Journal of Parallel and Distributed Computing
Mapping filtering streaming applications with communication costs
Proceedings of the twenty-first annual symposium on Parallelism in algorithms and architectures
Resource monitoring and management with OVIS to enable HPC in cloud computing environments
IPDPS '09 Proceedings of the 2009 IEEE International Symposium on Parallel&Distributed Processing
Robust data placement in urgent computing environments
IPDPS '09 Proceedings of the 2009 IEEE International Symposium on Parallel&Distributed Processing
Scientific workflows and clouds
Crossroads - Plugging Into the Cloud
A data placement strategy in scientific cloud workflows
Future Generation Computer Systems
File-Access Characteristics of Data-Intensive Workflow Applications
CCGRID '10 Proceedings of the 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing
P&P: A Combined Push-Pull Model for Resource Monitoring in Cloud Computing Environment
CLOUD '10 Proceedings of the 2010 IEEE 3rd International Conference on Cloud Computing
Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing
Volley: automated data placement for geo-distributed cloud services
NSDI'10 Proceedings of the 7th USENIX conference on Networked systems design and implementation
Data Sharing Options for Scientific Workflows on Amazon EC2
Proceedings of the 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis
On reducing i/o overheads in large-scale invariant subspace projections
Euro-Par'11 Proceedings of the 2011 international conference on Parallel Processing
Hi-index | 0.00 |
We consider the problem of optimizing the execution of data-intensive scientific workflows in the Cloud. We address the problem under the following scenario. The tasks of the workflows communicate through files; the output of a task is used by another task as an input file and if these tasks are assigned on different execution sites, a file transfer is necessary. The output files are to be stored at a site. Each execution site is to be assigned a certain percentage of the files and tasks. These percentages, called target weights, are pre-determined and reflect either user preferences or the storage capacity and computing power of the sites. The aim is to place the data files into and assign the tasks to the execution sites so as to reduce the cost associated with the file transfers, while complying with the target weights. To do this, we model the workflow as a hypergraph and with a hypergraph-partitioning-based formulation, we propose a heuristic which generates data placement and task assignment schemes simultaneously. We report simulation results on a number of real-life and synthetically generated scientific workflows. Our results show that the proposed heuristic is fast, and can find mappings and assignments which reduce file transfers, while respecting the target weights.