Future Generation Computer Systems - Special issue on metacomputing
Journal of Parallel and Distributed Computing
A taxonomy and survey of grid resource management systems for distributed computing
Software—Practice & Experience
Dynamic parallel access to replicated content in the internet
IEEE/ACM Transactions on Networking (TON)
Online Prediction of the Running Time of Tasks
Cluster Computing
Simulation of Dynamic Grid Replication Strategies in OptorSim
GRID '02 Proceedings of the Third International Workshop on Grid Computing
Data Management in an International Data Grid Project
GRID '00 Proceedings of the First IEEE/ACM International Workshop on Grid Computing
Replica Selection in the Globus Data Grid
CCGRID '01 Proceedings of the 1st International Symposium on Cluster Computing and the Grid
QoS as Middleware: Bandwidth Reservation System Design
HPDC '99 Proceedings of the 8th IEEE International Symposium on High Performance Distributed Computing
Distributed Job Scheduling on Computational Grids Using Multiple Simultaneous Requests
HPDC '02 Proceedings of the 11th IEEE International Symposium on High Performance Distributed Computing
Decoupling Computation and Data Scheduling in Distributed Data-Intensive Applications
HPDC '02 Proceedings of the 11th IEEE International Symposium on High Performance Distributed Computing
Software—Practice & Experience
Adjusted fair scheduling and non-linear workload prediction for QoS guarantees in grid computing
Computer Communications
MapReduce: simplified data processing on large clusters
OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
A Predictive Technique for Replica Selection in Grid Environment
CCGRID '07 Proceedings of the Seventh IEEE International Symposium on Cluster Computing and the Grid
Study of Different Replica Placement and Maintenance Strategies in Data Grid
CCGRID '07 Proceedings of the Seventh IEEE International Symposium on Cluster Computing and the Grid
Intelligent Scheduling and Replication in Datagrids: a Synergistic Approach
CCGRID '07 Proceedings of the Seventh IEEE International Symposium on Cluster Computing and the Grid
Fair Scheduling Algorithms in Grids
IEEE Transactions on Parallel and Distributed Systems
Data Consolidation: A Task Scheduling and Data Migration Technique for Grid Networks
CCGRID '08 Proceedings of the 2008 Eighth IEEE International Symposium on Cluster Computing and the Grid
Future Generation Computer Systems
Fair resource sharing in hierarchical virtual organizations for global grids
GRID '07 Proceedings of the 8th IEEE/ACM International Conference on Grid Computing
A framework for providing hard delay guarantees and user fairness in Grid computing
Future Generation Computer Systems
Future Generation Computer Systems
Multi-cost job routing and scheduling in Grid networks
Future Generation Computer Systems
Developing Scheduling Policies in gLite Middleware
CCGRID '09 Proceedings of the 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid
Integration of scheduling and replication in data grids
HiPC'04 Proceedings of the 11th international conference on High Performance Computing
Do we need a crystal ball for task migration?
HotPar'12 Proceedings of the 4th USENIX conference on Hot Topics in Parallelism
| Hi-index | 0.00 |
We examine a task scheduling and data migration problem for grid networks, which we refer to as the Data Consolidation (DC) problem. DC arises when a task concurrently requests multiple pieces of data, possibly scattered throughout the grid network, that have to be present at a selected site before the task's execution starts. In such a case, the scheduler and the data manager must select (i) the data replicas to be used, (ii) the site where these data will be gathered for the task to be executed, and (iii) the routing paths to be followed; this is assuming that the selected datasets are transferred concurrently to the execution site. The algorithms or policies for selecting the data replicas, the data consolidating site and the corresponding paths comprise a Data Consolidation scheme. We propose and experimentally evaluate several DC schemes of polynomial number of operations that attempt to estimate the cost of the concurrent data transfers, to avoid congestion that may appear due to these transfers and to provide fault tolerance. Our simulation results strengthen our belief that DC is an important problem that needs to be addressed in the design of data grids, and can lead, if performed efficiently, to significant benefits in terms of task delay, network load and other performance parameters.