Dynamic file migration in distributed computer systems
Communications of the ACM
Input/output characteristics of scalable parallel applications
Supercomputing '95 Proceedings of the 1995 ACM/IEEE conference on Supercomputing
Lessons from characterizating the input/output behavior of parallel scientific applications
Performance Evaluation - Special issue on tools for performance evaluation
File Assignment in Parallel I/O Systems with Minimal Variance of Service Time
IEEE Transactions on Computers
Comparative Models of the File Assignment Problem
ACM Computing Surveys (CSUR)
Host load prediction using linear models
Cluster Computing
GPFS: A Shared-Disk File System for Large Computing Clusters
FAST '02 Proceedings of the Conference on File and Storage Technologies
Workload Characterization of Input/Output Intensive Parallel Applications
Proceedings of the 9th International Conference on Computer Performance Evaluation: Modelling Techniques and Tools
Data partitioning and load balancing in parallel disk systems
The VLDB Journal — The International Journal on Very Large Data Bases
Design of an I/O Balancing File System on Web Server Clusters
ICPP '00 Proceedings of the 2000 International Workshop on Parallel Processing
Automatic ARIMA Time Series Modeling for Adaptive I/O Prefetching
IEEE Transactions on Parallel and Distributed Systems
Dynamic Metadata Management for Petabyte-Scale File Systems
Proceedings of the 2004 ACM/IEEE conference on Supercomputing
Handling Heterogeneity in Shared-Disk File Systems
Proceedings of the 2003 ACM/IEEE conference on Supercomputing
Communications of the ACM - Designing for the mobile device
Integrated Performance Monitoring of a Cosmology Application on Leading HEC Platforms
ICPP '05 Proceedings of the 2005 International Conference on Parallel Processing
IEEE Transactions on Parallel and Distributed Systems
The Business Strategy Perspective on the Development of Decision Support Systems
CIMCA '05 Proceedings of the International Conference on Computational Intelligence for Modelling, Control and Automation and International Conference on Intelligent Agents, Web Technologies and Internet Commerce Vol-2 (CIMCA-IAWTIC'06) - Volume 02
Improving Disk I/O Load Prediction Using Statistical Parameter History in Online for Grid Computing
IEICE - Transactions on Information and Systems
PVFS: a parallel file system for linux clusters
ALS'00 Proceedings of the 4th annual Linux Showcase & Conference - Volume 4
Models and framework for supporting runtime decisions in Web-based systems
ACM Transactions on the Web (TWEB)
Proceedings of the 2008 ACM/IEEE conference on Supercomputing
Proceedings of the 2008 ACM/IEEE conference on Supercomputing
Load prediction using hybrid model for computational grid
GRID '07 Proceedings of the 8th IEEE/ACM International Conference on Grid Computing
Predicting Running Time of Grid Tasks based on CPU Load Predictions
GRID '06 Proceedings of the 7th IEEE/ACM International Conference on Grid Computing
Small-file access in parallel file systems
IPDPS '09 Proceedings of the 2009 IEEE International Symposium on Parallel&Distributed Processing
A file assignment strategy independent of workload characteristic assumptions
ACM Transactions on Storage (TOS)
CA-NFS: A congestion-aware network file system
ACM Transactions on Storage (TOS)
Towards Exascale Distributed Data Management
International Journal of High Performance Computing Applications
Short-term prediction models for server management in Internet-based contexts
Decision Support Systems
Scalable massively parallel I/O to task-local files
Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis
PLFS: a checkpoint filesystem for parallel applications
Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis
I/O performance challenges at leadership scale
Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis
Adaptive Distributed Load Balancing Algorithm Based on Live Migration of Virtual Machines in Cloud
NCM '09 Proceedings of the 2009 Fifth International Joint Conference on INC, IMS and IDC
A study of client-based caching for parallel i/o
A study of client-based caching for parallel i/o
Self-Acting Load Balancing with Parallel Sub File Migration for Parallel File System
CSO '10 Proceedings of the 2010 Third International Joint Conference on Computational Science and Optimization - Volume 02
Panache: a parallel file system cache for global file access
FAST'10 Proceedings of the 8th USENIX conference on File and storage technologies
A Balanced Allocation Strategy for File Assignment in Parallel I/O Systems
NAS '10 Proceedings of the 2010 IEEE Fifth International Conference on Networking, Architecture, and Storage
A Large-Scale Study of Failures in High-Performance Computing Systems
IEEE Transactions on Dependable and Secure Computing
The International Exascale Software Project roadmap
International Journal of High Performance Computing Applications
A Segment-Level Adaptive Data Layout Scheme for Improved Load Balance in Parallel File Systems
CCGRID '11 Proceedings of the 2011 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing
| Hi-index | 0.00 |
Many solutions have been proposed to tackle the load imbalance issue of parallel file systems. However, all these solutions either adopt centralized algorithms, or lack considerations for both the network transmission and the tradeoff between benefits and side-effects of each dynamic file migration. Therefore, existing solutions will be prohibitively inefficient in large-scale parallel file systems. To address this problem, this paper presents SALB, a dynamic and adaptive load balancing algorithm which is totally based on a distributed architecture. To be also aware of the network transmission, SALB on the one hand adopts an adaptively adjusted load collection threshold in order to reduce the message exchanges for load collection, and on the other hand it employs an on-line load prediction model with a view to reducing the decision delay caused by the network transmission latency. Moreover, SALB employs an optimization model for selecting the migration candidates so as to balance the benefits and the side-effects of each dynamic file migration. Extensive experiments are conducted to prove the effectiveness of SALB. The results show that SALB achieves an optimal performance not only on the mean response time but also on the resource utilization among the schemes for comparison. The simulation results also indicate that SALB is able to deliver high scalability.