Using Disk Throughput Data in Predictions of End-to-End Grid Data Transfers
GRID '02 Proceedings of the Third International Workshop on Grid Computing
Predicting the Performance of Wide Area Data Transfers
IPDPS '02 Proceedings of the 16th International Parallel and Distributed Processing Symposium
Access Time Estimation for Tertiary Storage Systems
Euro-Par '02 Proceedings of the 8th International Euro-Par Conference on Parallel Processing
Predicting Sporadic Grid Data Transfers
HPDC '02 Proceedings of the 11th IEEE International Symposium on High Performance Distributed Computing
A distributed multi-storage I/O system for data intensive scientific computing
Parallel Computing - Special issue: Parallel and distributed scientific and engineering computing
A high-performance distributed parallel file system for data-intensive computations
Journal of Parallel and Distributed Computing
Using Regression Techniques to Predict Large Data Transfers
International Journal of High Performance Computing Applications
DHIS: discriminating hierarchical storage
SYSTOR '09 Proceedings of SYSTOR 2009: The Israeli Experimental Systems Conference
A VO-based two-swtage replica replacement algorithm
NPC'07 Proceedings of the 2007 IFIP international conference on Network and parallel computing
GEDAS: a data management system for data grid environments
ICCS'05 Proceedings of the 5th international conference on Computational Science - Volume Part I
| Hi-index | 0.00 |
I/O intensive applications have posed great challenges to computational scientists. A major problem of these applications is that users have to sacrifice performance requirement in order to satisfy storage capacity requirement in a conventional computing environment. Further performance improvement is impeded by the physical nature of these storage media even state-of-the-art I/O optimizations are employed.In this paper, we present a distributed multi-storage resource architecture that can satisfy both performance and capacity requirements by employing multiple storage resources. Compared to traditional single storage resource architecture, our architecture provides a more flexible and reliable computing environment. It can bring new opportunities for high performance computing as well as inherit state-of-the-art I/O optimization approaches that have already been developed. We also develop an Application Programming Interface (API) that provides transparent management and access to various storage resources in our computing environment. As I/O usually dominates the performance in I/O intensive applications, we establish an I/O performance prediction mechanism, which consists of a performance database and a prediction algorithm to help users better evaluate and schedule their applications. A tool is also developed to help users automatically generate the performance database. The experiments show that our multi-storage resource architecture is a promising platform for high performance distributed computing.