A framework for data management and transfer in grid environments

  • Authors:
  • Haojie Zhou;Xingwu Liu;Liqiang Cao;Li Zha

  • Affiliations:
  • Research Center for Grid and Service Computing, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China;Research Center for Grid and Service Computing, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China;Research Center for Grid and Service Computing, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China;Research Center for Grid and Service Computing, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China

  • Venue:
  • EUC'06 Proceedings of the 2006 international conference on Embedded and Ubiquitous Computing
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

The main obstacles to grid file management come from the fact that grid file resources are typically stored in heterogeneous and distributed environment and accessed through various protocols. In this paper, we propose a grid file management system called Vega [1][2] Hotfile2 for data-intensive application in widely distributed systems and grid environments. Widely distributed and heterogeneous storage resource can converge into a single uniform storage resource using virtualization techniques in Hotfile2. It is a grid middleware provides a uniform file resources view and a high level interface to access widely distributed and heterogeneous data resources. Presented here are two services that we believe are fundamental to any grid file management system: high-performance, secure transport and metadata management. In this paper, we discuss many aspects of our design and implementation in Hotfile2, present the file system interface designed to support distributed applications such as batch job. Experiments are done to illustrate its performance, extensibility, and various other features