Flexible and efficient transfer of multidimensional data in grid

Authors:
Huan Yang;Fei Shen;Linpeng Huang
Affiliations:
Shanghai JiaoTong University, Shanghai, China;Shanghai JiaoTong University, Shanghai, China;Shanghai JiaoTong University, Shanghai, China
Venue:
CHINA HPC '07 Proceedings of the 2007 Asian technology information program's (ATIP's) 3rd workshop on High performance computing in China: solution approaches to impediments for high performance computing
Year:
2007

Citing 8
Cited 0

The Grid File: An Adaptable, Symmetric Multikey File Structure

ACM Transactions on Database Systems (TODS)
PSockets: the case for application-level network striping for data intensive applications using high speed wide area networks

Proceedings of the 2000 ACM/IEEE conference on Supercomputing
The K-D-B-tree: a search structure for large multidimensional dynamic indexes

SIGMOD '81 Proceedings of the 1981 ACM SIGMOD international conference on Management of data
Data management and transfer in high-performance computational grid environments

Parallel Computing - Parallel data-intensive algorithms and applications
G-Tree: A New Data Structure for Organizing Multidimensional Data

IEEE Transactions on Knowledge and Data Engineering
The End-to-End Performance Effects of Parallel TCP Sockets on a Lossy Wide-Area Network

IPDPS '02 Proceedings of the 16th International Parallel and Distributed Processing Symposium
Secure, Efficient Data Transport and Replica Management for High-Performance Data-Intensive Computing

MSS '01 Proceedings of the Eighteenth IEEE Symposium on Mass Storage Systems and Technologies
Eliminating Replica Selection - Using Multiple Replicas to Accelerate Data Transfer on Grids

ICPADS '04 Proceedings of the Parallel and Distributed Systems, Tenth International Conference

Quantified Score

Hi-index	0.00

Visualization

Abstract

When grid applications operate on multidimensional data, query and transfer are two widely used operations. To support flexible query operation, a storage structure based on partition the multidimensional data is adopted. However, this will degrade the performance of the transfer. This paper investigates this problem and proposes the logically linked file (LLF) approach to solve it. By logically linking the qualified non-continuous data sets to a LLF and providing a specific transfer mechanism for LLF, the transfer performance is improved. The experiment shows the transfer performance isn't affected by the storage structure any more.