Allocation of partitioned data by using a neural network based approach

  • Authors:
  • Manghui Tu;Zhonghang Xia;Peng Li;Nasser Tadayon

  • Affiliations:
  • Computer Science and Information Science Department, Southern Utah University, UT 84720, USA;Department of Computer Science, West Kentucky University, USA;Computer Science Department, University of Texas at Dallas, USA;Computer Science and Information Science Department, Southern Utah University, UT 84720, USA

  • Venue:
  • Neurocomputing
  • Year:
  • 2009

Quantified Score

Hi-index 0.01

Visualization

Abstract

Secret sharing and erasure coding based approaches have been used in distributed storage systems to ensure confidentiality, integrity, and availability of critical information. However, these approaches introduce some access overhead since each access now needs to access multiple data sites to retrieve the original data. In this paper, we investigate allocation of data partitions in the network for data objects that are partitioned by using secret sharing schemes or erasure coding schemes. We define the problem as finding M nodes in the network to host the data partitions so that the total communication cost (the sum of the read communication cost and write communication cost in terms of communication distance) is minimized to read K partitions and update M partitions by all nodes in the network. This problem is NP-hard. We propose a self-organizing feature map (SOFM)-like algorithm and a heuristic algorithm for this problem. The numerical study shows both algorithms are effective for this problem.