Archive storage system design for long-term storage of massive amounts of data

  • Authors:
  • P. L. Bradshaw;K. W. Brannon;T. Clark;K. Dahman;S. Doraiswamy;L. Duyanovich;B. L. Hillsberg;W. Hineman;M. Kaczmarski;B. J. Klingenberg;X. Ma;R. Rees

  • Affiliations:
  • IBM Systems and Technology Group, Almaden Research Center, San Jose, California;IBM Research Division, Almaden Research Center, San Jose, California;IBM Tivoli Systems, Tucson, Arizona;IBM Systems and Technology Group, Tucson, Arizona;IBM Research Division, Almaden Research Center, San Jose, California;IBM Research Division, Almaden Research Center, San Jose, California;IBM Research Division, Almaden Research Center, San Jose, California;IBM Research Division, Almaden Research Center, San Jose, California;IBM Tivoli Systems, Tucson, Arizona;IBM Software Group, Almaden Research Center, San Jose, California;IBM Research Division, Almaden Research Center, San Jose, California;IBM Research Division, Almaden Research Center, San Jose, California

  • Venue:
  • IBM Journal of Research and Development
  • Year:
  • 2008

Quantified Score

Hi-index 0.01

Visualization

Abstract

A dramatic shift is underway in how organizations use computer storage. This shift will have a profound impact on storage system design. The requirement for storage of traditional transactional data is being supplemented by the necessity to store information for long periods. In 2005, a total of 2,700 petabytes of storage was allocated worldwide for information that required long-term retention, and this amount is expected to grow to an estimated 27,200 petabytes by 2010. In this paper, we review the requirements for long-term storage of data and describe an innovative approach for developing a highly scalable and flexible archive storage system using commercial off-the-shelf (COTS) components. Such a system is expected to be capable of preserving data for decades, providing efficient policy-based management of the data, and allowing efficient search and access to data regardless of data content or location.