Archiving the relaxed consistency web

  • Authors:

  • Zhiwu Xie;Herbert Van de Sompel;Jinyang Liu;Johann van Reenen;Ramiro Jordan

  • Affiliations:

  • Virginia Tech, Blacksburg, VA, USA;Los Alamos National Laboratory, Los Alamos, NM, USA;Howard Hughes Medical Institute, Ashburn, VA, USA;University of New Mexico, Albuquerque, NM, USA;University of New Mexico, Albuquerque, NM, USA

  • Venue:
  • Proceedings of the 22nd ACM international conference on Conference on information & knowledge management
  • Year:
  • 2013

Quantified Score

Hi-index 0.00

Visualization

Abstract

The historical, cultural, and intellectual importance of archiving the web has been widely recognized. Today, all countries with high Internet penetration rate have established high-profile archiving initiatives to crawl and archive the fast-disappearing web content for long-term use. As web technologies evolve, established web archiving techniques face challenges. This paper focuses on the potential impact of the relaxed consistency web design on crawler driven web archiving. Relaxed consistent websites may disseminate, albeit ephemerally, inaccurate and even contradictory information. If captured and preserved in the web archives as historical records, such information will degrade the overall archival quality. To assess the extent of such quality degradation, we build a simplified feed-following application and simulate its operation with synthetic workloads. The results indicate that a non-trivial portion of a relaxed consistency web archive may contain observable inconsistency, and the inconsistency window may extend significantly longer than that observed at the data store. We discuss the nature of such quality degradation and propose a few possible remedies.