Caching and database scaling in distributed shared-nothing information retrieval systems

  • Authors:
  • Anthony Tomasic;Hector Garcia-Molina

  • Affiliations:
  • Stanford University, Department of Computer Science, Margaret Jacks Hall, Stanford, CA;Stanford University, Department of Computer Science, Margaret Jacks Hall, Stanford, CA

  • Venue:
  • SIGMOD '93 Proceedings of the 1993 ACM SIGMOD international conference on Management of data
  • Year:
  • 1993

Quantified Score

Hi-index 0.00

Visualization

Abstract

A common class of existing information retrieval system provides access to abstracts. For example Stanford University, through its FOLIO system, provides access to the INSPECT database of abstracts of the literature on physics, computer science, electrical engineering, etc. In this paper this database is studied by using a trace-driven simulation. We focus on physical index design, inverted index caching, and database scaling in a distributed shared-nothing system. All three issues are shown to have a strong effect on response time and throughput. Database scaling is explored in two ways. One way assumes an “optimal” configuration for a single host and then linearly scales the database by duplicating the host architecture as needed. The second way determines the optimal number of hosts given a fixed database size.