Algorithmic challenges in web search engines

  • Authors:
  • Ricardo Baeza-Yates

  • Affiliations:
  • Spain & Santiago, Yahoo! Research, Barcelona, Chile

  • Venue:
  • WEA'06 Proceedings of the 5th international conference on Experimental Algorithms
  • Year:
  • 2006

Quantified Score

Hi-index 0.01

Visualization

Abstract

We present the main algorithmic challenges that large Web search engines face today. These challenges are present in all the modules of a Web retrieval system, ranging from the gathering of the data to be indexed (crawling) to the selection and ordering of the answers to a query (searching and ranking). Most of the challenges are ultimately related to the quality of the answer or the efficiency in obtaining it, although some are relevant even to the existence of current search engines: context based advertising. As the Web grows and changes at a fast pace, the algorithms behind these challenges must rely in large scale experimentation, both in data volume and computation time, to understand the main issues that affect them. We show examples of our own research and of the state of the art. The full version of this paper appears in [1] .