On caching search engine query results

Authors:
E.P Markatos
Affiliations:
Institute of Computer Science (ICS), Foundation for Research & Technology - Hellas (FORTH), P.O. Box 1385, Heraklion, Crete, GR-711-10 Greece, also University of Crete
Venue:
Computer Communications
Year:
2001

Citing 22
Cited 47

Data cache management using frequency-based replacement

SIGMETRICS '90 Proceedings of the 1990 ACM SIGMETRICS conference on Measurement and modeling of computer systems
The LRU-K page replacement algorithm for database disk buffering

SIGMOD '93 Proceedings of the 1993 ACM SIGMOD international conference on Management of data
Caching strategies to improve disk system performance

Computer
Main memory caching of Web documents

Proceedings of the fifth international World Wide Web conference on Computer networks and ISDN systems
Query caching and optimization in distributed mediator systems

SIGMOD '96 Proceedings of the 1996 ACM SIGMOD international conference on Management of data
Using predictive prefetching to improve World Wide Web latency

ACM SIGCOMM Computer Communication Review
Removal policies in network caches for World-Wide Web documents

Conference proceedings on Applications, technologies, architectures, and protocols for computer communications
Potential benefits of delta encoding and data compression for HTTP

SIGCOMM '97 Proceedings of the ACM SIGCOMM '97 conference on Applications, technologies, architectures, and protocols for computer communication
Proxy caching that estimates page load delays

Selected papers from the sixth international conference on World Wide Web
A case for delay-conscious caching of Web documents

Selected papers from the sixth international conference on World Wide Web
Locality-aware request distribution in cluster-based network servers

Proceedings of the eighth international conference on Architectural support for programming languages and operating systems
Speculative Data Dissemination and Service to Reduce Server Load, Network Traffic and Service Time in Distributed Information Systems

ICDE '96 Proceedings of the Twelfth International Conference on Data Engineering
A scalable and highly available web server

COMPCON '96 Proceedings of the 41st IEEE International Computer Conference
The case for geographical push-caching

HOTOS '95 Proceedings of the Fifth Workshop on Hot Topics in Operating Systems (HotOS-V)
Organization-based analysis of web-object sharing and caching

USITS'99 Proceedings of the 2nd conference on USENIX Symposium on Internet Technologies and Systems - Volume 2
Prefetching hyperlinks

USITS'99 Proceedings of the 2nd conference on USENIX Symposium on Internet Technologies and Systems - Volume 2
Exploring the bounds of web latency reduction from caching and prefetching

USITS'97 Proceedings of the USENIX Symposium on Internet Technologies and Systems on USENIX Symposium on Internet Technologies and Systems
Improving web server performance by caching dynamic data

USITS'97 Proceedings of the USENIX Symposium on Internet Technologies and Systems on USENIX Symposium on Internet Technologies and Systems
Cost-aware WWW proxy caching algorithms

USITS'97 Proceedings of the USENIX Symposium on Internet Technologies and Systems on USENIX Symposium on Internet Technologies and Systems
A hierarchical internet object cache

ATEC '96 Proceedings of the 1996 annual conference on USENIX Annual Technical Conference
Active cache: caching dynamic contents on the Web

Middleware '98 Proceedings of the IFIP International Conference on Distributed Systems Platforms and Open Distributed Processing
Defining high-speed protocols: five challenges and an example that survives the challenges

IEEE Journal on Selected Areas in Communications

Toward a distributed terabyte text retrieval system in China-US million book digital library

Proceedings of the 2nd ACM/IEEE-CS joint conference on Digital libraries
Boosting the performance of Web search engines: Caching and prefetching query results by exploiting historical usage data

ACM Transactions on Information Systems (TOIS)
Scalability analysis of three monitoring and information systems: MDS2, R-GMA, and Hawkeye

Journal of Parallel and Distributed Computing
The impact of caching on search engines

SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
Load-balancing and caching for collection selection architectures

Proceedings of the 2nd international conference on Scalable information systems
Static query result caching revisited

Proceedings of the 17th international conference on World Wide Web
Design trade-offs for search engine caching

ACM Transactions on the Web (TWEB)
Cache-aware load balancing for question answering

Proceedings of the 17th ACM conference on Information and knowledge management
A metric cache for similarity search

Proceedings of the 2008 ACM workshop on Large-Scale distributed systems for information retrieval
Caching content-based queries for robust and efficient image retrieval

Proceedings of the 12th International Conference on Extending Database Technology: Advances in Database Technology
A survey on session detection methods in query logs and a proposal for future evaluation

Information Sciences: an International Journal
Nearest-neighbor caching for content-match applications

Proceedings of the 18th international conference on World wide web
A Last-Resort Semantic Cache for Web Queries

SPIRE '09 Proceedings of the 16th International Symposium on String Processing and Information Retrieval
Beyond pages: supporting efficient, scalable entity search with dual-inversion index

Proceedings of the 13th International Conference on Extending Database Technology
Tuning the capacity of search engines: Load-driven routing and incremental caching to reduce and balance the load

ACM Transactions on Information Systems (TOIS)
A hybrid cache and prefetch mechanism for scientific literature search engines

ICWE'07 Proceedings of the 7th international conference on Web engineering
A refreshing perspective of search engine caching

Proceedings of the 19th international conference on World wide web
Caching search engine results over incremental indices

Proceedings of the 19th international conference on World wide web
Admission policies for caches of search engine results

SPIRE'07 Proceedings of the 14th international conference on String processing and information retrieval
Mining Query Logs: Turning Search Usage Data into Knowledge

Foundations and Trends in Information Retrieval
Caching search engine results over incremental indices

Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval
New caching techniques for web search engines

Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing
Stochastic query covering

Proceedings of the fourth ACM international conference on Web search and data mining
Caching query-biased snippets for efficient retrieval

Proceedings of the 14th International Conference on Extending Database Technology
Cost-Aware Strategies for Query Result Caching in Web Search Engines

ACM Transactions on the Web (TWEB)
On-line multi-threaded processing of web user-clicks on multi-core processors

VECPAR'10 Proceedings of the 9th international conference on High performance computing for computational science
Caching for realtime search

ECIR'11 Proceedings of the 33rd European conference on Advances in information retrieval
Timestamp-based result cache invalidation for web search engines

Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
Search result caching in peer-to-peer information retrieval networks

IRFC'11 Proceedings of the Second international conference on Multidisciplinary information retrieval facility
Small cache, big effect: provable load balancing for randomly partitioned cluster services

Proceedings of the 2nd ACM Symposium on Cloud Computing
Improving Europeana search experience using query logs

TPDL'11 Proceedings of the 15th international conference on Theory and practice of digital libraries: research and advanced technology for digital libraries
Learning to distribute queries into web search nodes

ECIR'2010 Proceedings of the 32nd European conference on Advances in Information Retrieval
Effective caching of shortest paths for location-based services

SIGMOD '12 Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data
Modeling static caching in web search engines

ECIR'12 Proceedings of the 34th European conference on Advances in Information Retrieval
Similarity caching in large-scale image retrieval

Information Processing and Management: an International Journal
A five-level static cache architecture for web search engines

Information Processing and Management: an International Journal
Prefetching query results and its impact on search engines

SIGIR '12 Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval
Online result cache invalidation for real-time web search

SIGIR '12 Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval
Cache-Based Query Processing for Search Engines

ACM Transactions on the Web (TWEB)
A multi-layer collaborative cache for question answering

Euro-Par'07 Proceedings of the 13th international Euro-Par conference on Parallel Processing
Web usage mining for enhancing search-result delivery and helping users to find interesting web content

Proceedings of the sixth ACM international conference on Web search and data mining
The impact of solid state drive on search engine cache management

Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
A financial cost metric for result caching

Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
Latency-aware strategy for static list caching in flash-based web search engines

Proceedings of the 22nd ACM international conference on Conference on information & knowledge management
Exploiting query term correlation for list caching in web search engines

Proceedings of the 22nd ACM international conference on Conference on information & knowledge management
Second Chance: A Hybrid Approach for Dynamic Result Caching and Prefetching in Search Engines

ACM Transactions on the Web (TWEB)
Improving the efficiency of multi-site web search engines

Proceedings of the 7th ACM international conference on Web search and data mining

Quantified Score

Hi-index	0.24

Visualization

Abstract

In this paper we explore the problem of Caching of Search Engine Query Results in order to reduce the computing and I/O requirements needed to support the functionality of a search engine of the World Wide Web. We study query traces from the EXCITE search engine and show that they have a significant amount of temporal locality that is, a significant percentage of the queries have been submitted more than once by the same or a different user. Using trace-driven simulation we demonstrate that medium-size caches can hold the results of most of the frequently submitted queries. Finally, we compare the effectiveness of static and dynamic caching and conclude that although dynamic caching can use large caches more effectively, static caching can perform better for (very) small caches.