IR evaluation methods for retrieving highly relevant documents
SIGIR '00 Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval
Ranking retrieval systems without relevance judgments
Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval
The costs and limits of availability for replicated services
SOSP '01 Proceedings of the eighteenth ACM symposium on Operating systems principles
Minimal replication cost for availability
Proceedings of the twenty-first annual symposium on Principles of distributed computing
Cumulated gain-based evaluation of IR techniques
ACM Transactions on Information Systems (TOIS)
Mining the Web for Synonyms: PMI-IR versus LSA on TOEFL
EMCL '01 Proceedings of the 12th European Conference on Machine Learning
WWW '03 Proceedings of the 12th international conference on World Wide Web
Journal of the American Society for Information Science and Technology
Using the web to obtain frequencies for unseen bigrams
Computational Linguistics - Special issue on web as corpus
Web-based models for natural language processing
ACM Transactions on Speech and Language Processing (TSLP)
Software Testing and Analysis: Process, Principles and Techniques
Software Testing and Analysis: Process, Principles and Techniques
Minimal test collections for retrieval evaluation
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
Dynamic test collections: measuring search effectiveness on the live web
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
Bias and the limits of pooling
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
Automatic extraction of dynamic record sections from search engine result pages
VLDB '06 Proceedings of the 32nd international conference on Very large data bases
Computational Linguistics
Introduction to Software Testing
Introduction to Software Testing
Quality Information Retrieval for the World Wide Web
WI-IAT '08 Proceedings of the 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology - Volume 01
Ranking Web Pages Using Machine Learning Approaches
WI-IAT '08 Proceedings of the 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology - Volume 03
Search engine statistics beyond the n-gram: application to noun compound bracketing
CONLL '05 Proceedings of the Ninth Conference on Computational Natural Language Learning
Semi-Proving: An Integrated Method for Program Proving, Testing, and Debugging
IEEE Transactions on Software Engineering
Testing a binary space partitioning algorithm with metamorphic testing
Proceedings of the 2011 ACM Symposium on Applied Computing
Hi-index | 0.00 |
Search services are the main interface through which people discover information on the Internet. A fundamental challenge in testing search services is the lack of oracles. The sheer volume of data on the Internet prohibits testers from verifying the results. Furthermore, it is difficult to objectively assess the ranking quality because different assessors can have very different opinions on the relevance of a Web page to a query. This paper presents a novel method for automatically testing search services without the need of a human oracle. The experimental findings reveal that some commonly used search engines, including Google, Yahoo!, and Live Search, are not as reliable as what most users would expect. For example, they may fail to find pages that exist in their own repositories, or rank pages in a way that is logically inconsistent. Suggestions are made for search service providers to improve their service quality. Copyright © 2010 John Wiley & Sons, Ltd.