Automatic text processing: the transformation, analysis, and retrieval of information by computer
Automatic text processing: the transformation, analysis, and retrieval of information by computer
An analysis of Internet search engines: assessment of over 200 search queries
Computers in Libraries
Integrating structured data and text: a relational approach
Journal of the American Society for Information Science
First 20 precision among World Wide Web search services (search engines)
Journal of the American Society for Information Science
GlOSS: text-source discovery over the Internet
ACM Transactions on Database Systems (TODS)
Genetics-Based Learning of New Heuristics: Rational Scheduling of Experiments and Generalization
IEEE Transactions on Knowledge and Data Engineering
Improvement of HITS-based algorithms on web documents
Proceedings of the 11th international conference on World Wide Web
Precision Evaluation of Search Engines
World Wide Web
Using titles and category names from editor-driven taxonomies for automatic evaluation
CIKM '03 Proceedings of the twelfth international conference on Information and knowledge management
A subjective measure of web search quality
Information Sciences—Informatics and Computer Science: An International Journal
A comprehensive model for web search evaluation
CSECS'06 Proceedings of the 5th WSEAS International Conference on Circuits, Systems, Electronics, Control & Signal Processing
Web search solved?: all result rankings the same?
CIKM '10 Proceedings of the 19th ACM international conference on Information and knowledge management
| Hi-index | 0.00 |
In this paper, we present a new method for automatically comparing the performance, such as precision, of search engines. Based on queries randomly selected from a specific domain of interest, the method uses robots to automatically query the target search engines, evaluates the relevance of the returned links to the query either automatically based on the vector space model or manually, and then applies statistic measures, including the probability of win and the Friedman statistic, to compare the performance of search engines. We show the experimental results of the new method on three search engines, AltaVista, Google, and InfoSeek. The method arrived at the same performance comparison result in applying either the automatic relevance evaluation method or the manual method. In addition, our results show that the probability of win is a better metric than the Friedman statistic in performance comparison. The advantage of the new method is that it is fast, flexible, consistent, and can adapt to the fast changing search engines.