Mining test oracles of web search engines

Authors:
Wujie Zheng;Hao Ma;Michael R. Lyu;Tao Xie;Irwin King
Affiliations:
Computer Science and Engineering, The Chinese University of Hong Kong, China;Internet Services Research Center (ISRC), Microsoft Research, Redmond, USA;Computer Science and Engineering, The Chinese University of Hong Kong, China;Department of Computer Science, North Carolina State University, USA;Computer Science and Engineering, The Chinese University of Hong Kong, China
Venue:
ASE '11 Proceedings of the 2011 26th IEEE/ACM International Conference on Automated Software Engineering
Year:
2011

Citing 7
Cited 0

An Approach to Program Testing

ACM Computing Surveys (CSUR)
Dynamically Discovering Likely Program Invariants to Support Program Evolution

IEEE Transactions on Software Engineering - Special issue on 1999 international conference on software engineering
Finding failures by cluster analysis of execution profiles

ICSE '01 Proceedings of the 23rd International Conference on Software Engineering
Mining specifications

POPL '02 Proceedings of the 29th ACM SIGPLAN-SIGACT symposium on Principles of programming languages
Fast Algorithms for Mining Association Rules in Large Databases

VLDB '94 Proceedings of the 20th International Conference on Very Large Data Bases
KDD CUP-2005 report: facing a great challenge

ACM SIGKDD Explorations Newsletter
Beyond position bias: examining result attractiveness as a source of presentation bias in clickthrough data

Proceedings of the 19th international conference on World wide web

Quantified Score

Hi-index	0.00

Visualization

Abstract

Web search engines have major impact in people's everyday life. It is of great importance to test the retrieval effectiveness of search engines. However, it is labor-intensive to judge the relevance of search results for a large number of queries, and these relevance judgments may not be reusable since the Web data change all the time. In this work, we propose to mine test oracles of Web search engines from existing search results. The main idea is to mine implicit relationships between queries and search results, e.g., some queries may have fixed top 1 result while some may not, and some Web domains may appear together in top 10 results. We define a set of items of queries and search results, and mine frequent association rules between these items as test oracles. Experiments on major search engines show that our approach mines many high-confidence rules that help understand search engines and detect suspicious search results.