A re-examination of relevance: toward a dynamic, situational definition
Information Processing and Management: an International Journal
User-defined relevance criteria: an exploratory study
Journal of the American Society for Information Science - Special issue: relevance research
Journal of the American Society for Information Science - Special topic issue on the history of documentation and information science: part II
A cognitive model of document use during a research project. Study I. document selection
Journal of the American Society for Information Science
Children's Internet searching on complex problems: performance and process analyses
Journal of the American Society for Information Science - Special issue on user-centered cooperative systems
First 20 precision among World Wide Web search services (search engines)
Journal of the American Society for Information Science
Children's relevance criteria and information seeking on electronic resources
Journal of the American Society for Information Science
Evaluating evaluation measure stability
SIGIR '00 Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval
Information Processing and Management: an International Journal
Journal of the American Society for Information Science and Technology
Regions and levels: measuring and mapping users' relevance judgments
Journal of the American Society for Information Science and Technology
Searching the Web: the public and their queries
Journal of the American Society for Information Science and Technology
Journal of the American Society for Information Science
Differences and similarities in information seeking: children and adults as web users
Information Processing and Management: an International Journal - Issues of context in information retrieval
Information Processing and Management: an International Journal
Journal of the American Society for Information Science and Technology
The concept of relevance in IR
Journal of the American Society for Information Science and Technology
New measurements for search engine evaluation proposed and tested
Information Processing and Management: an International Journal
The Turn: Integration of Information Seeking and Retrieval in Context (The Information Retrieval Series)
A study of results overlap and uniqueness among major web search engines
Information Processing and Management: an International Journal
Web Search: Public Searching of the Web (Information Science and Knowledge Management)
Web Search: Public Searching of the Web (Information Science and Knowledge Management)
On the reliability of information retrieval metrics based on graded relevance
Information Processing and Management: an International Journal - Special issue: AIRS2005: Information retrieval research in Asia
Information Processing and Management: an International Journal
Journal of the American Society for Information Science and Technology
Precision-at-ten considered redundant
Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval
Quantitative comparisons of search engine results
Journal of the American Society for Information Science and Technology
Searchers' relevance judgments and criteria in evaluating web pages in a learning style perspective
Proceedings of the second international symposium on Information interaction in context
How children search the internet with keyword interfaces
Proceedings of the 8th International Conference on Interaction Design and Children
Explaining User Performance in Information Retrieval: Challenges to IR Evaluation
ICTIR '09 Proceedings of the 2nd International Conference on Theory of Information Retrieval: Advances in Information Retrieval Theory
The foundation of the concept of relevance
Journal of the American Society for Information Science and Technology
Children's roles using keyword search interfaces at home
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Evaluating leading web search engines on children's queries
HCII'11 Proceedings of the 14th international conference on Human-computer interaction: users and applications - Volume Part IV
Following the signs: children's use of visual cues to facilitate website evaluation
HCII'11 Proceedings of the 14th international conference on Human-computer interaction: users and applications - Volume Part IV
The Filter Bubble: What the Internet Is Hiding from You
The Filter Bubble: What the Internet Is Hiding from You
Hi-index | 0.00 |
This study employed benchmarking and intellectual relevance judgment in evaluating Google, Yahoo!, Bing, Yahoo! Kids, and Ask Kids on 30 queries that children formulated to find information for specific tasks. Retrieved hits on given queries were benchmarked to Google's and Yahoo! Kids' top-five ranked hits retrieved. Relevancy of hits was judged on a graded scale; precision was calculated using the precision-at-ten metric (P@10). Yahoo! and Bing produced a similar percentage in hit overlap with Google (nearly 30%), but differed in the ranking of hits. Ask Kids retrieved 11% in hit overlap with Google versus 3% by Yahoo! Kids. The engines retrieved 26 hits across query clusters that overlapped with Yahoo! Kids' top-five ranked hits. Precision (P) that the engines produced across the queries was P = 0.48 for relevant hits, and P = 0.28 for partially relevant hits. Precision by Ask Kids was P = 0.44 for relevant hits versus P = 0.21 by Yahoo! Kids. Bing produced the highest total precision (TP) of relevant hits (TP = 0.86) across the queries, and Yahoo! Kids yielded the lowest (TP = 0.47). Average precision (AP) of relevant hits was AP = 0.56 by leading engines versus AP = 0.29 by small engines. In contrast, average precision of partially relevant hits was AP = 0.83 by small engines versus AP = 0.33 by leading engines. Average precision of relevant hits across the engines was highest on two-word queries and lowest on one-word queries. Google performed best on natural language queries; Bing did the same (P = 0.69) on two-word queries. The findings have implications for search engine ranking algorithms, relevance theory, search engine design, research design, and information literacy. © 2012 Wiley Periodicals, Inc.