The significance of the Cranfield tests on index languages
SIGIR '91 Proceedings of the 14th annual international ACM SIGIR conference on Research and development in information retrieval
Query expansion using lexical-semantic relations
SIGIR '94 Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval
Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval
SIGIR '94 Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval
Variations in relevance assessments and the measurement of retrieval effectiveness
Journal of the American Society for Information Science - Special issue: evaluation of information retrieval systems
A vector space model for automatic indexing
Communications of the ACM
Model-based feedback in the language modeling approach to information retrieval
Proceedings of the tenth international conference on Information and knowledge management
The Philosophy of Information Retrieval Evaluation
CLEF '01 Revised Papers from the Second Workshop of the Cross-Language Evaluation Forum on Evaluation of Cross-Language Information Retrieval Systems
WWW '03 Proceedings of the 12th international conference on World Wide Web
A study of smoothing methods for language models applied to information retrieval
ACM Transactions on Information Systems (TOIS)
An exploration of axiomatic approaches to information retrieval
Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval
TREC: Experiment and Evaluation in Information Retrieval (Digital Libraries and Electronic Publishing)
Semantic search via XML fragments: a high-precision approach to IR
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
An Adaptation of the Vector-Space Model for Ontology-Based Information Retrieval
IEEE Transactions on Knowledge and Data Engineering
Proceedings of the sixteenth ACM conference on Conference on information and knowledge management
Relevance assessment: are judges exchangeable and does it matter
Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval
Introduction to Information Retrieval
Introduction to Information Retrieval
Sindice.com: a document-oriented lookup index for open linked data
International Journal of Metadata, Semantics and Ontologies
Crowdsourcing for relevance evaluation
ACM SIGIR Forum
SPARK: A Keyword Search Engine on Relational Databases
ICDE '08 Proceedings of the 2008 IEEE 24th International Conference on Data Engineering
Financial incentives and the "performance of crowds"
Proceedings of the ACM SIGKDD Workshop on Human Computation
Overview of the INEX 2008 Ad Hoc Track
Advances in Focused Retrieval
Hermes: Data Web search on a pay-as-you-go integration infrastructure
Web Semantics: Science, Services and Agents on the World Wide Web
Language-model-based ranking for queries on RDF-graphs
Proceedings of the 18th ACM conference on Information and knowledge management
Fast, cheap, and creative: evaluating translation quality using Amazon's Mechanical Turk
EMNLP '09 Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 1 - Volume 1
The Probabilistic Relevance Framework: BM25 and Beyond
Foundations and Trends in Information Retrieval
Proceedings of the international conference on Multimedia information retrieval
Ad-hoc object retrieval in the web of data
Proceedings of the 19th international conference on World wide web
Computing information retrieval performance measures efficiently in the presence of tied scores
ECIR'08 Proceedings of the IR research, 30th European conference on Advances in information retrieval
A framework for evaluating database keyword search strategies
CIKM '10 Proceedings of the 19th ACM international conference on Information and knowledge management
Overview of the INEX 2009 entity ranking track
INEX'09 Proceedings of the Focused retrieval and evaluation, and 8th international conference on Initiative for the evaluation of XML retrieval
Editorial: The Semantic Web Challenge, 2009
Web Semantics: Science, Services and Agents on the World Wide Web
Repeatable and reliable search system evaluation using crowdsourcing
Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
Effective and efficient entity search in RDF data
ISWC'11 Proceedings of the 10th international conference on The semantic web - Volume Part I
SemSearchPro - Using semantics throughout the search process
Web Semantics: Science, Services and Agents on the World Wide Web
Crowdsourcing assessments for XML ranked retrieval
ECIR'2010 Proceedings of the 32nd European conference on Advances in Information Retrieval
Heterogeneous web data search using relevance-based on the fly data integration
Proceedings of the 21st international conference on World Wide Web
| Hi-index | 0.00 |
An increasing amount of structured data on the Web has attracted industry attention and renewed research interest in what is collectively referred to as semantic search. These solutions exploit the explicit semantics captured in structured data such as RDF for enhancing document representation and retrieval, or for finding answers by directly searching over the data. These data have been used for different tasks and a wide range of corresponding semantic search solutions have been proposed in the past. However, it has been widely recognized that a standardized setting to evaluate and analyze the current state-of-the-art in semantic search is needed to monitor and stimulate further progress in the field. In this paper, we present an evaluation framework for semantic search, analyze the framework with regard to repeatability and reliability, and report on our experiences on applying it in the Semantic Search Challenge 2010 and 2011.