A critical investigation of recall and precision as measures of retrieval system performance
ACM Transactions on Information Systems (TOIS)
Approaches to passage retrieval in full text information systems
SIGIR '93 Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval
Time, relevance and interaction modelling for information retrieval
Proceedings of the 20th annual international ACM SIGIR conference on Research and development in information retrieval
Evaluating evaluation measure stability
SIGIR '00 Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval
Cumulated gain-based evaluation of IR techniques
ACM Transactions on Information Systems (TOIS)
Two axioms for evaluation measures in information retrieval
SIGIR '84 Proceedings of the 7th annual international ACM SIGIR conference on Research and development in information retrieval
Focussed Structured Document Retrieval
SPIRE 2002 Proceedings of the 9th International Symposium on String Processing and Information Retrieval
WWW '03 Proceedings of the 12th international conference on World Wide Web
The overlap problem in content-oriented XML retrieval evaluation
Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval
Evaluating implicit measures to improve web search
ACM Transactions on Information Systems (TOIS)
Ranking Complex Relationships on the Semantic Web
IEEE Internet Computing
Redundant documents and search effectiveness
Proceedings of the 14th ACM international conference on Information and knowledge management
Learning user interaction models for predicting web search result preferences
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
Strict and vague interpretation of XML-retrieval queries
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
XQuery full-text extensions explained
IBM Systems Journal
IIiX Proceedings of the 1st international conference on Information interaction in context
eXtended cumulated gain measures for the evaluation of content-oriented XML retrieval
ACM Transactions on Information Systems (TOIS)
Precision recall with user modeling (PRUM): Application to structured information retrieval
ACM Transactions on Information Systems (TOIS)
Evaluating relevant in context: document retrieval with a twist
SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
Sound and complete relevance assessment for XML retrieval
ACM Transactions on Information Systems (TOIS)
Focused Access to XML Documents
Structural relevance: a common basis for the evaluation of structured document retrieval
Proceedings of the 17th ACM conference on Information and knowledge management
Proceedings of the Second ACM International Conference on Web Search and Data Mining
Representing User Navigation in XML Retrieval with Structural Summaries
ECIR '09 Proceedings of the 31th European Conference on IR Research on Advances in Information Retrieval
AxPRE Summaries: Exploring the (Semi-)Structure of XML Web Collections
ICDE '08 Proceedings of the 2008 IEEE 24th International Conference on Data Engineering
ACM SIGMOD Record
INEX'05 Proceedings of the 4th international conference on Initiative for the Evaluation of XML Retrieval
HiXEval: highlighting XML retrieval evaluation
INEX'05 Proceedings of the 4th international conference on Initiative for the Evaluation of XML Retrieval
Narrowed extended XPath i (NEXI)
INEX'04 Proceedings of the Third international conference on Initiative for the Evaluation of XML Retrieval
Reliability tests for the XCG and inex-2002 metrics
INEX'04 Proceedings of the Third international conference on Initiative for the Evaluation of XML Retrieval
Mixture models, overlap, and structural hints in XML element retrieval
INEX'04 Proceedings of the Third international conference on Initiative for the Evaluation of XML Retrieval
Effects of Terms Recognition Mistakes on Requests Processing for Interactive Information Retrieval
International Journal of Information Retrieval Research
Hi-index | 0.00 |
A structured document retrieval (SDR) system aims to minimize the effort users spend to locate relevant information by retrieving parts of documents. To evaluate the range of SDR tasks, from element to passage to tree retrieval, numerous task-specific measures have been proposed. This has resulted in SDR evaluation measures that cannot easily be compared with respect to each other and across tasks. In previous work, we defined the SDR task of tree retrieval where passage and element are special cases. In this paper, we look in greater detail into tree retrieval to identify the main components of SDR evaluation: relevance, navigation, and redundancy. Our goal is to evaluate SDR within a single probabilistic framework based on these components. This framework, called Extended Structural Relevance (ESR), calculates user expected gain in relevant information depending on whether it is seen via hits (relevant results retrieved), unseen via misses (relevant results not retrieved), or possibly seen via near-misses (relevant results accessed via navigation). We use these expectations as parameters to formulate evaluation measures for tree retrieval. We then demonstrate how existing task-specific measures, if viewed as tree retrieval, can be formulated, computed and compared using our framework. Finally, we experimentally validate ESR across a range of SDR tasks.