Evaluation of the user interface in an information retrieval system: a model
Information Processing and Management: an International Journal
Information Processing and Management: an International Journal
Language and representation in information retrieval
Language and representation in information retrieval
A re-examination of relevance: toward a dynamic, situational definition
Information Processing and Management: an International Journal
Information retrieval interaction
Information retrieval interaction
Variations in relevance judgments and the evaluation of retrieval performance
Information Processing and Management: an International Journal
On the evaluation of IR systems
Information Processing and Management: an International Journal - Special issue on evaluation issues in information retrieval
Information retrieval evaluation in practice: a case study approach
Information Processing and Management: an International Journal
Toward a theory of user-based relevance: a call for a new paradigm of inquiry
Journal of the American Society for Information Science - Special issue: relevance research
The relevance of recall and precision in user evaluation
Journal of the American Society for Information Science - Special issue: relevance research
OHSUMED: an interactive retrieval evaluation and new large test collection for research
SIGIR '94 Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval
Information seeking in electronic environments
Information seeking in electronic environments
Task complexity affects information seeking and use
Information Processing and Management: an International Journal
Variations in relevance assessments and the measurement of retrieval effectiveness
Journal of the American Society for Information Science - Special issue: evaluation of information retrieval systems
Evaluating interactive systems in TREC
Journal of the American Society for Information Science - Special issue: evaluation of information retrieval systems
Time, relevance and interaction modelling for information retrieval
Proceedings of the 20th annual international ACM SIGIR conference on Research and development in information retrieval
Journal of the American Society for Information Science - Special topic issue on the history of documentation and information science: part II
Logical models in information retrieval: introduction and overview
Information Processing and Management: an International Journal
Users' criteria for relevance evaluation: a cross-situational comparison
Information Processing and Management: an International Journal
Information Processing and Management: an International Journal
Variations in relevance judgments and the measurement of retrieval effectiveness
Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval
Task Analysis for Human-Computer Interaction
Task Analysis for Human-Computer Interaction
Information Retrieval Experiment
Information Retrieval Experiment
Information Retrieval
Human-Computer Interaction
Getting into information retrieval
Lectures on information retrieval
Evaluation of Text Retrieval Systems
Programming and Computing Software
Information Processing and Management: an International Journal
Getting into Information Retrieval
ESSIR '00 Proceedings of the Third European Summer-School on Lectures on Information Retrieval-Revised Lectures
How well does result relevance predict session satisfaction?
SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
Evaluation of a mobile information system in context
Information Processing and Management: an International Journal
Journal of the American Society for Information Science and Technology
A faceted approach to conceptualizing tasks in information seeking
Information Processing and Management: an International Journal
How does clickthrough data reflect retrieval quality?
Proceedings of the 17th ACM conference on Information and knowledge management
AMT '09 Proceedings of the 5th International Conference on Active Media Technology
Factors affecting click-through behavior in aggregated search interfaces
CIKM '10 Proceedings of the 19th ACM international conference on Information and knowledge management
A cross-domain analysis of task and genre effects on perceptions of usefulness
Information Processing and Management: an International Journal
| Hi-index | 0.00 |
Past research has identified many different types ofrelevance in information retrieval (IR). So far, however, mostevaluation of IR systems has been through batch experiments conductedwith test collections containing only expert, topical relevancejudgements. Recently, there has been some movement away from thistraditional approach towards interactive, more user-centred methodsof evaluation. However, these are expensive for evaluators in termsboth of time and of resources. This paper describes a new evaluationmethodology, using a task-oriented test collection, which combinesthe advantages of traditional non-interactive testing with a moreuser-centred emphasis. The main features of a task-oriented testcollection are the adoption of the task, rather than the query, asthe primary unit of evaluation and the naturalistic character of therelevance judgements.