Fashionable trends and feasible strategies in information management
Information Processing and Management: an International Journal
A re-examination of relevance: toward a dynamic, situational definition
Information Processing and Management: an International Journal
Introduction to information storage and retrieval systems
Information retrieval
Relevance feedback and other query modification techniques
Information retrieval
The state of retrieval system evaluation
Information Processing and Management: an International Journal - Special issue on evaluation issues in information retrieval
The pragmatics of information retrieval experimentation, revisited
Information Processing and Management: an International Journal - Special issue on evaluation issues in information retrieval
Information Processing and Management: an International Journal
The relevance of recall and precision in user evaluation
Journal of the American Society for Information Science - Special issue: relevance research
Overview of the second text retrieval conference (TREC-2)
TREC-2 Proceedings of the second conference on Text retrieval conference
TREC-2 Proceedings of the second conference on Text retrieval conference
Relevance judgments for assessing recall
Information Processing and Management: an International Journal
Information behaviour: an interdisciplinary perspective
Information Processing and Management: an International Journal
Automated information retrieval: theory and methods
Automated information retrieval: theory and methods
Information and Information Systems
Information and Information Systems
Information Retrieval Experiment
Information Retrieval Experiment
Information Retrieval
Getting into information retrieval
Lectures on information retrieval
Getting into Information Retrieval
ESSIR '00 Proceedings of the Third European Summer-School on Lectures on Information Retrieval-Revised Lectures
| Hi-index | 0.00 |
The contrivances of 'Recall' and 'Precision' are customarily used to assess the effectiveness of document retrieval systems. Despite their extensive use in experiments, including the recent TREC experiments, and their dominance in mathematical discussions of system performance, there has been continual questioning of their validity since their introduction with the Cranfield experiments. This has largely centred on critical analysis of the (pre-mathematical) concept of 'relevance'. Those analyses have now led to the near-consensual view amongst relevance theoreticians that two types of relevance require to be distinguished, namely (1) document 'topicality', where cognition acts as a largely passive receiving agent of knowledge that has an objective, a priori or public character, and (2) 'psychological relevance', where cognition is more actively and creatively involved in the knowledge encoded in the document, i.e. where relevance is non-public (subjective) and conditioned by the user's context and experience (etc) at a particular time. (Various synonyms for these terms have been suggested.) The continued use of P and R in their initial, largely Cranfield, form in document retrieval experiments, especially the continued, uncritical, use of 'Recall', suggest that experimentalists have largely failed to internalise this distinction, since (1) Recall is meaningless under one of these viewpoints, and (2) Precision is ambiguous when the separate validities of each are recognised. Several ways in which this distinction should influence the design of future experiments on document-retrieval/cognition interactions are suggested, involving the choice of, and amendments to the definitions of, these basic measures. Lastly, a generic 3-valued vectorial approach is suggested as a means of integrating both perspectives on 'relevance' within a common evaluative framework.