An evaluation of retrieval effectiveness for a full-text document-retrieval system

Authors:
David C. Blair;M. E. Maron
Affiliations:
The Univ. of Michigan, Ann Arbor;The Univ. of California, Berkeley
Venue:
Communications of the ACM
Year:
1985

Citing 0
Cited 147

Another look at automatic text-retrieval systems

Communications of the ACM
Parallel free-text search on the connection machine system

Communications of the ACM - Special issue on parallelism
Towards an intelligent and personalized retrieval system

ISMIS '86 Proceedings of the ACM SIGART international symposium on Methodologies for intelligent systems
The information lens: an intelligent system for information sharing in organizations

CHI '86 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Intelligent information-sharing systems

Communications of the ACM
A large-scale computer conferencing system

IBM Systems Journal
Some observations on retrieval from a large technical document database

ACM SIGIR Forum
Conceptual organization of case law knowledge bases

ICAIL '87 Proceedings of the 1st international conference on Artificial intelligence and law
A connectionist approach to conceptual information retrieval

ICAIL '87 Proceedings of the 1st international conference on Artificial intelligence and law
Parallel text search methods

Communications of the ACM
Comment on some recent comments on information retrieval

ACM SIGIR Forum
Using latent semantic analysis to improve access to textual information

CHI '88 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
How do the experts do it? The use of ethnographic methods as an aid to understanding the cognitive processing and retrieval of large bodies of text

SIGIR '88 Proceedings of the 11th annual international ACM SIGIR conference on Research and development in information retrieval
Access by content of documents in an office information system

SIGIR '88 Proceedings of the 11th annual international ACM SIGIR conference on Research and development in information retrieval
Development of a large, concept-oriented database for information retrieval

SIGIR '88 Proceedings of the 11th annual international ACM SIGIR conference on Research and development in information retrieval
Probabilistic and genetic algorithms in document retrieval

Communications of the ACM
Towards an integrated maintenance advisor

HYPERTEXT '89 Proceedings of the second annual ACM conference on Hypertext
Information retrieval from hypertext: update on the dynamic medical handbook project

HYPERTEXT '89 Proceedings of the second annual ACM conference on Hypertext
Full text indexing based on lexical relations an application: software libraries

SIGIR '89 Proceedings of the 12th annual international ACM SIGIR conference on Research and development in information retrieval
Performance and Architectural Issues for String Matching

IEEE Transactions on Computers
Knowledge and natural language processing

Communications of the ACM
Scaling up output capacity and performance results from information systems prototypes

ACM Transactions on Database Systems (TODS)
SCISOR: extracting information from on-line news

Communications of the ACM
Evaluation of an expert system for searching in full text

SIGIR '90 Proceedings of the 13th annual international ACM SIGIR conference on Research and development in information retrieval
On hypertext

SIGIR '90 Proceedings of the 13th annual international ACM SIGIR conference on Research and development in information retrieval
Analyzing explicitly-structured discourse in a limited domain: trouble and failure reports

HLT '89 Proceedings of the workshop on Speech and Natural Language
Intertwining query construction and relevance evaluation

CHI '91 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Beyond boolean search: FLEXICON, a legal tex-based intelligent system

ICAIL '91 Proceedings of the 3rd international conference on Artificial intelligence and law
Conceptual information retrieval in litigation support systems

ICAIL '91 Proceedings of the 3rd international conference on Artificial intelligence and law
Retrieval performance in Ferret a conceptual information retrieval system

SIGIR '91 Proceedings of the 14th annual international ACM SIGIR conference on Research and development in information retrieval
Search improvement via automatic query reformulation

ACM Transactions on Information Systems (TOIS) - Special issue on research and development in information retrieval
Conceptual Modeling of Multimedia Documents

Computer
An Information Retrieval Approach for Automatically Constructing Software Libraries

IEEE Transactions on Software Engineering
Personalized information delivery: an analysis of information filtering methods

Communications of the ACM - Special issue on information filtering
Towards the integration of a query mechanism and navigation for retrieval of data on multimedia documents

ACM SIGIR Forum
FLEXICON: an evaluation of a statistical ranking model adapted to intelligent legal text management

ICAIL '93 Proceedings of the 4th international conference on Artificial intelligence and law
SIGIR '93: school of library and information science, Univ. of Pittsburgh

ACM SIGIR Forum
Natural language vs. Boolean query evaluation: a comparison of retrieval performance

SIGIR '94 Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval
From text to hypertext by indexing

ACM Transactions on Information Systems (TOIS)
Evaluation of evaluation in information retrieval

SIGIR '95 Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval
A case-based approach to intelligent information retrieval

SIGIR '95 Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval
A hybrid CBR-IR approach to legal information retrieval

ICAIL '95 Proceedings of the 5th international conference on Artificial intelligence and law
Information product evaluation as asynchronous communication in context: a model for organizational research

Proceedings of the first ACM international conference on Digital libraries
Customized information extraction as a basis for resource discovery

ACM Transactions on Computer Systems (TOCS)
Browsing and querying in online documentation: a study of user interfaces and the interaction process

ACM Transactions on Computer-Human Interaction (TOCHI)
Supporting Search for Reusable Software Objects

IEEE Transactions on Software Engineering - Special issue: best papers of the 17th International Conference on Software Engineering (ICSE-17)
An evolutionary approach to constructing effective software reuse repositories

ACM Transactions on Software Engineering and Methodology (TOSEM)
The ARCHIMEDES Network System: a system for searching and accessing information in multiplemultimedia sources

Journal of Intelligent Information Systems
Another nail to the coffin of faceted controlled-vocabulary component classification and retrieval

Proceedings of the 1997 symposium on Software reusability
A first step to formally evaluate collaborative work

GROUP '97 Proceedings of the international ACM SIGGROUP conference on Supporting group work: the integration challenge
The knowledge in multiple human relevance judgments

ACM Transactions on Information Systems (TOIS)
Technical correspondence

Communications of the ACM - Special issue: computing in the frontiers of science and engineering
The architecture of static hypertexts

HYPERTEXT '87 Proceedings of the ACM conference on Hypertext
Intelligent jurisprudence research: a new concept

ICAIL '99 Proceedings of the 7th international conference on Artificial intelligence and law
Generating exception structures for legal information serving

ICAIL '99 Proceedings of the 7th international conference on Artificial intelligence and law
A study of user behavior in an immersive virtual environment for digital libraries

DL '00 Proceedings of the fifth ACM conference on Digital libraries
A novel method for the evaluation of Boolean query effectiveness across a wide operational range

SIGIR '00 Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval
IR evaluation methods for retrieving highly relevant documents

SIGIR '00 Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval
Coplink: a case of intelligent analysis and knowledge management

ICIS '99 Proceedings of the 20th international conference on Information Systems
Liberal relevance criteria of TREC -: counting on negligible documents?

SIGIR '02 Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval
Cumulated gain-based evaluation of IR techniques

ACM Transactions on Information Systems (TOIS)
Networked information management

BT Technology Journal
Measuring Search Engine Quality

Information Retrieval
Information Retrieval Special Issue: Conceptual, Linguistic and Task-Based IR: Research at the University of Tampere

Information Retrieval
Extensions to the STAIRS Study—Empirical Evidence for the Hypothesised Ineffectiveness of Boolean Queries in Large Full-Text Databases

Information Retrieval
Using Coplink to Analyze Criminal-Justice Data

Computer
Reusing Software: Issues and Research Directions

IEEE Transactions on Software Engineering
Problems of music information retrieval in the real world

Information Processing and Management: an International Journal
The challenge of commercial document retrieval, part I: major issues, and a framework based on search exhaustivity, determinacy of representation and document collection size

Information Processing and Management: an International Journal
Exemplary documents: a foundation for information retrieval design

Information Processing and Management: an International Journal
Knowledge management: hype, hope, or help?

Journal of the American Society for Information Science and Technology
Subject Knowledge, Source of Terms, and Term Selection in Query Expansion: An Analytical Study

Proceedings of the 24th BCS-IRSG European Colloquium on IR Research: Advances in Information Retrieval
CONSTRUE/TIS: A System for Content-Based Indexing of a Database of News Stories

IAAI '90 Proceedings of the The Second Conference on Innovative Applications of Artificial Intelligence
Online Dynamic Reordering for Interactive Data Processing

VLDB '99 Proceedings of the 25th International Conference on Very Large Data Bases
The Rufus System: Information Organization for Semi-Structured Data

VLDB '93 Proceedings of the 19th International Conference on Very Large Data Bases
Document Clustering Using the 1 + 1 Dimensional Self-Organising Map

IDEAL '02 Proceedings of the Third International Conference on Intelligent Data Engineering and Automated Learning
Finding Objects by Grouping Primitives

Shape, Contour and Grouping in Computer Vision
Text-Source Discovery and GlOSS Update in a Dynamic Web

PADKK '00 Proceedings of the 4th Pacific-Asia Conference on Knowledge Discovery and Data Mining, Current Issues and New Applications
A Retrospective Evaluation Method for Exact-Match and Best-Match Queries Applying an Interactive Query Performance Analyser

Proceedings of the 24th BCS-IRSG European Colloquium on IR Research: Advances in Information Retrieval
Online dynamic reordering

The VLDB Journal — The International Journal on Very Large Data Bases
Using graded relevance assessments in IR evaluation

Journal of the American Society for Information Science and Technology
User-oriented evaluation methods for information retrieval: a case study based on conceptual models for query expansion

Exploring artificial intelligence in the new millennium
Students' conceptual structure, search process, and outcome while preparing a research proposal: a longitudinal case study

Journal of the American Society for Information Science and Technology
Current Status of the Evaluation of Information Retrieval

Journal of Medical Systems
Documents similarity measurement using field association terms

Information Processing and Management: an International Journal
A comprehensive and systematic model of user evaluation of web search engines: I. theory and background

Journal of the American Society for Information Science and Technology
Building a large thesaurus for information retrieval

ANLC '88 Proceedings of the second conference on Applied natural language processing
Complementing WordNet with Roget's and corpus-based thesauri for information retrieval

EACL '99 Proceedings of the ninth conference on European chapter of the Association for Computational Linguistics
A Linear Least Squares Fit mapping method for information retrieval from natural language texts

COLING '92 Proceedings of the 14th conference on Computational linguistics - Volume 2
Comparison of Two Schemes for Automatic Keyword Extraction from MEDLINE for Functional Gene Clustering

CSB '04 Proceedings of the 2004 IEEE Computational Systems Bioinformatics Conference
The influence of relevance levels on the effectiveness of interactive information retrieval

Journal of the American Society for Information Science and Technology
When will information retrieval be "good enough"?

Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval
Bibliographic database access using free-text and controlled vocabulary: an evaluation

Information Processing and Management: an International Journal
Newsmap: a knowledge map for online news

Decision Support Systems - Special issue: Collaborative work and knowledge management
The data-document distinction revisited

ACM SIGMIS Database
Digital government portal: a tool for digital government research

dg.o '04 Proceedings of the 2004 annual national conference on Digital government research
Evaluating patent retrieval in the third NTCIR workshop

Information Processing and Management: an International Journal - Special issue: Formal methods for information retrieval
A decision-theoretic approach to the evaluation of information retrieval systems

Information Processing and Management: an International Journal
A semantic-based fully visual application for matchmaking and query refinement in B2C e-marketplaces

ICEC '06 Proceedings of the 8th international conference on Electronic commerce: The new e-commerce: innovations for conquering current barriers, obstacles and limitations to conducting successful business on the internet
Software reuse: survey and research directions

Journal of Management Information Systems - Special section: Managing virtual workplaces and teleworking with information technology
GANNET: a machine learning approach to document retrieval

Journal of Management Information Systems - Special section: Information technology and IT organizational impact
Component-based visual clustering using the self-organizing map

Neural Networks
CMedPort: an integrated approach to facilitating Chinese medical information seeking

Decision Support Systems
Automated criminal link analysis based on domain knowledge: Research Articles

Journal of the American Society for Information Science and Technology
The search problem posed by large heterogeneous data sets in litigation: possible future approaches to research

Proceedings of the 11th international conference on Artificial intelligence and law
Kascade: a new Open directory way of access to Internet information

Information Services and Use
NetBook: a data model to support knowledge exploration

VLDB '85 Proceedings of the 11th international conference on Very Large Data Bases - Volume 11
Answering form-based web queries using the data-mining approach

Journal of Intelligent Information Systems
Evaluating epistemic uncertainty under incomplete assessments

Information Processing and Management: an International Journal
Looking back: On relevance, probabilistic indexing and information retrieval

Information Processing and Management: an International Journal
Effects of performance feedback on users' evaluations of an interactive IR system

Proceedings of the second international symposium on Information interaction in context
Toward automatic facet analysis and need negotiation: Lessons from mediated search

ACM Transactions on Information Systems (TOIS)
Concept and Context in Legal Information Retrieval

Proceedings of the 2008 conference on Legal Knowledge and Information Systems: JURIX 2008: The Twenty-First Annual Conference
Users' stopping behaviors and estimates of recall

Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval
Incorporating syntactic dependency information towards improved coding of lengthy medical concepts in clinical reports

BioNLP '09 Proceedings of the Workshop on Current Trends in Biomedical Natural Language Processing
Using CBR to drive IR

IJCAI'95 Proceedings of the 14th international joint conference on Artificial intelligence - Volume 1
Document categorization in legal electronic discovery: computer classification vs. manual review

Journal of the American Society for Information Science and Technology
A study on the semantic relatedness of query and document terms in information retrieval

EMNLP '09 Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 3 - Volume 3
Evaluating patent retrieval in the third NTCIR workshop

Information Processing and Management: an International Journal - Special issue: Formal methods for information retrieval
Adapting to the data explosion: ensuring justice for all

SMC'09 Proceedings of the 2009 IEEE international conference on Systems, Man and Cybernetics
Impedance matching of humans * machines in high-Q information retrieval systems

SMC'09 Proceedings of the 2009 IEEE international conference on Systems, Man and Cybernetics
From Keyword Search to Exploration: Designing Future Search Interfaces for the Web

Foundations and Trends in Web Science
Effects of position and number of relevant documents retrieved on users' evaluations of system performance

ACM Transactions on Information Systems (TOIS)
Towards effective genomic information retrieval: The impact of query complexity and expansion strategies

Journal of Information Science
Scatter matters: Regularities and implications for the scatter of healthcare information on the Web

Journal of the American Society for Information Science and Technology
Agent-based semantic search at motoso.de

MATES'09 Proceedings of the 7th German conference on Multiagent system technologies
CrimeLink explorer: using domain knowledge to facilitate automated crime association analysis

ISI'03 Proceedings of the 1st NSF/NIJ conference on Intelligence and security informatics
Massively parallel AI

AAAI'90 Proceedings of the eighth National conference on Artificial intelligence - Volume 2
E-discovery revisited: the need for artificial intelligence beyond information retrieval

Artificial Intelligence and Law
Evaluation of information retrieval for E-discovery

Artificial Intelligence and Law
Automation of legal sensemaking in e-discovery

Artificial Intelligence and Law
A new tangible user interface for machine learning document review

Artificial Intelligence and Law
Afterword: data, knowledge, and e-discovery

Artificial Intelligence and Law
A novel hash-based streaming scheme for energy efficient full-text search in wireless data broadcast

DASFAA'11 Proceedings of the 16th international conference on Database systems for advanced applications - Volume Part I
User behavior in zero-recall ecommerce queries

Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
Query suggestions in the absence of query logs

Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
Improving system performance for stochastic activity network: A simulation approach

Computers and Industrial Engineering
Applying VSM and LCS to develop an integrated text retrieval mechanism

Expert Systems with Applications: An International Journal
Guided interactive information access for e-citizens

EGOV'05 Proceedings of the 4th international conference on Electronic Government
Keeping Found Things Found: The Study and Practice of Personal Information Management: The Study and Practice of Personal Information Management

Keeping Found Things Found: The Study and Practice of Personal Information Management: The Study and Practice of Personal Information Management
Semantic indexing of legal documents

Semantic Processing of Legal Texts
No (e-)democracy without (e-)knowledge

TCGOV'05 Proceedings of the 2005 international conference on E-Government: towards Electronic Democracy
The elusive tale: leveraging the study of information seeking and knowledge organization to improve access to and discovery of folktales

Journal of the American Society for Information Science and Technology
Relevance: An improved framework for explicating the notion

Journal of the American Society for Information Science and Technology
A model for automated rating of case law

Proceedings of the Fourteenth International Conference on Artificial Intelligence and Law
Explicitly integrating MeSH thesaurus help into health information retrieval systems: An empirical user study

Information Processing and Management: an International Journal
Knowledge-based approaches in software documentation: A systematic literature review

Information and Software Technology

Quantified Score

Hi-index	48.32

Visualization

Abstract

An evaluation of a large, operational full-text document-retrieval system (containing roughly 350,000 pages of text) shows the system to be retrieving less than 20 percent of the documents relevant to a particular search. The findings are discussed in terms of the theory and practice of full-text document retrieval.