Domain-independent automatic keyphrase indexing with small training sets

  • Authors:

  • Olena Medelyan;Ian H. Witten

  • Affiliations:

  • Department of Computer Science, University of Waikato, Private Bag 3105, Hamilton 3240, New Zealand;Department of Computer Science, University of Waikato, Private Bag 3105, Hamilton 3240, New Zealand

  • Venue:
  • Journal of the American Society for Information Science and Technology
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

Keyphrases are widely used in both physical and digital libraries as a brief, but precise, summary of documents. They help organize material based on content, provide thematic access, represent search results, and assist with navigation. Manual assignment is expensive because trained human indexers must reach an understanding of the document and select appropriate descriptors according to defined cataloging rules. We propose a new method that enhances automatic keyphrase extraction by using semantic information about terms and phrases gleaned from a domain-specific thesaurus. The key advantage of the new approach is that it performs well with very little training data. We evaluate it on a large set of manually indexed documents in the domain of agriculture, compare its consistency with a group of six professional indexers, and explore its performance on smaller collections of documents in other domains and of French and Spanish documents. © 2008 Wiley Periodicals, Inc.