Applying Machine Learning to Subject Classification and Subject Description for Information Retrieval

  • Authors:
  • Sally Jo Cunningham;Brent Summers

  • Affiliations:
  • -;-

  • Venue:
  • ANNES '95 Proceedings of the 2nd New Zealand Two-Stream International Conference on Artificial Neural Networks and Expert Systems
  • Year:
  • 1995

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper describes an experiment in applying a standard supervised machine learning algorithm (C4.5) to the problem of developing subject classification rules for documents. This algorithms is found to produce surprisingly concise models of document classifications. While the models are highly accurate on the training sets, evaluation over test sets or through cross-validation shows a significant decrease in classification accuracy. Given the difficult nature of the experimental task, however, the results of this investigation are promising and merit further study. An additional algorithm, 1R, is shown to be highly effective in generating lists of candidate terms for subject descriptions.