An Evaluation of Passage-Based Text Categorization

  • Authors:
  • Jinsuk Kim;Myoung Ho Kim

  • Affiliations:
  • Center for Computational Biology & Bioinformatics, Korea Institute of Science and Technology Information, P.O. Box 122, Yuseong-gu, Daejeon, Republic of Korea 305-600. jinsuk@kisti.re. ...;Department of Electrical Engineering & Computer Science, Korea Advanced Institute of Science and Technology, 373-1, Guseong-dong, Yuseong-gu, Daejeon, Republic of Korea 305-701. mhkim@ ...

  • Venue:
  • Journal of Intelligent Information Systems
  • Year:
  • 2004

Quantified Score

Hi-index 0.00

Visualization

Abstract

Researches in text categorization have been confined to whole-document-level classification, probably due to lack of full-text test collections. However, full-length documents available today in large quantities pose renewed interests in text classification. A document is usually written in an organized structure to present its main topic(s). This structure can be expressed as a sequence of subtopic text blocks, or passages. In order to reflect the subtopic structure of a document, we propose a new passage-level or passage-based text categorization model, which segments a test document into several passages, assigns categories to each passage, and merges the passage categories to the document categories. Compared with traditional document-level categorization, two additional steps, passage splitting and category merging, are required in this model. Using four subsets of the Reuters text categorization test collection and a full-text test collection of which documents are varying from tens of kilobytes to hundreds, we evaluate the proposed model, especially the effectiveness of various passage types and the importance of passage location in category merging. Our results show simple windows are best for all test collections tested in these experiments. We also found that passages have different degrees of contribution to the main topic(s), depending on their location in the test document.