TopCat: Data Mining for Topic Identification in a Text Corpus

  • Authors:
  • Chris Clifton;Robert Cooley;Jason Rennie

  • Affiliations:
  • IEEE;IEEE;-

  • Venue:
  • IEEE Transactions on Knowledge and Data Engineering
  • Year:
  • 2004

Quantified Score

Hi-index 0.01

Visualization

Abstract

TopCat (Topic Categories) is a technique for identifying topics that recur in articles in a text corpus. Natural language processing techniques are used to identify key entities in individual articles, allowing us to represent an article as a set of items. This allows us to view the problem in a database/data mining context: Identifying related groups of items. This paper presents a novel method for identifying related items based on traditional data mining techniques. Frequent itemsets are generated from the groups of items, followed by clusters formed with a hypergraph partitioning scheme. We present an evaluation against a manually categorized ground truth news corpus; it shows this technique is effective in identifying topics in collections of news articles.