Frequent itemset based hierarchical document clustering using Wikipedia as external knowledge

  • Authors:
  • G. V. R. Kiran;Ravi Shankar;Vikram Pudi

  • Affiliations:
  • International Institute of Information Technology, Hyderabad;International Institute of Information Technology, Hyderabad;International Institute of Information Technology, Hyderabad

  • Venue:
  • KES'10 Proceedings of the 14th international conference on Knowledge-based and intelligent information and engineering systems: Part II
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

High dimensionality is a major challenge in document clustering. Some of the recent algorithms address this problem by using frequent itemsets for clustering. But, most of these algorithms neglect the semantic relationship between the words. On the other hand there are algorithms that take care of the semantic relations between the words by making use of external knowledge contained in Word Net, Mesh, Wikipedia, etc but do not handle the high dimensionality. In this paper we present an efficient solution that addresses both these problems. We propose a hierarchical clustering algorithm using closed frequent itemsets that use Wikipedia as an external knowledge to enhance the document representation. We evaluate our methods based on F-Score on standard datasets and show our results to be better than existing approaches.