Identifying Variable-Length Meaningful Phrases with Correlation Functions

  • Authors:
  • Hyoung-rae Kim;Philip K. Chan

  • Affiliations:
  • Florida Institute of Technology;Florida Institute of Technology

  • Venue:
  • ICTAI '04 Proceedings of the 16th IEEE International Conference on Tools with Artificial Intelligence
  • Year:
  • 2004

Quantified Score

Hi-index 0.00

Visualization

Abstract

Finding meaningful phrases in a document has been studied in various information retrieval systems in order to improve the performance. Many previous statistical phrase-finding methods had a different aim such as document classification. Some are hybridized with statistical and syntactic grammatical methods; others use correlation heuristics between words. We propose a new phrase-finding algorithm that adds correlated words one by one to the phrases found in the previous stage, maintaining high correlation within a phrase. Our results indicate that our algorithm finds more meaningful phrases than an existing algorithm. Furthermore, the previous algorithm could be improved by applying different correlation functions.