Recognition of word collocation habits using frequency rank ratio and inter-term intimacy

  • Authors:
  • Peng Tang;Tommy W. S. Chow

  • Affiliations:
  • Department of Electronic Engineering, City University of Hong Kong, Hong Kong;Department of Electronic Engineering, City University of Hong Kong, Hong Kong

  • Venue:
  • Expert Systems with Applications: An International Journal
  • Year:
  • 2013

Quantified Score

Hi-index 12.05

Visualization

Abstract

An effective algorithm for extracting two useful features from text documents for analyzing word collocation habits, ''Frequency Rank Ratio'' (FRR) and ''Intimacy'', is proposed. FRR is derived from a ranking index of a word according to its word frequency. Intimacy, computed by a compact language model called Influence Language Model (ILM), measures how close a word is to others within the same sentence. Using the proposed features, a visualization framework is developed for word collocation analysis. To evaluate our proposed framework, two corpora are designed and collected from the real-life data covering diverse topics and genres. Extensive simulations are conducted to illustrate the feasibility and effectiveness of our visualization framework. Our results demonstrate that the proposed features and algorithm are able to conduct reliable text analysis efficiently.