Text relatedness based on a word thesaurus

  • Authors:

  • George Tsatsaronis;Iraklis Varlamis;Michalis Vazirgiannis

  • Affiliations:

  • Department of Computer and Information Science, Norwegian University of Science and Technology, Norway;Department of Informatics and Telematics, Harokopio University, Greece;Department of Informatics, Athens University of Economics and Business, Greece

  • Venue:
  • Journal of Artificial Intelligence Research
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

The computation of relatedness between two fragments of text in an automated manner requires taking into account a wide range of factors pertaining to the meaning the two fragments convey, and the pairwise relations between their words. Without doubt, a measure of relatedness between text segments must take into account both the lexical and the semantic relatedness between words. Such a measure that captures well both aspects of text relatedness may help in many tasks, such as text retrieval, classification and clustering. In this paper we present a new approach for measuring the semantic relatedness between words based on their implicit semantic links. The approach exploits only a word thesaurus in order to devise implicit semantic links between words. Based on this approach, we introduce Omiotis, a new measure of semantic relatedness between texts which capitalizes on the word-to-word semantic relatedness measure (SR) and extends it to measure the relatedness between texts. We gradually validate our method: we first evaluate the performance of the semantic relatedness measure between individual words, covering word-to-word similarity and relatedness, synonym identification and word analogy; then, we proceed with evaluating the performance of our method in measuring text-to-text semantic relatedness in two tasks, namely sentence-to-sentence similarity and paraphrase recognition. Experimental evaluation shows that the proposed method outperforms every lexicon-based method of semantic relatedness in the selected tasks and the used data sets, and competes well against corpus-based and hybrid approaches.