Sentence similarity measurement based on shallow parsing

Authors:
Lin Li;Yiming Zhou;Boqiu Yuan;Jun Wang;Xia Hu
Affiliations:
School of Computer Science and Engineering, Beihang University, Beijing, China;School of Computer Science and Engineering, Beihang University, Beijing, China;School of Computer Science and Engineering, Beihang University, Beijing, China;School of Computer Science and Engineering, Beihang University, Beijing, China;School of Computer Science and Engineering, Beihang University, Beijing, China
Venue:
FSKD'09 Proceedings of the 6th international conference on Fuzzy systems and knowledge discovery - Volume 7
Year:
2009

Citing 12
Cited 0

WordNet: a lexical database for English

Communications of the ACM
A syntactic approach for searching similarities within sentences

Proceedings of the eleventh international conference on Information and knowledge management
Introduction to special issue on machine learning approaches to shallow parsing

The Journal of Machine Learning Research
Discovery of inference rules for question-answering

Natural Language Engineering
Improving text categorization using the importance of sentences

Information Processing and Management: an International Journal
BLEU: a method for automatic evaluation of machine translation

ACL '02 Proceedings of the 40th Annual Meeting on Association for Computational Linguistics
Sentence Similarity Based on Semantic Nets and Corpus Statistics

IEEE Transactions on Knowledge and Data Engineering
Unsupervised construction of large paraphrase corpora: exploiting massively parallel news sources

COLING '04 Proceedings of the 20th international conference on Computational Linguistics
Sentence Similarity based on Dynamic Time Warping

ICSC '07 Proceedings of the International Conference on Semantic Computing
Corpus-based and knowledge-based measures of text semantic similarity

AAAI'06 Proceedings of the 21st national conference on Artificial intelligence - Volume 1
Joint inference in information extraction

AAAI'07 Proceedings of the 22nd national conference on Artificial intelligence - Volume 1
LexRank: graph-based lexical centrality as salience in text summarization

Journal of Artificial Intelligence Research

Quantified Score

Hi-index	0.00

Visualization

Abstract

The paper proposes a novel method to determine sentence similarities. First two compared sentences are parsed by shallow-parsing and all noun phrases, verb phrases and preposition phrases of each sentence are extracted. Then the similarity between each kind of phrases is calculated based on a semantic vector method. The overall sentence similarity is defined as a combination of semantic similarities of the three kinds of phrases. Experiments show that the proposed method has a high performance in F-measure (81.6%) and Recall (97.4%).