VENSES – a linguistically-based system for semantic evaluation

Authors:
Rodolfo Delmonte;Sara Tonelli;Marco Aldo Piccolino Boniforti;Antonella Bristot
Affiliations:
Department of Language Sciences, Laboratory of Computational Linguistics, University Ca' Foscari, Venice, Italy;Department of Language Sciences, Laboratory of Computational Linguistics, University Ca' Foscari, Venice, Italy;Department of Language Sciences, Laboratory of Computational Linguistics, University Ca' Foscari, Venice, Italy;Department of Language Sciences, Laboratory of Computational Linguistics, University Ca' Foscari, Venice, Italy
Venue:
MLCW'05 Proceedings of the First international conference on Machine Learning Challenges: evaluating Predictive Uncertainty Visual Object Classification, and Recognizing Textual Entailment
Year:
2005

Citing 10
Cited 3

DIRT @SBT@discovery of inference rules from text

Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining
Open-domain textual question answering techniques

Natural Language Engineering
Unsupervised construction of large paraphrase corpora: exploiting massively parallel news sources

COLING '04 Proceedings of the 20th international conference on Computational Linguistics
Wide-coverage semantic representations from a CCG parser

COLING '04 Proceedings of the 20th international conference on Computational Linguistics
A probabilistic classification approach for lexical textual entailment

AAAI'05 Proceedings of the 20th national conference on Artificial intelligence - Volume 3
Evaluating GETARUNS parser with GREVAL test suite

ROMAND '04 Proceedings of the 3rd Workshop on RObust Methods in Analysis of Natural Language Data
Text understanding with GETARUNS for Q/A and summarization

TextMean '04 Proceedings of the 2nd Workshop on Text Meaning and Interpretation
Measuring the semantic similarity of texts

EMSEE '05 Proceedings of the ACL Workshop on Empirical Modeling of Semantic Equivalence and Entailment
Local textual inference: can it be defined or circumscribed?

EMSEE '05 Proceedings of the ACL Workshop on Empirical Modeling of Semantic Equivalence and Entailment
The PASCAL recognising textual entailment challenge

MLCW'05 Proceedings of the First international conference on Machine Learning Challenges: evaluating Predictive Uncertainty Visual Object Classification, and Recognizing Textual Entailment

An approach for textual entailment recognition based on stacking and voting

MICAI'06 Proceedings of the 5th Mexican international conference on Artificial Intelligence
A statistics-based semantic textual entailment system

MICAI'11 Proceedings of the 10th Mexican international conference on Advances in Artificial Intelligence - Volume Part I
Semantic annotation for textual entailment recognition

MICAI'12 Proceedings of the 11th Mexican international conference on Advances in Computational Intelligence - Volume Part II

Quantified Score

Hi-index	0.00

Visualization

Abstract

The system for semantic evaluation VENSES (Venice Semantic Evaluation System) is organized as a pipeline of two subsystems: the first is a reduced version of GETARUN, our system for Text Understanding. The output of the system is a flat list of augmented head-dependent structures with Grammatical Relations and Semantic Roles labels. The evaluation system is made up of two main modules: the first is a sequence of linguistic rules; the second is a quantitatively based measurement of input structures and predicates. VENSES measures semantic similarity which may range from identical linguistic items, to synonymous, lexically similar, or just morphologically derivable. Both modules go through General Consistency checks which are targeted to high level semantic attributes like presence of modality, negation, and opacity operators, temporal and spatial location checks. Results in cws, recall and precision are homogeneous for both training and test corpus and fare higher than 60%.