Distributional semantics and compositionality 2011: shared task description and results

Authors:
Chris Biemann;Eugenie Giesbrecht
Affiliations:
Technical University of Darmstadt, Hochschulstr, Darmstadt, Germany;FZI Forschungszentrum Informatik, Karlsruhe, Germany
Venue:
DiSCo '11 Proceedings of the Workshop on Distributional Semantics and Compositionality
Year:
2011

Citing 4
Cited 5

Multiword Expressions: A Pain in the Neck for NLP

CICLing '02 Proceedings of the Third International Conference on Computational Linguistics and Intelligent Text Processing
A statistical approach to the semantics of verb-particles

MWE '03 Proceedings of the ACL 2003 workshop on Multiword expressions: analysis, acquisition and treatment - Volume 18
Automatic identification of non-compositional multi-word expressions using latent semantic analysis

MWE '06 Proceedings of the Workshop on Multiword Expressions: Identifying and Exploiting Underlying Properties
From frequency to meaning: vector space models of semantics

Journal of Artificial Intelligence Research

Identifying collocations to measure compositionality: shared task system description

DiSCo '11 Proceedings of the Workshop on Distributional Semantics and Compositionality
Measuring the compositionality of collocations via word co-occurrence vectors: shared task system description

DiSCo '11 Proceedings of the Workshop on Distributional Semantics and Compositionality
Exemplar-based word-space model for compositionality detection: shared task system description

DiSCo '11 Proceedings of the Workshop on Distributional Semantics and Compositionality
Domain and function: a dual-space model of semantic relations and compositions

Journal of Artificial Intelligence Research
Unsupervised feature adaptation for cross-domain NLP with an application to compositionality grading

CICLing'13 Proceedings of the 14th international conference on Computational Linguistics and Intelligent Text Processing - Volume Part I

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper gives an overview of the shared task at the ACL-HLT 2011 DiSCo (Distributional Semantics and Compositionality) workshop. We describe in detail the motivation for the shared task, the acquisition of datasets, the evaluation methodology and the results of participating systems. The task of assigning a numerical score for a phrase according to its compositionality showed to be hard. Many groups reported features that intuitively should work, yet showed no correlation with the training data. The evaluation reveals that most systems outperform simple baselines, yet have difficulties in reliably assigning a compositionality score that closely matches the gold standard. Overall, approaches based on word space models performed slightly better than methods relying solely on statistical association measures.