DIRT @SBT@discovery of inference rules from text
Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining
Learning surface text patterns for a Question Answering system
ACL '02 Proceedings of the 40th Annual Meeting on Association for Computational Linguistics
A general framework for distributional similarity
EMNLP '03 Proceedings of the 2003 conference on Empirical methods in natural language processing
Preemptive information extraction using unrestricted relation discovery
HLT-NAACL '06 Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics
Automatic paraphrase acquisition from news articles
HLT '02 Proceedings of the second international conference on Human Language Technology Research
Learning entailment rules for unary templates
COLING '08 Proceedings of the 22nd International Conference on Computational Linguistics - Volume 1
Cheap and fast---but is it good?: evaluating non-expert annotations for natural language tasks
EMNLP '08 Proceedings of the Conference on Empirical Methods in Natural Language Processing
Towards cross-lingual textual entailment
HLT '10 Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics
"Ask not what textual entailment can do for you..."
ACL '10 Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics
Global learning of focused entailment graphs
ACL '10 Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics
CSLDAMT '10 Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk
Learning first-order Horn clauses from web text
EMNLP '10 Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing
Types of common-sense knowledge needed for recognizing textual entailment
HLT '11 Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers - Volume 2
Divide and conquer: crowdsourcing the creation of cross-lingual textual entailment corpora
EMNLP '11 Proceedings of the Conference on Empirical Methods in Natural Language Processing
Identifying relations for open information extraction
EMNLP '11 Proceedings of the Conference on Empirical Methods in Natural Language Processing
Perspectives on crowdsourcing annotations for natural language processing
Language Resources and Evaluation
| Hi-index | 0.00 |
The importance of inference rules to semantic applications has long been recognized and extensive work has been carried out to automatically acquire inference-rule resources. However, evaluating such resources has turned out to be a non-trivial task, slowing progress in the field. In this paper, we suggest a framework for evaluating inference-rule resources. Our framework simplifies a previously proposed "instance-based evaluation" method that involved substantial annotator training, making it suitable for crowdsourcing. We show that our method produces a large amount of annotations with high inter-annotator agreement for a low cost at a short period of time, without requiring training expert annotators.