Towards effective tutorial feedback for explanation questions: a dataset and baselines

Authors:
Myroslava O. Dzikovska;Rodney D. Nielsen;Chris Brew
Affiliations:
University of Edinburgh, Edinburgh, UK;University of Colorado at Boulder, Boulder, CO;Educational Testing Service, Princeton, NJ
Venue:
NAACL HLT '12 Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Year:
2012

Citing 14
Cited 3

Evaluating text categorization

HLT '91 Proceedings of the workshop on Speech and Natural Language
Pilot-Testing a Tutorial Dialogue System That Supports Self-Explanation

ITS '02 Proceedings of the 6th International Conference on Intelligent Tutoring Systems
It's a contradiction---no, it's not: a case study using functional relations

EMNLP '08 Proceedings of the Conference on Empirical Methods in Natural Language Processing
Recognizing entailment in intelligent tutoring systems*

Natural Language Engineering
The third PASCAL recognizing textual entailment challenge

RTE '07 Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing
Using Natural Language Processing to Analyze Tutorial Dialogue Corpora Across Domains Modalities

Proceedings of the 2009 conference on Artificial Intelligence in Education: Building Learning Systems that Care: From Knowledge Representation to Affective Modelling
The “DeMAND” coding scheme: A “common language” for representing and analyzing student discourse

Proceedings of the 2009 conference on Artificial Intelligence in Education: Building Learning Systems that Care: From Knowledge Representation to Affective Modelling
BEETLE II: a system for tutoring and computational linguistics experimentation

ACLDemos '10 Proceedings of the ACL 2010 System Demonstrations
Intelligent tutoring with natural language support in the BEETLE II system

EC-TEL'10 Proceedings of the 5th European conference on Technology enhanced learning conference on Sustaining TEL: from innovation to learning and practice
Learning to grade short answer questions using semantic similarity measures and dependency graph alignments

HLT '11 Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1
Learning multi-class theories in ILP

ILP'10 Proceedings of the 20th international conference on Inductive logic programming
Talk like an electrician: student dialogue mimicking behavior in an intelligent tutoring system

AIED'11 Proceedings of the 15th international conference on Artificial intelligence in education
AutoTutor: A simulation of a human tutor

Cognitive Systems Research
Evaluating language understanding accuracy with respect to objective outcomes in a dialogue system

EACL '12 Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics

Short answer assessment: establishing links between research strands

Proceedings of the Seventh Workshop on Building Educational Applications Using NLP
Exploiting collaborative filtering techniques for automatic assessment of student free-text responses

Proceedings of the 22nd ACM international conference on Conference on information & knowledge management
Assessing elementary students' science competency with text analytics

Proceedings of the Fourth International Conference on Learning Analytics And Knowledge

Quantified Score

Hi-index	0.00

Visualization

Abstract

We propose a new shared task on grading student answers with the goal of enabling well-targeted and flexible feedback in a tutorial dialogue setting. We provide an annotated corpus designed for the purpose, a precise specification for a prediction task and an associated evaluation methodology. The task is feasible but non-trivial, which is demonstrated by creating and comparing three alternative baseline systems. We believe that this corpus will be of interest to the researchers working in textual entailment and will stimulate new developments both in natural language processing in tutorial dialogue systems and textual entailment, contradiction detection and other techniques of interest for a variety of computational linguistics tasks.