Be Brief, And They Shall Learn: Generating Concise Language Feedback for a Computer Tutor

  • Authors:
  • Barbara Di Eugenio;Davide Fossati;Susan Haller;Dan Yu;Michael Glass

  • Affiliations:
  • Department of Computer Science, University of Illinois at Chicago, Chicago, IL, 60302, USA. {bdieugen,dfossa1}@uic.edu;Department of Computer Science, University of Illinois at Chicago, Chicago, IL, 60302, USA. {bdieugen,dfossa1}@uic.edu;Department of Computer Science, SUNY Potsdam, Potsdam NY 13676, USA. hallersm@potsdam.edu;Albert A. Webb Associates, Riverside, CA 92506, USA. danyu79@gmail.com;Department of Math and Computer Science, Valparaiso University, Valparaiso, IN 46383, USA. michael.glass@valpo.edu

  • Venue:
  • International Journal of Artificial Intelligence in Education
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

To investigate whether more concise Natural Language feedback improves learning, we developed two Natural Language generators (DIAG-NLP1 and DIAG-NLP2), to provide feedback in an Intelligent Tutoring System that teaches troubleshooting. We systematically evaluated them in a three way comparison that included the original system, which generates overly repetitive feedback. We found that DIAG-NLP2, the generator which intuitively produces the best, corpus-based language, does engender the most learning. Distinguishing features of the more effective feedback are: it obeys Grice's maxim of brevity, it is more directive and uses a specific type of referring expressions. Interestingly, simpler ways of restructuring the original repetitive feedback as done in DIAG-NLP1, such as exploiting the hierarchical structure of the domain, were not effective. Since the design of interfaces to Intelligent Tutoring Systems often includes verbal feedback, we suggest that: if the number of different contexts in which verbal feedback is provided is high, such feedback should be based on corpus studies, and generated by techniques more sophisticated than template filling.