Fully automated generation of question-answer pairs for scripted virtual instruction

  • Authors:

  • Pascal Kuyten;Timothy Bickmore;Svetlana Stoyanchev;Paul Piwek;Helmut Prendinger;Mitsuru Ishizuka

  • Affiliations:

  • Graduate School of Information Science & Technology, The University of Tokyo, Japan;College of Computer and Information Science, Northeastern University, Boston, Massachusetts;Spoken Language Processing Group, Department of Computer Science, Columbia University, New York;NLG Group, Centre for Research in Computing, The Open University, Walton Hall, Milton Keynes, UK;National Institute of Informatics, Tokyo, Japan;Graduate School of Information Science & Technology, The University of Tokyo, Japan

  • Venue:
  • IVA'12 Proceedings of the 12th international conference on Intelligent Virtual Agents
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

We introduce a novel approach for automatically generating a virtual instructor from textual input only. Our fully implemented system first analyzes the rhetorical structure of the input text and then creates various question-answer pairs using patterns. These patterns have been derived from correlations found between rhetorical structure of monologue texts and question-answer pairs in the corresponding dialogues. A selection of the candidate pairs is verbalized into a diverse collection of question-answer pairs. Finally the system compiles the collection of question-answer pairs into scripts for a virtual instructor. Our end-to-end system presents questions in pre-fixed order and the agent answers them. Our system was evaluated with a group of twenty-four subjects. The evaluation was conducted using three informed consent documents of clinical trials from the domain of colon cancer. Each of the documents was explained by a virtual instructor using 1) text, 2) text and agent monologue, and 3) text and agent performing question-answering. Results show that an agent explaining an informed consent document did not provide significantly better comprehension scores, but did score higher on satisfaction, compared to two control conditions.