Evaluating a conversation-centered interactive drama

  • Authors:

  • Manish Mehta;Steven Dow;Michael Mateas;Blair MacIntyre

  • Affiliations:

  • Georgia Institute of Technology, Atlanta, GA;Georgia Institute of Technology, Atlanta, GA;University of California, Santa Cruz, Santa Cruz, CA;Georgia Institute of Technology, Atlanta, GA

  • Venue:
  • Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

There is a growing interest in developing technologies for creating interactive dramas [13, 22]. Evaluating them, however, remains an open research problem. In this paper, we present a method for evaluating the technical and design approaches employed in a conversation-centered interactive drama. This method correlates players' subjective experience during conversational breakdowns, captured using retrospective protocols, with the corresponding AI processing in the input language understanding and dialog management subsystems. The methodology is employed to analyze conversation breakdowns in the interactive drama Façade. We find that the narrative cues offered by an interactive drama, coupled with believable character performance, can allow players to interpretively bridge system limitations and avoid experiencing a conversation breakdown. Further, we find that, contrary to standard practice for task-oriented conversation systems, using shallowly understood information as part of the system output hampers the player experience in an interactive drama.