An Empirical Study of the Effect of Agent Competence on User Performance and Perception

  • Authors:
  • Jun Xiao;John Stasko;Richard Catrambone

  • Affiliations:
  • Georgia Institute of Technology;Georgia Institute of Technology;Georgia Institute of Technology

  • Venue:
  • AAMAS '04 Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems - Volume 1
  • Year:
  • 2004

Quantified Score

Hi-index 0.00

Visualization

Abstract

We studied the role of the competence of an interface agent that helped users to learn and use a text editor. Participants in the study made a set of changes to a document with the aid of one of four interface agents. The agent would respond to participantsý spoken questions as well as make proactive suggestions using a synthesized voice. The agents varied in the quality of responses and suggestions. One group of participants could also access a help screen. Results revealed that the perceived utility of the agent was influenced by the types of errors it made, while participants' subjective impressions of the agent related to the perceptions of its embodiment. Additionally, allowing participants to choose their preferred assistance styles improved objective performance. We relate quantitative findings with qualitative interview data and discuss implications for the design and the implementation of systems with interface agents.