Eye gaze patterns in conversations: there is more to conversational agents than meets the eyes

  • Authors:

  • Roel Vertegaal;Robert Slagter;Gerrit van der Veer;Anton Nijholt

  • Affiliations:

  • Queen's University, Canada;Telematics Institute, The Netherlands;Vrije Universiteit, The Netherlands;Twente University, The Netherlands

  • Venue:
  • Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
  • Year:
  • 2001

Quantified Score

Hi-index 0.02

Visualization

Abstract

In multi-agent, multi-user environments, users as well as agents should have a means of establishing who is talking to whom. In this paper, we present an experiment aimed at evaluating whether gaze directional cues of users could be used for this purpose. Using an eye tracker, we measured subject gaze at the faces of conversational partners during four-person conversations. Results indicate that when someone is listening or speaking to individuals, there is indeed a high probability that the person looked at is the person listened (p=88%) or spoken to (p=77%). We conclude that gaze is an excellent predictor of conversational attention in multiparty conversations. As such, it may form a reliable source of input for conversational systems that need to establish whom the user is speaking or listening to. We implemented our findings in FRED, a multi-agent conversational system that uses eye input to gauge which agent the user is listening or speaking to.