The validity of the stimulated retrospective think-aloud method as measured by eye tracking

  • Authors:
  • Zhiwei Guan;Shirley Lee;Elisabeth Cuddihy;Judith Ramey

  • Affiliations:
  • University of Washington, Seattle (UW), Seattle, WA;University of Washington, Seattle (UW), Seattle, WA;University of Washington, Seattle (UW), Seattle, WA;University of Washington, Seattle (UW), Seattle, WA

  • Venue:
  • Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
  • Year:
  • 2006

Quantified Score

Hi-index 0.01

Visualization

Abstract

Retrospective Think aloud (RTA) is a usability method that collects the verbalization of a user's performance after the performance is over. There has been little work done to investigate the validity and reliability of RTA. This paper reports on an experiment investigating these issues with a form of the method called stimulated RTA. By comparing subjects' verbalizations with their eye movements, we support the validity and reliability of stimulated RTA: the method provides a valid account of what people attended to in completing tasks, it has a low risk of introducing fabrications, and its validity isn't affected by task complexity. More detailed analysis of RTA shows that it also provides additional information about user's inferences and strategies in completing tasks. The findings of this study provide valuable support for usability practitioners to use RTA and to trust the users' performance information collected by this method in a usability study.