Outliers in usability testing: how to treat usability problems found for only one test participant?

  • Authors:
  • Asbjørn Følstad;Effie Lai-Chong Law;Kasper Hornbæk

  • Affiliations:
  • SINTEF, Blindern, Oslo, Norway;University of Leicester, Leicester, UK;University of Copenhagen, Njalsgade, DK

  • Venue:
  • Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

In usability testing, usability problems are often found for only one test participant. The literature does not help in deciding whether such single-user problems should be accepted or rejected as usability problems. To help us understand how such decisions are made in practical usability testing, 89 practitioners described how they dealt with single-user problems in their latest usability test. Single-user problems was accepted, rejected, or reported as outliers. This decision depended on problem severity, participant profile, sample size, and judgments on whether the problem is an artifact of the test situation.