Analysis of strategies for improving and estimating the effectiveness of heuristic evaluation

  • Authors:
  • Effie Lai-Chong Law;Ebba Thora Hvannberg

  • Affiliations:
  • Eidgenössische Technische Hochschule Zürich, Switzerland;University of Iceland, Iceland

  • Venue:
  • Proceedings of the third Nordic conference on Human-computer interaction
  • Year:
  • 2004

Quantified Score

Hi-index 0.00

Visualization

Abstract

Effectiveness of Heuristic Evaluation (HE) has severely been challenged. The present study investigated two strategies for improving the effectiveness of HE: Selection of usability guidelines (Gerhardt-Powals' cognitive engineering principles vs. Nielsen's heuristics); Provision of support and training. We also examined various strategies for estimating the effectiveness of HE. An empirical study with a balanced within- and between subject 2x2 factorial design involving 18 novice evaluators was conducted. 290 non-consolidated usability problems (UPs) identified in 35 HE test sessions were evaluated against 88 actual UPs found in the user tests (n = 17) performed earlier. Nielsen's heuristics were proved to be more effective in enabling the evaluators to uncover significantly more UPs, though in general the effectiveness of HE was low. Implications for future work are drawn.