Effect of evaluators' cognitive style on heuristic evaluation: Field dependent and field independent evaluators

  • Authors:

  • Chen Ling;Gavriel Salvendy

  • Affiliations:

  • School of Industrial Engineering, University of Oklahoma, 202 W. Boyd, Room 124, Norman, OK 73019, USA;School of Industrial Engineering, Purdue University, Grissom Hall, Room 263, 315 N. Grant St., West Lafayette, IN 47907, USA and Department of Industrial Engineering, Tsinghua University, Beijing, ...

  • Venue:
  • International Journal of Human-Computer Studies
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

Heuristic evaluation is a widely used usability evaluation method [Rosenbaum et al., 2000. A toolkit for strategic usability: results from workshops, panels, and surveys. In: Little, R., Nigay, L. (Eds.), In: Proceedings of ACM CHI 2000 Conference, New York, pp. 337-344]. But it suffers from large variability in the evaluation results due to differences among evaluators [Nielsen, 1993. Usability Engineering. Academic Press, Boston, MA]. The evaluation performance of evaluators with two types of cognitive styles-ten field independent (FI) subjects and ten field dependent (FD) subjects were compared. The results indicated that the FI subjects produced evaluation results with significantly higher thoroughness (t"1"8=3.49, p=0.0026), validity (t"1"8=4.26, p=0.0005), effectiveness (t"1"8=5.14, p=0.0001), and sensitivity (t"1"8=3.16, p=0.005) than the FD subjects. When assessing their own evaluation experiences, the FI subjects felt it was easier to find usability problems than the FD subjects (t"1"8=2.10, p=0.049), but the FD subjects felt more guided during the evaluation than the FI subjects (t"1"8=2.28, p=0.035).