Heuristic evaluation of user interfaces
CHI '90 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
CHI '03 Extended Abstracts on Human Factors in Computing Systems
CHI '07 Extended Abstracts on Human Factors in Computing Systems
Introducing item response theory for measuring usability inspection processes
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
A comparative test of web accessibility evaluation methods
Proceedings of the 10th international ACM SIGACCESS conference on Computers and accessibility
Comparative usability evaluation (CUE-4)
Behaviour & Information Technology
The effect of group discussions in usability inspection: a pilot study
Proceedings of the 5th Nordic conference on Human-computer interaction: building bridges
Supporting novice usability practitioners with usability engineering tools
International Journal of Human-Computer Studies
Guideline aggregation: web accessibility evaluation for older users
Proceedings of the 2009 International Cross-Disciplinary Conference on Web Accessibililty (W4A)
How much does expertise matter?: a barrier walkthrough study with experts and non-experts
Proceedings of the 11th international ACM SIGACCESS conference on Computers and accessibility
Testability and validity of WCAG 2.0: the expertise effect
Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility
Is accessibility conformance an elusive property? A study of validity and reliability of WCAG 2.0
ACM Transactions on Accessible Computing (TACCESS)
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Hi-index | 0.00 |
Evaluators who examine the same system using the same usability evaluation method tend to report substantially different sets of problems. This so-called evaluator effect means that different evaluations point to considerably different revisions of the evaluated system. The first step in coping with the evaluator effect is to acknowledge its existence. In this study 11 usability specialists individually inspected a website and then met in four groups to combine their findings into group outputs. Although the overlap in reported problems between any two evaluators averaged only 9%, the 11 evaluators felt that they were largely in agreement. The evaluators perceived their disparate observations as mulitiple sources of evidence in support of the same issues, not as disagreements. Thus, the group work increased the evaluators' confidence in their individual inspections, rather than alerted them to the evaluator effect.