What makes evaluators to find more usability problems?: a meta-analysis for individual detection rates

Authors:
Wonil Hwang;Gavriel Salvendy
Affiliations:
School of Industrial & Information System Engineering, Soongsil University, Seoul, South Korea;School of Industrial Engineering, Purdue University, West Lafayette, IN and Department of Industrial Engineering, Tsinghua University, Beijing, P.R. China
Venue:
HCI'07 Proceedings of the 12th international conference on Human-computer interaction: interaction design and usability
Year:
2007

Citing 10
Cited 2

Refining the test phase of usability evaluation: how many subjects is enough?

Human Factors - Special issue: measurement in human factors
Finding usability problems through heuristic evaluation

CHI '92 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Estimating the number of subjects needed for a thinking aloud test

International Journal of Human-Computer Studies
Validating the SUE inspection technique

AVI '00 Proceedings of the working conference on Advanced visual interfaces
Empirical development of a heuristic evaluation methodology for shared workspace groupware

CSCW '02 Proceedings of the 2002 ACM conference on Computer supported cooperative work
Perspective-based Usability Inspection: An Empirical Validationof Efficacy

Empirical Software Engineering
Testing web sites: five users is nowhere near enough

CHI '01 Extended Abstracts on Human Factors in Computing Systems
Analysis of combinatorial user effect in international usability tests

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
An assessment of the experimental literature on electronic support of group work: results of a meta-analysis

Human-Computer Interaction
Interacting with hypertext: a meta-analysis of experimental studies

Human-Computer Interaction

Weak inter-rater reliability in heuristic evaluation of video games

CHI '11 Extended Abstracts on Human Factors in Computing Systems
Gamers as usability evaluators: a study in the domain of virtual worlds

Proceedings of the 11th Brazilian Symposium on Human Factors in Computing Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

Since many empirical results have been accumulated in usability evaluation research, it would be very useful to provide usability practitioners with generalized guidelines by analyzing the combined results. This study aims at estimating individual detection rate for user-based testing and heuristic evaluation through meta-analysis, and finding significant factors, which affect individual detection rates. Based on the results of 18 user-based testing and heuristic evaluation experiments, individual detection rates in user-based testing and heuristic evaluation were estimated as 0.36 and 0.14, respectively. Expertise and task type were found as significant factors to improve individual detection rate in heuristic evaluation.