Heuristic evaluation of user interfaces
CHI '90 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Refining the test phase of usability evaluation: how many subjects is enough?
Human Factors - Special issue: measurement in human factors
A mathematical model of the finding of usability problems
CHI '93 Proceedings of the INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems
A Comprehensive Evaluation of Capture-Recapture Models for Estimating Software Defect Content
IEEE Transactions on Software Engineering
CHI '03 Extended Abstracts on Human Factors in Computing Systems
Sample sizes for usability tests: mostly math, not magic
interactions - Waits & Measures
Usability testing: what have we overlooked?
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Introducing item response theory for measuring usability inspection processes
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
HCI... not as it should be: inferential statistics in HCI research
BCS-HCI '07 Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI...but not as we know it - Volume 1
A pattern-based usability inspection method: first empirical performance measures and future issues
BCS-HCI '07 Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI...but not as we know it - Volume 2
Controlling the usability evaluation process under varying defect visibility
Proceedings of the 23rd British HCI Group Annual Conference on People and Computers: Celebrating People and Technology
Sample size in usability studies
Communications of the ACM
Reviewing and Extending the Five-User Assumption: A Grounded Procedure for Interaction Evaluation
ACM Transactions on Computer-Human Interaction (TOCHI)
Journal of Biomedical Informatics
A grounded procedure for managing data and sample size of a home medical device assessment
HCI'13 Proceedings of the 15th international conference on Human-Computer Interaction: human-centred design approaches, methods, tools, and environments - Volume Part I
| Hi-index | 0.02 |
Current prediction models for usability evaluations are based on stochastic distributions derived from series of Bernoulli processes. The underlying assumption of these models is a homogeneous detection probability despite of it being intuitively unrealistic. This paper contributes a simple statistical test for existence of heterogeneity in the process. The compound beta-binomial model is proposed to incorporate sources of heterogeneity and compared to the binomial model. Analysis of several data sets from the literature illustrates the methods and reveals that heterogeneity occurs in most situations. Finally, it is demonstrated how heterogeneity biases the prediction of evaluation processes. Open research questions are discussed and preliminary advice for practitioners for controlling their processes is given.