Is the `Figure of Merit' Really That Meritorious?

Authors:
Jarinee Chattratichart;Gitte Lindgaard
Affiliations:
Faculty of Computing, Information Systems and Mathematics, Kingston University, London, United Kingdom;HOTLab, Psychology Department, Carleton University, Ottawa, Canada
Venue:
INTERACT '09 Proceedings of the 12th IFIP TC 13 International Conference on Human-Computer Interaction: Part I
Year:
2009

Citing 10
Cited 0

A mathematical model of the finding of usability problems

INTERCHI '93 Proceedings of the INTERCHI '93 conference on Human factors in computing systems
Foundations of statistical natural language processing

Foundations of statistical natural language processing
Sale must end: should discount methods be cleared off HCI's shelves?

interactions
Determining the effectiveness of the usability problem inspector: a theory-based model and tool for finding usability problems

Determining the effectiveness of the usability problem inspector: a theory-based model and tool for finding usability problems
Applying user testing data to UEM performance metrics

CHI '04 Extended Abstracts on Human Factors in Computing Systems
Comparative usability evaluation

Behaviour & Information Technology
Current practice in measuring usability: Challenges to usability studies and research

International Journal of Human-Computer Studies
A comparative evaluation of heuristic-based usability inspection methods

CHI '08 Extended Abstracts on Human Factors in Computing Systems
Damaged merchandise? a review of experiments that compare usability evaluation methods

Human-Computer Interaction
Supporting novice usability practitioners with usability engineering tools

International Journal of Human-Computer Studies

Quantified Score

Hi-index	0.00

Visualization

Abstract

Studies comparing performance of Usability Evaluation Methods (UEMs) led to three standard metrics, namely, validity, thoroughness, and effectiveness, calculated from lab-based usability test results. The effectiveness metric, E = T × V, was proposed as the `figure of merit' [7] that would give a balanced account of validity and thoroughness. This paper provides an analysis of the formula to caution future researchers and usability practitioners against its use, proposes an alternative formula, and discusses the limitations of the common baseline approach to UEM comparison.