Systematic review: A systematic review of effect size in software engineering experiments
Information and Software Technology
Damaged merchandise? a review of experiments that compare usability evaluation methods
Human-Computer Interaction
HCI... not as it should be: inferential statistics in HCI research
BCS-HCI '07 Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI...but not as we know it - Volume 1
Powerful and consistent analysis of likert-type ratingscales
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Likert-type scales, statistical methods, and effect sizes
Communications of the ACM
Why ask why?: considering motivation in visualization evaluation
Proceedings of the 2012 BELIV Workshop: Beyond Time and Errors - Novel Evaluation Methods for Visualization
Waving to a touch interface: descriptive field study of a multipurpose multimodal public display
Proceedings of the 2nd ACM International Symposium on Pervasive Displays
| Hi-index | 0.02 |
CHI researchers typically use a significance testing approach to statistical analysis when testing hypotheses during usability evaluations. However, the appropriateness of this approach is under increasing criticism, with statisticians, economists, and psychologists arguing against the use of routine interpretation of results using "canned" p values. Three problems with current practice - the fallacy of the transposed conditional, a neglect of power, and the reluctance to interpret the size of effects - can lead us to build weak theories based on vaguely specified hypothesis, resulting in empirical studies which produce results that are of limited practical or scientific use. Using publicly available data presented at CHI 2010 [19] as an example we address each of the three concerns and promote consideration of the magnitude and actual importance of effects, as opposed to statistical significance, as the new criteria for evaluating CHI research.