Finding usability problems through heuristic evaluation
CHI '92 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
The evaluator effect in usability tests
CHI 98 Cconference Summary on Human Factors in Computing Systems
The state of the art in automating usability evaluation of user interfaces
ACM Computing Surveys (CSUR)
CHI '02 Extended Abstracts on Human Factors in Computing Systems
The Evaluator Effect during First-Time Use of the Cognitive Walkthrough Technique
Proceedings of HCI International (the 8th International Conference on Human-Computer Interaction) on Human-Computer Interaction: Ergonomics and User Interfaces-Volume I - Volume I
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
International Journal of Human-Computer Studies
The relationship between accessibility and usability of websites
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Damaged merchandise? a review of experiments that compare usability evaluation methods
Human-Computer Interaction
International Journal of Human-Computer Studies
Web accessibility testing: when the method is the culprit
ICCHP'06 Proceedings of the 10th international conference on Computers Helping People with Special Needs
Validity and reliability of web accessibility guidelines
Proceedings of the 11th international ACM SIGACCESS conference on Computers and accessibility
Designing for auditory web access: accessibility and cellphone users
Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A)
On the testability of WCAG 2.0 for beginners
Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A)
Web not for all: a large scale study of web accessibility
Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A)
Accessibility challenges and tool features: an IBM Web developer perspective
Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A)
Testability and validity of WCAG 2.0: the expertise effect
Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility
Macroscopic characterisations of Web accessibility
The New Review of Hypermedia and Multimedia - Web Accessibility
Development and trial of an educational tool to support the accessibility evaluation process
Proceedings of the International Cross-Disciplinary Conference on Web Accessibility
Crosschecking the mobile web for people with visual impairments
Proceedings of the International Cross-Disciplinary Conference on Web Accessibility
ACM SIGACCESS Accessibility and Computing
A tool to support the web accessibility evaluation process for novices
Proceedings of the 16th annual joint conference on Innovation and technology in computer science education
The interplay between web aesthetics and accessibility
The proceedings of the 13th international ACM SIGACCESS conference on Computers and accessibility
Is accessibility conformance an elusive property? A study of validity and reliability of WCAG 2.0
ACM Transactions on Accessible Computing (TACCESS)
Evaluation of the effectiveness of a tool to support novice auditors
Proceedings of the International Cross-Disciplinary Conference on Web Accessibility
Integrating manual and automatic evaluations to measure accessibility barriers
ICCHP'12 Proceedings of the 13th international conference on Computers Helping People with Special Needs - Volume Part I
| Hi-index | 0.00 |
Manual accessibility evaluation plays an important role in validating the accessibility of Web pages. This role has become increasingly critical with the advent of the Web Content Accessibility Guidelines (WCAG) 2.0 and their reliance on user evaluation to validate certain conformance measures. However, the role of expertise, in such evaluations, is unknown and has not previously been studied. This paper sets out to investigate the interplay between expert and non-expert evaluation by conducting a Barrier Walkthrough (BW) study with 19 expert and 51 non-expert judges. The BW method provides an evaluation framework that can be used to manually assess the accessibility of Web pages for different user groups including motor impaired, hearing impaired, low vision, cognitive impaired, etc. We conclude that the level of expertise is an important factor in the quality of accessibility evaluation of Web pages. Expert judges spent significantly less time than non-experts; rated themselves as more productive and confident than non-experts; and ranked and rated pages differently against each type of disability. Finally, both effectiveness and reliability of the expert judges are significantly higher than non-expert judges.