Comparing accessibility evaluation tools: a method for tool effectiveness

Authors:
Giorgio Brajnik
Affiliations:
Università di Udine, Dip. di Matematica e Informatica, Italy
Venue:
Universal Access in the Information Society
Year:
2004

Citing 0
Cited 29

Automatic accessibility evaluation of dynamic web pages generated through XSLT

W4A '05 Proceedings of the 2005 International Cross-Disciplinary Workshop on Web Accessibility (W4A)
Detecting Anomaly and Failure in Web Applications

IEEE MultiMedia
Quantitative metrics for measuring web accessibility

W4A '07 Proceedings of the 2007 international cross-disciplinary conference on Web accessibility (W4A)
SAMBA: a semi-automatic method for measuring barriers of accessibility

Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibility
Effects of sampling methods on web accessibility evaluations

Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibility
The BenToWeb Test Case Suites for the Web Content Accessibility Guidelines (WCAG) 2.0

ICCHP '08 Proceedings of the 11th international conference on Computers Helping People with Special Needs
Beyond Conformance: The Role of Accessibility Evaluation Methods

WISE '08 Proceedings of the 2008 international workshops on Web Information Systems Engineering
Model-based automatic usability validation: a tool concept for improving web-based UIs

Proceedings of the 5th Nordic conference on Human-computer interaction: building bridges
Hera-FFX: a Firefox add-on for semi-automatic web accessibility evaluation

Proceedings of the 2009 International Cross-Disciplinary Conference on Web Accessibililty (W4A)
Transition of accessibility evaluation tools to new standards

Proceedings of the 2009 International Cross-Disciplinary Conference on Web Accessibililty (W4A)
Exploring Automatic CSS Accessibility Evaluation

ICWE '9 Proceedings of the 9th International Conference on Web Engineering
A lightweight methodology to improve web accessibility

Proceedings of the 2009 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists
Validity and reliability of web accessibility guidelines

Proceedings of the 11th international ACM SIGACCESS conference on Computers and accessibility
Detection of layout-purpose TABLE tags based on machine learning

UAHCI'07 Proceedings of the 4th international conference on Universal access in human-computer interaction: applications and services
The usability of accessibility evaluation tools

UAHCI'07 Proceedings of the 4th international conference on Universal access in human-computer interaction: applications and services
The BenToWeb XHTML 1.0 test suite for the web content accessibility guidelines 2.0: last call working draft

UAHCI'07 Proceedings of the 4th international conference on Universal access in human-computer interaction: applications and services
Considering web accessibility in information retrieval systems

ICWE'07 Proceedings of the 7th international conference on Web engineering
Automatic evaluation of mobile web accessibility

ERCIM'06 Proceedings of the 9th conference on User interfaces for all
Testability and validity of WCAG 2.0: the expertise effect

Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility
Automatic web accessibility metrics: Where we are and where we can go

Interacting with Computers
An investigation of tool support for accessibility assessment throughout the development process of web sites

Journal of Web Engineering
Enriching information retrieval results with web accessibility measurement

Journal of Web Engineering
Barriers common to mobile and disabled web users

Interacting with Computers
Test case description language (TCDL): test case metadata for conformance evaluation

ICCHP'06 Proceedings of the 10th international conference on Computers Helping People with Special Needs
Test case management tools for accessibility testing

ICCHP'06 Proceedings of the 10th international conference on Computers Helping People with Special Needs
Is accessibility conformance an elusive property? A study of validity and reliability of WCAG 2.0

ACM Transactions on Accessible Computing (TACCESS)
The design of RIA accessibility evaluation tool

Advances in Engineering Software
Benchmarking web accessibility evaluation tools: measuring the harm of sole reliance on automated tests

Proceedings of the 10th International Cross-Disciplinary Conference on Web Accessibility
Web accessibility snapshot: an effort to reveal coding guidelines conformance

Proceedings of the 10th International Cross-Disciplinary Conference on Web Accessibility

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper claims that effectiveness of automatic tools for evaluating web site accessibility has to be itself evaluated, given the increasingly important role that these tools play. The paper presents a comparison method for a pair of tools that takes into account correctness, completeness and specificity in supporting the task of assessing the conformance of a web site with respect to established guidelines. The paper presents data acquired during a case study based on comparing LIFT Machine with Bobby. The data acquired from the case study is used to assess the strengths and weaknesses of the comparison method. The conclusion is that even though there is room for improvement of the method, it is already capable of providing accurate and reliable conclusions.