User interface evaluation in the real world: a comparison of four techniques
CHI '91 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Usability testing vs. heuristic evaluation: was there a contest?
ACM SIGCHI Bulletin
Comparison of empirical testing and walkthrough methods in user interface evaluation
CHI '92 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
A mathematical model of the finding of usability problems
INTERCHI '93 Proceedings of the INTERCHI '93 conference on Human factors in computing systems
INTERCHI '93 Proceedings of the INTERCHI '93 conference on Human factors in computing systems
Adding multimedia collections to the Dexter Model
ECHT '94 Proceedings of the 1994 ACM European conference on Hypermedia technology
ACM Transactions on Computer-Human Interaction (TOCHI)
Usability studies of WWW sites: heuristic evaluation vs. laboratory testing
SIGDOC '97 Proceedings of the 15th annual international conference on Computer documentation
A comparison of usability techniques for evaluating design
DIS '97 Proceedings of the 2nd conference on Designing interactive systems: processes, practices, methods, and techniques
Pushing reuse in hypermedia design: golden rules, design patterns and constructive templates
Proceedings of the ninth ACM conference on Hypertext and hypermedia : links, objects, time and space---structure in hypermedia systems: links, objects, time and space---structure in hypermedia systems
Proceedings of the ninth ACM conference on Hypertext and hypermedia : links, objects, time and space---structure in hypermedia systems: links, objects, time and space---structure in hypermedia systems
Using while moving: HCI issues in fieldwork environments
ACM Transactions on Computer-Human Interaction (TOCHI) - Special issue on human-computer interaction with mobile systems
A comparison of usage evaluation and inspection methods for assessing groupware usability
GROUP '01 Proceedings of the 2001 International ACM SIGGROUP Conference on Supporting Group Work
Proceedings of the second Nordic conference on Human-computer interaction
Evaluating usability methods: why the current literature fails the practitioner
interactions - The digital muse: HCI in support of creativity
Evaluating Educational Software Environments
ICCE '02 Proceedings of the International Conference on Computers in Education
Comparing usability problems and redesign proposals as input to practical systems development
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Current practice in measuring usability: Challenges to usability studies and research
International Journal of Human-Computer Studies
Systematic evaluation of e-learning systems: an experimental validation
Proceedings of the 4th Nordic conference on Human-computer interaction: changing roles
A comparative study of two usability evaluation methods using a web-based e-learning application
Proceedings of the 2007 annual research conference of the South African institute of computer scientists and information technologists on IT research in developing countries
Guidelines for Eliciting Usability Functionalities
IEEE Transactions on Software Engineering
Metaphors of human thinking for usability inspection and design
ACM Transactions on Computer-Human Interaction (TOCHI)
Damaged merchandise? a review of experiments that compare usability evaluation methods
Human-Computer Interaction
International Journal of Human-Computer Studies
Editorial: Using e-learning design patterns to augment learners' experiences
Computers in Human Behavior
The Effect of Brand on the Evaluation of Websites
INTERACT '09 Proceedings of the 12th IFIP TC 13 International Conference on Human-Computer Interaction: Part II
A holistic approach to the evaluation of e-learning systems
UAHCI'07 Proceedings of the 4th international conference on Universal access in human-computer interaction: applications and services
A case study of using eye tracking techniques to evaluate the usability of e-learning courses
International Journal of Learning Technology
Gamers as usability evaluators: a study in the domain of virtual worlds
Proceedings of the 11th Brazilian Symposium on Human Factors in Computing Systems
An EUD Approach to the Design of Educational Games
International Journal of Distance Education Technologies
Critérios para Identificação do Foco de Métodos de Avaliação para Sistemas Colaborativos
Proceedings of the X Brazilian Symposium in Collaborative Systems
| Hi-index | 0.00 |
Evaluating e-learning systems is a complex activity which requires considerations of several criteria addressing quality in use as well as educational quality. Heuristic evaluation is a widespread method for usability evaluation, yet its output is often prone to subjective variability, primarily due to the generality of many heuristics. This paper presents the pattern-based (PB) inspection, which aims at reducing this drawback by exploiting a set of evaluation patterns to systematically drive inspectors in their evaluation activities. The application of PB inspection to the evaluation of e-learning systems is reported in this paper together with a study that compares this method to heuristic evaluation and user testing. The study involved 73 novice evaluators and 25 end users, who evaluated an e-learning application using one of the three techniques. The comparison metric was defined along six major dimensions, covering concepts of classical test theory and pragmatic aspects of usability evaluation. The study showed that evaluation patterns, capitalizing on the reuse of expert evaluators know-how, provide a systematic framework which reduces reliance on individual skills, increases inter-rater reliability and output standardization, permits the discovery of a larger set of different problems and decreases evaluation cost. Results also indicated that evaluation in general is strongly dependent on the methodological apparatus as well as on judgement bias and individual preferences of evaluators, providing support to the conceptualisation of interactive quality as a subjective judgement, recently brought forward by the UX research agenda.