C4.5: programs for machine learning
C4.5: programs for machine learning
A visual medium for programmatic control of interactive applications
Proceedings of the SIGCHI conference on Human Factors in Computing Systems
Programming by example: visual generalization in programming by example
Communications of the ACM
Interaction techniques for ambiguity resolution in recognition-based interfaces
UIST '00 Proceedings of the 13th annual ACM symposium on User interface software and technology
Accuracy measures for evaluating computer pointing devices
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
More than dotting the i's --- foundations for crossing-based interfaces
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Machine Learning
Computer Vision: A Modern Approach
Computer Vision: A Modern Approach
Cursor measures for motion-impaired computer users
Proceedings of the fifth international ACM conference on Assistive technologies
Usability in the real world: assessing medical information technologies in patient's homes
Journal of Biomedical Informatics - Patient safety
A comparison of static, adaptive, and adaptable menus
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Developing steady clicks:: a method of cursor assistance for people with motor impairments
Proceedings of the 8th international ACM SIGACCESS conference on Computers and accessibility
Data Mining
Disruption and recovery of computing tasks: field study, analysis, and directions
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Understanding pointing problems in real world computing environments
Proceedings of the 10th international ACM SIGACCESS conference on Computers and accessibility
Sikuli: using GUI screenshots for search and automation
Proceedings of the 22nd annual ACM symposium on User interface software and technology
Prefab: implementing advanced behaviors using pixel-based reverse engineering of interface structure
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
UIMarks: quick graphical interaction with specific targets
UIST '10 Proceedings of the 23nd annual ACM symposium on User interface software and technology
Ability-Based Design: Concept, Principles and Examples
ACM Transactions on Accessible Computing (TACCESS)
Content and hierarchy in pixel-based methods for reverse engineering interface structure
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
CHI '11 Extended Abstracts on Human Factors in Computing Systems
Investigating the accessibility of program selection menus of a digital TV interface
HCII'11 Proceedings of the 14th international conference on Human-computer interaction: users and applications - Volume Part IV
Using graphical representation of user interfaces as visual references
Proceedings of the 24th annual ACM symposium adjunct on User interface software and technology
Creating contextual help for GUIs using screenshots
Proceedings of the 24th annual ACM symposium on User interface software and technology
Associating the visual representation of user interfaces with their internal structures and metadata
Proceedings of the 24th annual ACM symposium on User interface software and technology
Menu controller: making existing software more accessible for people with motor impairments
Proceedings of the 4th International Conference on PErvasive Technologies Related to Assistive Environments
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Accurate measurements of pointing performance from in situ observations
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Waken: reverse engineering usage information and interface structure from software videos
Proceedings of the 25th annual ACM symposium on User interface software and technology
Patina: dynamic heatmaps for visualizing application usage
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
| Hi-index | 0.00 |
Information about the location and size of the targets that users interact with in real world settings can enable new innovations in human performance assessment and soft-ware usability analysis. Accessibility APIs provide some information about the size and location of targets. How-ever this information is incomplete because it does not sup-port all targets found in modern interfaces and the reported sizes can be inaccurate. These accessibility APIs access the size and location of targets through low-level hooks to the operating system or an application. We have developed an alternative solution for target identification that leverages visual affordances in the interface, and the visual cues produced as users interact with targets. We have used our novel target identification technique in a hybrid solution that combines machine learning, computer vision, and accessibility API data to find the size and location of targets users select with 89% accuracy. Our hybrid approach is superior to the performance of the accessibility API alone: in our dataset of 1355 targets covering 8 popular applications, only 74% of the targets were correctly identified by the API alone.