Design for conversation: lessons from Cognoter
International Journal of Man-Machine Studies - Computer-supported cooperative work and groupware. Part 1
Nonvisual presentation of graphical user interfaces: contrasting two approaches
CHI '94 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Where the action is: the foundations of embodied interaction
Where the action is: the foundations of embodied interaction
MacVisSTA: a system for multimodal analysis
Proceedings of the 6th international conference on Multimodal interfaces
Assembling the senses: towards the design of cooperative interfaces for visually impaired users
CSCW '04 Proceedings of the 2004 ACM conference on Computer supported cooperative work
MultiVis: improving access to visualisations for visually impaired people
CHI '06 Extended Abstracts on Human Factors in Computing Systems
Tac-tiles: multimodal pie charts for visually impaired users
Proceedings of the 4th Nordic conference on Human-computer interaction: changing roles
Sensory substitution using tactile pin arrays: human factors, technology and applications
Signal Processing - Special section: Multimodal human-computer interfaces
Flow in games (and everything else)
Communications of the ACM
The body surface as a communication system: The state of the art after 50 years
Presence: Teleoperators and Virtual Environments
Proceedings of the 1st international conference on PErvasive Technologies Related to Assistive Environments
Multimodal vision glove for touchscreens
Proceedings of the 10th international ACM SIGACCESS conference on Computers and accessibility
Fun to develop embodied skill: how games help the blind to understand pointing
Proceedings of the 3rd International Conference on PErvasive Technologies Related to Assistive Environments
MLMI'05 Proceedings of the Second international conference on Machine Learning for Multimodal Interaction
| Hi-index | 0.00 |
This paper presents research that shows that a high degree of skilled performance is required for multimodal discourse support. We discuss how students who are blind or visually impaired (SBVI) were able to understand the instructor's pointing gestures during planar geometry and trigonometry classes. For that, the SBVI must attend to the instructor's speech and have simultaneous access to the instructional graphic material, and to the where the instructor is pointing. We developed the Haptic Deictic System - HDS, capable of tracking the instructor's pointing and informing the SBVI, through a haptic glove, where she needs to move her hand understand the instructor's illustration-augmented discourse. Several challenges had to be overcome before the SBVI were able to engage in fluid multimodal discourse with the help of the HDS. We discuss how such challenges were addressed with respect to perception and discourse (especially to mathematics instruction).