Programming by voice: a hands-free approach for motorically challenged children

Authors:
Amber Wagner;Ramaraju Rudraraju;Srinivasa Datla;Avishek Banerjee;Mandar Sudame;Jeff Gray
Affiliations:
University of Alabama, Tuscaloosa, Alabama, USA;UAB, Birmingham, Alabama, USA;UAB, Birmingham, Alabama, USA;UAB, Birmingham, Alabama, USA;UAB, Birmingham, Alabama, USA;University of Alabama, Tuscaloosa, Alabama, USA
Venue:
CHI '12 Extended Abstracts on Human Factors in Computing Systems
Year:
2012

Citing 4
Cited 2

Web-Application Development Using the Model/View/Controller Design Pattern

EDOC '01 Proceedings of the 5th IEEE International Conference on Enterprise Distributed Object Computing
Automating Change Evolution in Model-Driven Engineering

Computer
Longitudinal study of people learning to use continuous voice-based cursor control

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Ability-Based Design: Concept, Principles and Examples

ACM Transactions on Accessible Computing (TACCESS)

Automation of VUIto GUI mapping

CHI '13 Extended Abstracts on Human Factors in Computing Systems
Observations of first learners with different capabilities (abstract only)

Proceedings of the 45th ACM technical symposium on Computer science education

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper introduces a voice-driven tool applied to an Initial Programming Environment (IPE), which gives motorically challenged individuals the opportunity to learn programming skills; in particular, our project allows programming by voice within Scratch. Although the native Scratch environment allows users to create a program by arranging graphical blocks logically, such visual languages are completely dependent on the use of a mouse and keyboard. This modality of interaction limits users based on physical abilities. Our solution is a tool, called Myna, which is a voice-driven Java application executed parallel to Scratch. Myna processes voice commands from the user, interprets those commands according to a pre-defined grammar, and simulates synonymous actions of a mouse and keyboard within Scratch. The resulting environment assists those with a motor disability (particularly young children) in learning the joy of programming. This extended abstract describes the motivation behind the project, a technical description of Myna, and defines the current work in progress.