Building visual language parsers
CHI '91 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Multimodal interfaces for dynamic interactive maps
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Integration and synchronization of input modes during multimodal human-computer interaction
Proceedings of the ACM SIGCHI Conference on Human factors in computing systems
Recent advances of grammatical inference
Theoretical Computer Science - Special issue on algorithmic learning theory
The syntactic process
Usability Testing and Research
Usability Testing and Research
A Version Space Approach to Learning Context-free Grammars
Machine Learning
Synthesizing Context Free Grammars from Sample Strings Based on Inductive CYK Algorithm
ICGI '00 Proceedings of the 5th International Colloquium on Grammatical Inference: Algorithms and Applications
Incremental Learning of Context Free Grammars
ICGI '02 Proceedings of the 6th International Colloquium on Grammatical Inference: Algorithms and Applications
Controlling Content Realization with Functional Unification Grammars
Proceedings of the 6th International Workshop on Natural Language Generation: Aspects of Automated Natural Language Generation
Building a large annotated corpus of English: the penn treebank
Computational Linguistics - Special issue on using large corpora: II
Unification-based multimodal integration
ACL '98 Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics
Finite-state multimodal integration and understanding
Natural Language Engineering
Multi-modal combinatory categorial grammar
EACL '03 Proceedings of the tenth conference on European chapter of the Association for Computational Linguistics - Volume 1
Multimodal fusion: a new hybrid strategy for dialogue systems
Proceedings of the 8th international conference on Multimodal interfaces
An Efficient Multimodal Language Processor for Parallel Input Strings in Multimodal Input Fusion
ICSC '07 Proceedings of the International Conference on Semantic Computing
UI on the Fly: generating a multimodal user interface
HLT-NAACL-Short '04 Proceedings of HLT-NAACL 2004: Short Papers
Unification-based grammars and tabular parsing for graphical languages
Journal of Visual Languages and Computing
A usability-evaluation metric based on a soft-computing approach
IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans
Multimodal retrieval with relevance feedback based on genetic programming
Multimedia Tools and Applications
| Hi-index | 0.00 |
This paper presents a new multimodal grammar generation system (MGGS) that allows defining a multimodal grammar in a very easy and intuitive way, overcoming the difficulties arising from the textual description of grammar production rules. The novelty of the proposed approach relies in adopting a "by example" paradigm to define a multimodal grammar. This paradigm consists in providing concrete examples of multimodal sentences, and enabling a grammar inference algorithm to automatically generate the grammar rules to parse those sentences. The main contribution of this approach is that it is general enough to be applicable for whatever modalities and in whichever domains. Moreover, the use of the grammar inference algorithm for automating the grammar writing and updating processes, reduces costs of the grammar development and maintenance. In the first part of this paper, we present the MGGS, describing in detail the methodology we have implemented in this system. More specifically, the multimodal attribute grammar and the grammar inference algorithm are illustrated. The second part describes the experiment aimed at observing participants while interacting with the system in order to provide some real data about the usability of the system. Results of this experiment showed that the proposed system facilitates the grammar definition and updating, and it is more suitable also for nonexpert people as it does not require the learning of the grammar notation.