Benchmarking fusion engines of multimodal interactive systems

Authors:
Bruno Dumas;Rolf Ingold;Denis Lalanne
Affiliations:
University of Fribourg, Fribourg, Switzerland;University of Fribourg, Friboug, Switzerland;University of Fribourg, Fribourg, Switzerland
Venue:
Proceedings of the 2009 international conference on Multimodal interfaces
Year:
2009

Citing 13
Cited 4

A design space for multimodal systems: concurrent processing and data fusion

CHI '93 Proceedings of the INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems
QuickSet: multimodal interaction for distributed applications

MULTIMEDIA '97 Proceedings of the fifth ACM international conference on Multimedia
Ten myths of multimodal interaction

Communications of the ACM
“Put-that-there”: Voice and gesture at the graphics interface

SIGGRAPH '80 Proceedings of the 7th annual conference on Computer graphics and interactive techniques
A Real-Time Framework for Natural Multimodal Interaction with Large Screen Displays

ICMI '02 Proceedings of the 4th IEEE International Conference on Multimodal Interfaces
Toward a theory of organized multimodal integration patterns during human-computer interaction

Proceedings of the 5th international conference on Multimodal interfaces
A framework for rapid development of multimodal interfaces

Proceedings of the 5th international conference on Multimodal interfaces
Unification-based multimodal integration

ACL '98 Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics
Implementation and evaluation of a constraint-based multimodal fusion system for speech and 3D pointing gestures

Proceedings of the 6th international conference on Multimodal interfaces
ICARE software components for rapidly developing multimodal interfaces

Proceedings of the 6th international conference on Multimodal interfaces
Strengths and weaknesses of software architectures for the rapid creation of tangible and multimodal interfaces

Proceedings of the 2nd international conference on Tangible and embedded interaction
Designing the user interface for multimodal speech and pen-based gesture applications: state-of-the-art systems and future research directions

Human-Computer Interaction
An open source workbench for prototyping multimodal interactions based on off-the-shelf heterogeneous components

Proceedings of the 1st ACM SIGCHI symposium on Engineering interactive computing systems

Adaptive multimodal fusion

UAHCI'11 Proceedings of the 6th international conference on Universal access in human-computer interaction: design for all and eInclusion - Volume Part I
Fusion in multimodal interactive systems: an HMM-based algorithm for user-induced adaptation

Proceedings of the 4th ACM SIGCHI symposium on Engineering interactive computing systems
Using the transferable belief model for multimodal input fusion in companion systems

MPRSS'12 Proceedings of the First international conference on Multimodal Pattern Recognition of Social Signals in Human-Computer-Interaction
CoWME: a general framework to evaluate cognitive workload during multimodal interaction

Proceedings of the 15th ACM on International conference on multimodal interaction

Quantified Score

Hi-index	0.00

Visualization

Abstract

This article proposes an evaluation framework to benchmark the performance of multimodal fusion engines. The paper first introduces different concepts and techniques associated with multimodal fusion engines and further surveys recent implementations. It then discusses the importance of evaluation as a mean to assess fusion engines, not only from the user perspective, but also at a performance level. The article further proposes a benchmark and a formalism to build testbeds for assessing multimodal fusion engines. In its last section, our current fusion engine and the associated system HephaisTK are evaluated thanks to the evaluation framework proposed in this article. The article concludes with a discussion on the proposed quantitative evaluation, suggestions to build useful testbeds, and proposes some future improvements.