Multimodal mathematical expressions recognition: case of speech and handwriting

Authors:
Sofiane Medjkoune;Harold Mouchere;Simon Petitrenaud;Christian Viard-Gaudin
Affiliations:
IRCCyN, LUNAM University, University of Nantes, France,LIUM, LUNAM University, University of Le Mans, France;IRCCyN, LUNAM University, University of Nantes, France;LIUM, LUNAM University, University of Le Mans, France;IRCCyN, LUNAM University, University of Nantes, France
Venue:
HCI'13 Proceedings of the 15th international conference on Human-Computer Interaction: interaction modalities and techniques - Volume Part IV
Year:
2013

Citing 7
Cited 0

The transferable belief model

Artificial Intelligence
Multimodal human-computer interaction: A survey

Computer Vision and Image Understanding
Conjunctive and disjunctive combination of belief functions induced by nondistinct bodies of evidence

Artificial Intelligence
Towards Handwritten Mathematical Expression Recognition

ICDAR '09 Proceedings of the 2009 10th International Conference on Document Analysis and Recognition
Multimodal Signal Processing: Theory and applications for human-computer interaction

Multimodal Signal Processing: Theory and applications for human-computer interaction
HAMEX - A Handwritten and Audio Dataset of Mathematical Expressions

ICDAR '11 Proceedings of the 2011 International Conference on Document Analysis and Recognition
ICFHR 2012 Competition on Recognition of On-Line Mathematical Expressions (CROHME 2012)

ICFHR '12 Proceedings of the 2012 International Conference on Frontiers in Handwriting Recognition

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this work, we propose to combine two modalities, handwriting and speech, to build a mathematical expression recognition system. Based on two sub-systems which process each modality, we explore various fusion methods to resolve ambiguities which naturally occur independently. The results that are reported on the HAMEX bimodal database show an improvement with respect to a mono-modal based system.