A global learning approach for an online handwritten mathematical expression recognition system

  • Authors:

  • Ahmad-Montaser Awal;Harold Mouchère;Christian Viard-Gaudin

  • Affiliations:

  • Laboratoire d'étude des Mécanismes Cognitifs, Université Lumière Lyon2, Lyon, France;LUNAM Université, Université de Nantes, IRCCyN/IVC, Nantes, France;LUNAM Université, Université de Nantes, IRCCyN/IVC, Nantes, France

  • Venue:
  • Pattern Recognition Letters
  • Year:
  • 2014

Quantified Score

Hi-index 0.10

Visualization

Abstract

Despite the recent advances in handwriting recognition, handwritten two-dimensional (2D) languages are still a challenge. Electrical schemas, chemical equations and mathematical expressions (MEs) are examples of such 2D languages. In this case, the recognition problem is particularly difficult due to the two dimensional layout of the language. This paper presents an online handwritten mathematical expression recognition system that handles mathematical expression recognition as a simultaneous optimization of expression segmentation, symbol recognition, and 2D structure recognition under the restriction of a mathematical expression grammar. The originality of the approach is a global strategy allowing learning mathematical symbols and spatial relations directly from complete expressions. A new contextual modeling is proposed for combining syntactic and structural information. Those models are used to find the most likely combination of segmentation/recognition hypotheses proposed by a 2D segmentation scheme. Thus, models are based on structural information concerning the symbol layout. The system is tested with a new public database of mathematical expressions which was used in the CHROME competition. We have also produced a large base of semi-synthetic expressions which are used to train and test the global learning approach. We obtain very promising results on both synthetic and real expressions databases, as well as in the recent CHROME competition.