Educational Video Understanding: Mapping Handwritten Text to Textbook Chapters

  • Authors:
  • Lijun Tang;John R. Kender

  • Affiliations:
  • Columbia University, NY;Columbia University, NY

  • Venue:
  • ICDAR '05 Proceedings of the Eighth International Conference on Document Analysis and Recognition
  • Year:
  • 2005

Quantified Score

Hi-index 0.00

Visualization

Abstract

Handwritten text frames appear frequently in educational videos and can be used as an important cue for semantic analysis of educational videos. We detect text frames using a motion pattern analyzing algorithm. Then we extract binary handwritten word images from the text frames in various visual formats: handwritten slides, electronic slides, handwriting on chalkboard, etc. We propose a handwritten word recognition method, using combined dynamic programming stroke-based character segmentation with optimal statistical handwritten character recognition. In parallel, we construct a small vocabulary from topic words taken from table-of-contents of course materials such as the course textbook. We use the handwritten word recognition results to query this table-of-contents structure, implemented as latent semantic analysis matrix operations. We are able to spot the most likely discussed chapters and topic words for each frame. We evaluate the overall approach on 12 videos of two courses, and the results are encouraging.