Language determination: natural language processing from scanned document images

  • Authors:
  • Penelope Sibun;A. Lawrence Spitz

  • Affiliations:
  • Fuji Xerox Palo Alto Laboratory, Palo Alto, CA;Fuji Xerox Palo Alto Laboratory, Palo Alto, CA

  • Venue:
  • ANLC '94 Proceedings of the fourth conference on Applied natural language processing
  • Year:
  • 1994

Quantified Score

Hi-index 0.00

Visualization

Abstract

Many documents are available to a computer only as images from paper. However, most natural language processing systems expect their input as character-coded text, which may be difficult or expensive to extract accurately from the page. We describe a method for converting a document image into character shape codes and word shape tokens. We believe that this representation, which is both cheap and robust, is sufficient for many NLP tasks. In this paper, we show that the representation is sufficient for determining which of 23 languages the document is written in, using only a small number of features, with greater than 90% accuracy overall.