Twenty Years of Document Image Analysis in PAMI
IEEE Transactions on Pattern Analysis and Machine Intelligence
Handwriting Recognition Using Position Sensitive Letter N-Gram Matching
ICDAR '03 Proceedings of the Seventh International Conference on Document Analysis and Recognition - Volume 1
Characteristics and retrieval effectiveness of n-gram string similarity matching on Malay documents
ACACOS'11 Proceedings of the 10th WSEAS international conference on Applied computer and applied computational science
| Hi-index | 0.14 |
The inherent statistical characteristics, including the economy, entropy, and redundancy, of a very large set containing 93681 words from the Shorter Oxford English Dictionary is investigated. Analytical n-gram statistics are also presented for applications in natural language understanding, text processing, test compression, error detection and correction, and speech synthesis and recognition. Experimental results show how the distribution of n-grams in the dictionary varies from the ideal as n increases from 2 to 5, that is, from bigrams to pentagrams; it is shown that the corresponding redundancy increases from 0.1067 to 0.3409. The results are of interest because, (1) the dictionary provides a finite list for deterministic analyses, (2) each entry (word) appears once, compared to free-running text where words are repeated, and (3) all entries, even rarely occurring ones, have equal weight.