Probabilistic logic with minimum perplexity: Application to language modeling

Authors:
Djamel Bouchaffra
Affiliations:
Department of Computer Science, Oakland University, 131 Dodge Hall, Rochester, MI 48309, USA
Venue:
Pattern Recognition
Year:
2005

Citing 8
Cited 0

Probabilistic logic

Artificial Intelligence
Class-based n-gram models of natural language

Computational Linguistics
Artificial intelligence: a modern approach

Artificial intelligence: a modern approach
Foundations of statistical natural language processing

Foundations of statistical natural language processing
Postprocessing of Recognized Strings Using Nonstationary Markovian Models

IEEE Transactions on Pattern Analysis and Machine Intelligence
Modified Kneser-Ney Smoothing of n-gram Models

Modified Kneser-Ney Smoothing of n-gram Models
Similarity-based estimation of word cooccurrence probabilities

ACL '94 Proceedings of the 32nd annual meeting on Association for Computational Linguistics
The zero-frequency problem: estimating the probabilities of novel events in adaptive text compression

IEEE Transactions on Information Theory

Quantified Score

Hi-index	0.01

Visualization

Abstract

Any statistical model based on training encounters sparse configurations. These data are those that have not been encountered (or seen) during the training phase. This inherent problem is a big challenge to many scientific communities. The statistical estimation of rare events is usually performed through the maximum likelihood (ML) criterion. However, it is well-known that the ML estimator is sensitive to extreme values that is therefore non-reliable. To answer this challenge, we propose a novel approach based on probabilistic logic (PL) and the minimal perplexity criterion. In our approach, configurations are considered as probabilistic events such as predicates related through logical connectors. Our method was applied to estimate word trigram probability values from a corpus. Experimental results conducted on several test sets show that the PL method with minimal perplexity has outperformed both the ''Absolute Discounting'', and the ''Good-Turing Discounting'' techniques.