Learning internal representations by error propagation
Parallel distributed processing: explorations in the microstructure of cognition, vol. 1
Statistical methods for speech recognition
Statistical methods for speech recognition
Neural Networks for Pattern Recognition
Neural Networks for Pattern Recognition
Comparing Simple Recurrent Networks and n-Grams in a Large Corpus
Applied Intelligence
A neural probabilistic language model
The Journal of Machine Learning Research
| Hi-index | 0.00 |
In language engineering, language models are employed in order to improve system performance. These language models are usually N-gram models which are estimated from large text databases using the occurrence frequencies of these N-grams. An alternative to conventional frequency-based estimation of N-gram probabilities consists in using neural networks to this end. These "connectionist N-gram models", although their training is very time-consuming, present a pair of interesting advantages over the conventional approach: networks provide an implicit smoothing in their estimations and the number of free parameters does not grow exponentially with N. Some experimental works provide empirical evidence on the capability of multilayer perceptrons and simple recurrent networks to emulate N-gram models, and proposes new directions for extending neural networks-based language models.