A neural probabilistic language model
The Journal of Machine Learning Research
Unsupervised deduplication using cross-field dependencies
Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining
Separating Precision and Mean in Dirichlet-Enhanced High-Order Markov Models
ECML '07 Proceedings of the 18th European conference on Machine Learning
Modeling Topic and Role Information in Meetings Using the Hierarchical Dirichlet Process
MLMI '08 Proceedings of the 5th international workshop on Machine Learning for Multimodal Interaction
A stochastic memoizer for sequence data
ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
Smoothing a tera-word language model
HLT-Short '08 Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies: Short Papers
Sampling alignment structure under a Bayesian translation model
EMNLP '08 Proceedings of the Conference on Empirical Methods in Natural Language Processing
Shared logistic normal distributions for soft parameter tying in unsupervised grammar induction
NAACL '09 Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics
Bayesian unsupervised word segmentation with nested Pitman-Yor language modeling
ACL '09 Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 1 - Volume 1
Unsupervised and constrained Dirichlet process mixture models for verb clustering
GEMS '09 Proceedings of the Workshop on Geometrical Models of Natural Language Semantics
Using prosodic features in language models for meetings
MLMI'07 Proceedings of the 4th international conference on Machine learning for multimodal interaction
Hierarchical pitman-yor language model for information retrieval
Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval
Topic models with power-law using Pitman-Yor process
Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining
A Bayesian method for robust estimation of distributional similarities
ACL '10 Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics
Active learning for constrained Dirichlet process mixture models
GEMS '10 Proceedings of the 2010 Workshop on GEometrical Models of Natural Language Semantics
Training continuous space language models: some practical issues
EMNLP '10 Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing
Unsupervised induction of tree substitution grammars for dependency parsing
EMNLP '10 Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing
Nonparametric word segmentation for machine translation
COLING '10 Proceedings of the 23rd International Conference on Computational Linguistics
Hierarchical Bayesian language models for conversational speech recognition
IEEE Transactions on Audio, Speech, and Language Processing
Communications of the ACM
ACM Transactions on Modeling and Computer Simulation (TOMACS)
An unsupervised model for joint phrase alignment and extraction
HLT '11 Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1
A hierarchical Pitman-Yor process HMM for unsupervised part of speech induction
HLT '11 Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1
A Bayesian model for unsupervised semantic parsing
HLT '11 Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1
An empirical investigation of discounting in cross-domain language models
HLT '11 Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers - Volume 2
Insertion operator for Bayesian tree substitution grammars
HLT '11 Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers - Volume 2
Producing Power-Law Distributions and Damping Word Frequencies with Two-Stage Language Models
The Journal of Machine Learning Research
Sampling table configurations for the hierarchical poisson-dirichlet process
ECML PKDD'11 Proceedings of the 2011 European conference on Machine learning and knowledge discovery in databases - Volume Part I
A new unsupervised approach to word segmentation
Computational Linguistics
Distance Dependent Chinese Restaurant Processes
The Journal of Machine Learning Research
Discovering morphological paradigms from plain text using a Dirichlet process mixture model
EMNLP '11 Proceedings of the Conference on Empirical Methods in Natural Language Processing
Proteome coverage prediction for integrated proteomics datasets
RECOMB'10 Proceedings of the 14th Annual international conference on Research in Computational Molecular Biology
Mr. LDA: a flexible large scale topic modeling package using variational inference in MapReduce
Proceedings of the 21st international conference on World Wide Web
The latent words language model
Computer Speech and Language
Mixtures of Gaussian wells: Theory, computation, and application
Computational Statistics & Data Analysis
Technical term recognition with semi-supervised learning using hierarchical bayesian language models
NLDB'12 Proceedings of the 17th international conference on Applications of Natural Language Processing and Information Systems
IEA/AIE'12 Proceedings of the 25th international conference on Industrial Engineering and Other Applications of Applied Intelligent Systems: advanced research in applied artificial intelligence
A scalable distributed syntactic, semantic, and lexical language model
Computational Linguistics
Hierarchical Bayesian language modelling for the linguistically informed
EACL '12 Proceedings of the Student Research Workshop at the 13th Conference of the European Chapter of the Association for Computational Linguistics
Continuous space translation models with neural networks
NAACL HLT '12 Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
A hierarchical dirichlet process model for joint part-of-speech and morphology induction
NAACL HLT '12 Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Unsupervised part of speech inference with particle filters
WILS '12 Proceedings of the NAACL-HLT Workshop on the Induction of Linguistic Structure
Bootstrapping a unified model of lexical and phonetic acquisition
ACL '12 Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers - Volume 1
Bayesian symbol-refined tree substitution grammars for syntactic parsing
ACL '12 Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers - Volume 1
Measuring the influence of long range dependencies with neural network language models
WLM '12 Proceedings of the NAACL-HLT 2012 Workshop: Will We Ever Really Replace the N-gram Model? On the Future of Language Modeling for HLT
A phrase-discovering topic model using hierarchical Pitman-Yor processes
EMNLP-CoNLL '12 Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning
A bayesian model for learning SCFGs with discontiguous rules
EMNLP-CoNLL '12 Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning
Modelling sequential text with an adaptive topic model
EMNLP-CoNLL '12 Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning
Exact sampling and decoding in high-order hidden Markov models
EMNLP-CoNLL '12 Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning
Unsupervised bayesian part of speech inference with particle gibbs
ECML PKDD'12 Proceedings of the 2012 European conference on Machine Learning and Knowledge Discovery in Databases - Volume Part I
Large-scale statistical modeling of motion patterns: a Bayesian nonparametric approach
Proceedings of the Eighth Indian Conference on Computer Vision, Graphics and Image Processing
Margin-maximizing classification of sequential data with infinitely-long temporal dependencies
Expert Systems with Applications: An International Journal
Smoothing for bracketing induction
IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
Proceedings of the 7th ACM international conference on Web search and data mining
Bayesian Constituent Context Model for Grammar Induction
IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP)
Genre-Based Music Language Modeling with Latent Hierarchical Pitman-Yor Process Allocation
IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP)
Intelligent Cooperative Control for Urban Tracking
Journal of Intelligent and Robotic Systems
| Hi-index | 0.02 |
We propose a new hierarchical Bayesian n-gram model of natural languages. Our model makes use of a generalization of the commonly used Dirichlet distributions called Pitman-Yor processes which produce power-law distributions more closely resembling those in natural languages. We show that an approximation to the hierarchical Pitman-Yor language model recovers the exact formulation of interpolated Kneser-Ney, one of the best smoothing methods for n-gram language models. Experiments verify that our model gives cross entropy results superior to interpolated Kneser-Ney and comparable to modified Kneser-Ney.