A practical part-of-speech tagger

Authors:
Doug Cutting;Julian Kupiec;Jan Pedersen;Penelope Sibun
Affiliations:
Xerox Palo Alto Research Center, Palo Alto, CA;Xerox Palo Alto Research Center, Palo Alto, CA;Xerox Palo Alto Research Center, Palo Alto, CA;Xerox Palo Alto Research Center, Palo Alto, CA
Venue:
ANLC '92 Proceedings of the third conference on Applied natural language processing
Year:
1992

Citing 9
Cited 137

Compilers: principles, techniques, and tools

Compilers: principles, techniques, and tools
Natural Language Modeling for Phoneme-to-Text Transcription

IEEE Transactions on Pattern Analysis and Machine Intelligence
Grammatical category disambiguation by statistical optimization

Computational Linguistics
Common LISP: the language (2nd ed.)

Common LISP: the language (2nd ed.)
Probabilistic models of short and long distance word dependencies in running text

HLT '89 Proceedings of the workshop on Speech and Natural Language
The art of computer programming, volume 3: (2nd ed.) sorting and searching

The art of computer programming, volume 3: (2nd ed.) sorting and searching
A stochastic parts program and noun phrase parser for unrestricted text

ANLC '88 Proceedings of the second conference on Applied natural language processing
Finite-state parsing and disambiguation

COLING '90 Proceedings of the 13th conference on Computational linguistics - Volume 2
Augmenting a hidden Markov model for phrase-dependent word tagging

HLT '89 Proceedings of the workshop on Speech and Natural Language

MURAX: a robust linguistic approach for question answering using an on-line encyclopedia

SIGIR '93 Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval
Metadata for mixed-media access

ACM SIGMOD Record
Galaxy of news: an approach to visualizing and understanding expansive news landscapes

UIST '94 Proceedings of the 7th annual ACM symposium on User interface software and technology
Automatic stochastic tagging of natural language texts

Computational Linguistics
Deterministic part-of-speech tagging with finite-state transducers

Computational Linguistics
NetSerf: using semantic knowledge to find Internet information archives

SIGIR '95 Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval
Learning morpho-lexical probabilities from an untagged corpus with an application to Hebrew

Computational Linguistics
Transformation-based error-driven learning and natural language processing: a case study in part-of-speech tagging

Computational Linguistics
Application-embedded retrieval from distributed free-text collections

AAAI '99/IAAI '99 Proceedings of the sixteenth national conference on Artificial intelligence and the eleventh Innovative applications of artificial intelligence conference innovative applications of artificial intelligence
A Machine Learning Approach to POS Tagging

Machine Learning
Learning to lemmatise slovene words

Learning language in logic
ILP in part-of-speech tagging — an overview

Learning language in logic
Accessing Foreign Languages with Compass

Machine Translation
Syllable-pattern-based unknown-morpheme segmentation and estimation for hybrid part-of-speech tagging of Korean

Computational Linguistics
Natural Language Processing and Digital Libraries

Information Extraction: Towards Scalable, Adaptable Systems
A Simple Spanish Part of Speech Tagger for Detection and Correction of Accentuation Error

TSD '99 Proceedings of the Second International Workshop on Text, Speech and Dialogue
Improved Learning for Hidden Markov Models Using Penalized Training

AICS '02 Proceedings of the 13th Irish International Conference on Artificial Intelligence and Cognitive Science
Part-of-Speech Tagging with Evolutionary Algorithms

CICLing '02 Proceedings of the Third International Conference on Computational Linguistics and Intelligent Text Processing
Using Multiattribute Prediction Suffix Graphs for Spanish Part-of-Speech Tagging

IDA '01 Proceedings of the 4th International Conference on Advances in Intelligent Data Analysis
Tagging with Small Training Corpora

IDA '01 Proceedings of the 4th International Conference on Advances in Intelligent Data Analysis
Automatic Structuring of Written Texts

TSD '99 Proceedings of the Second International Workshop on Text, Speech and Dialogue
Morphosyntactic Tagging of Slovene Using Progol

ILP '99 Proceedings of the 9th International Workshop on Inductive Logic Programming
Linguistic Processing of Biomedical Texts

PorTAL '02 Proceedings of the Third International Conference on Advances in Natural Language Processing
Exploitation of Unlabeled Sequences in Hidden Markov Models

IEEE Transactions on Pattern Analysis and Machine Intelligence
Tagging English text with a probabilistic model

Computational Linguistics
Adaptive multilingual sentence boundary disambiguation

Computational Linguistics
Automatic rule induction for unknown-word guessing

Computational Linguistics
Retrieving NASA problem reports: a case study in natural language information retrieval

Data & Knowledge Engineering - NLDB2002
Robustness beyond shallowness: incremental deep parsing

Natural Language Engineering
A natural language system for retrieval of captioned images

Natural Language Engineering
Extracting molecular binding relationships from biomedical text

ANLC '00 Proceedings of the sixth conference on Applied natural language processing
TnT: a statistical part-of-speech tagger

ANLC '00 Proceedings of the sixth conference on Applied natural language processing
Tagging accurately: don't guess if you know

ANLC '94 Proceedings of the fourth conference on Applied natural language processing
Does Baum-Welch re-estimation help taggers?

ANLC '94 Proceedings of the fourth conference on Applied natural language processing
Adaptive sentence boundary disambiguation

ANLC '94 Proceedings of the fourth conference on Applied natural language processing
A probabilistic model for text categorization: based on a single random variable with multiple values

ANLC '94 Proceedings of the fourth conference on Applied natural language processing
A simple rule-based part of speech tagger

ANLC '92 Proceedings of the third conference on Applied natural language processing
A maximum entropy approach to identifying sentence boundaries

ANLC '97 Proceedings of the fifth conference on Applied natural language processing
An annotation scheme for free word order languages

ANLC '97 Proceedings of the fifth conference on Applied natural language processing
Probabilistic and rule-based tagger of an inflective language: a comparison

ANLC '97 Proceedings of the fifth conference on Applied natural language processing
Unsupervised learning of part-of-speech guessing rules

Natural Language Engineering
Corpus-based method for automatic identification of support verbs for nominalizations

EACL '95 Proceedings of the seventh conference on European chapter of the Association for Computational Linguistics
Distributional part-of-speech tagging

EACL '95 Proceedings of the seventh conference on European chapter of the Association for Computational Linguistics
Tagging French: comparing a statistical and a constraint-based method

EACL '95 Proceedings of the seventh conference on European chapter of the Association for Computational Linguistics
A syntax-based part-of-speech analyser

EACL '95 Proceedings of the seventh conference on European chapter of the Association for Computational Linguistics
POS disambiguation and unknown word guessing with decision trees

EACL '99 Proceedings of the ninth conference on European chapter of the Association for Computational Linguistics
Morphological disambiguation by voting constraints

ACL '98 Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics
A flexible POS tagger using an automatically acquired language model

ACL '98 Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics
Comparing a linguistic and a stochastic tagger

ACL '98 Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics
Cross-language headline generation for Hindi

ACM Transactions on Asian Language Information Processing (TALIP)
Tagging English by path voting constraints

COLING '98 Proceedings of the 17th international conference on Computational linguistics - Volume 2
Combining stochastic and rule-based methods for disambiguation in agglutinative languages

COLING '98 Proceedings of the 17th international conference on Computational linguistics - Volume 1
Use of mutual information based character clusters in dictionary-less morphological analysis of Japanese

COLING '98 Proceedings of the 17th international conference on Computational linguistics - Volume 1
An algorithm for finding noun phrase correspondences in bilingual corpora

ACL '93 Proceedings of the 31st annual meeting on Association for Computational Linguistics
Tagset reduction without information loss

ACL '95 Proceedings of the 33rd annual meeting on Association for Computational Linguistics
Sense disambiguation using semantic relations and adjacency information

ACL '95 Proceedings of the 33rd annual meeting on Association for Computational Linguistics
Automatic alignment in parallel corpora

ACL '94 Proceedings of the 32nd annual meeting on Association for Computational Linguistics
Integrating multiple knowledge sources to disambiguate word sense: an exemplar-based approach

ACL '96 Proceedings of the 34th annual meeting on Association for Computational Linguistics
Fast parsing using pruning and grammar specialization

ACL '96 Proceedings of the 34th annual meeting on Association for Computational Linguistics
Extended models and tools for high-performance part-of-speech tagger

COLING '00 Proceedings of the 18th conference on Computational linguistics - Volume 1
Statistical morphological disambiguation for agglutinative languages

COLING '00 Proceedings of the 18th conference on Computational linguistics - Volume 1
A stochastic parser based on a structural word prediction model

COLING '00 Proceedings of the 18th conference on Computational linguistics - Volume 1
Content characterization using word shape tokens

COLING '94 Proceedings of the 15th conference on Computational linguistics - Volume 2
Representing information need with semantic relations

COLING '94 Proceedings of the 15th conference on Computational linguistics - Volume 2
Recognizing text genres with simple metrics using discriminant analysis

COLING '94 Proceedings of the 15th conference on Computational linguistics - Volume 2
Probabilistic tagging with feature structures

COLING '94 Proceedings of the 15th conference on Computational linguistics - Volume 1
Part-of-speech tagging with neural networks

COLING '94 Proceedings of the 15th conference on Computational linguistics - Volume 1
A stochastic Japanese morphological analyzer using a forward-DP backward-A* N-best search algorithm

COLING '94 Proceedings of the 15th conference on Computational linguistics - Volume 1
MULTEXT: Multilingual Text Tools and Corpora

COLING '94 Proceedings of the 15th conference on Computational linguistics - Volume 1
Improving part-of-speech tagging using lexicalized HMMs

Natural Language Engineering
Automatic acquisition of hyponyms from large text corpora

COLING '92 Proceedings of the 14th conference on Computational linguistics - Volume 2
Unsupervised learning of a rule-based Spanish Part of Speech tagger

COLING '96 Proceedings of the 16th conference on Computational linguistics - Volume 1
N-th order Ergodic Multigram HMM for modeling of languages without marked word boundaries

COLING '96 Proceedings of the 16th conference on Computational linguistics - Volume 1
Tagging and chunking with bigrams

COLING '00 Proceedings of the 18th conference on Computational linguistics - Volume 2
Linguistic indeterminacy as a source of errors in tagging

COLING '96 Proceedings of the 16th conference on Computational linguistics - Volume 2
Context-based spelling correction for Japanese OCR

COLING '96 Proceedings of the 16th conference on Computational linguistics - Volume 2
Tagging spoken language using written language statistics

COLING '96 Proceedings of the 16th conference on Computational linguistics - Volume 2
The interaction of domain knowledge and linguistic structure in natural language processing: interpreting hypernymic propositions in biomedical text

Journal of Biomedical Informatics - Special issue: Unified medical language system
Factor matrix text filtering and clustering: Research Articles

Journal of the American Society for Information Science and Technology
Applying co-training methods to statistical parsing

NAACL '01 Proceedings of the second meeting of the North American Chapter of the Association for Computational Linguistics on Language technologies
Independence and commitment: assumptions for rapid training and execution of rule-based POS taggers

ACL '00 Proceedings of the 38th Annual Meeting on Association for Computational Linguistics
A simple rule-based part of speech tagger

HLT '91 Proceedings of the workshop on Speech and Natural Language
A report of recent progress in transformation-based error-driven learning

HLT '94 Proceedings of the workshop on Human Language Technology
How to integrate linguistic information in FILES and generate feedback for grammar errors

STAR '01 Proceedings of the ACL 2001 Workshop on Sharing Tools and Resources - Volume 15
Unsupervised, corpus-based method for extending a biomedical terminology

BioMed '02 Proceedings of the ACL-02 workshop on Natural language processing in the biomedical domain - Volume 3
Exploring adjectival modification in biomedical discourse across two genres

BioMed '03 Proceedings of the ACL 2003 workshop on Natural language processing in biomedicine - Volume 13
Domain-specific language models and lexicons for tagging

Journal of Biomedical Informatics
The importance of the lexicon in tagging biological text

Natural Language Engineering
Identifying important concepts from medical documents

Journal of Biomedical Informatics
Toward unsupervised whole-corpus tagging

COLING '04 Proceedings of the 20th international conference on Computational Linguistics
Part of speech tagging in context

COLING '04 Proceedings of the 20th international conference on Computational Linguistics
Unsupervised large-vocabulary word sense disambiguation with graph-based algorithms for sequence data labeling

HLT '05 Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing
Learning morphological disambiguation rules for Turkish

HLT-NAACL '06 Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics
Morphological richness offsets resource demand- experiences in constructing a POS tagger for Hindi

COLING-ACL '06 Proceedings of the COLING/ACL on Main conference poster sessions
A probabilistic multimedia retrieval model and its evaluation

EURASIP Journal on Applied Signal Processing
Multi-candidate reduction: Sentence compression as a tool for document summarization tasks

Information Processing and Management: an International Journal
Exploiting redundancy in natural language to penetrate Bayesian spam filters

WOOT '07 Proceedings of the first USENIX workshop on Offensive Technologies
TaxaMiner: an experimentation framework for automated taxonomy bootstrapping

International Journal of Web and Grid Services
Part-of-speech tagging of modern hebrew text

Natural Language Engineering
Authorship attribution

Foundations and Trends in Information Retrieval
Boosted Bayesian network classifiers

Machine Learning
Improving Text Summarization Using Noun Retrieval Techniques

KES '08 Proceedings of the 12th international conference on Knowledge-Based Intelligent Information and Engineering Systems, Part II
Using target-language information to train part-of-speech taggers for machine translation

Machine Translation
CRF Models for Tamil Part of Speech Tagging and Chunking

ICCPOL '09 Proceedings of the 22nd International Conference on Computer Processing of Oriental Languages. Language Technology for the Knowledge-based Economy
Morphological Disambiguation of Turkish Text with Perceptron Algorithm

CICLing '07 Proceedings of the 8th International Conference on Computational Linguistics and Intelligent Text Processing
Part-of-Speech Tagging Using Word Probability Based on Category Patterns

CICLing '07 Proceedings of the 8th International Conference on Computational Linguistics and Intelligent Text Processing
Automatic part-of-speech tagging for Bengali: an approach for morphologically rich languages in a poor resource scenario

ACL '07 Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions
Creating a test corpus of clinical notes manually tagged for part-of-speech information

JNLPBA '04 Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and its Applications
Towards full automation of lexicon construction

CLS '04 Proceedings of the HLT-NAACL Workshop on Computational Lexical Semantics
Abstraction summarization for managing the biomedical research literature

CLS '04 Proceedings of the HLT-NAACL Workshop on Computational Lexical Semantics
Representations for category disambiguation

COLING '08 Proceedings of the 22nd International Conference on Computational Linguistics - Volume 1
Automation of treebank annotation

NeMLaP3/CoNLL '98 Proceedings of the Joint Conferences on New Methods in Language Processing and Computational Natural Language Learning
A preliminary look into the use of named entity information for bioscience text tokenization

HLT-SRWS '04 Proceedings of the Student Research Workshop at HLT-NAACL 2004
Robust ending guessing rules with application to Slavonic languages

ROMAND '04 Proceedings of the 3rd Workshop on RObust Methods in Analysis of Natural Language Data
POS tagging of dialectal Arabic: a minimally supervised approach

Semitic '05 Proceedings of the ACL Workshop on Computational Approaches to Semitic Languages
Inferring shallow-transfer machine translation rules from small parallel corpora

Journal of Artificial Intelligence Research
Constructing lexicon with morpho-syntactic features from untagged corpora

ECC'09 Proceedings of the 3rd international conference on European computing conference
Studying the advantages of a messy evolutionary algorithm for natural language tagging

GECCO'03 Proceedings of the 2003 international conference on Genetic and evolutionary computation: PartII
Noun retrieval effect on text summarization and delivery of personalized news articles to the user's desktop

Data & Knowledge Engineering
A semantics-enhanced language model for unsupervised word sense disambiguation

CICLing'08 Proceedings of the 9th international conference on Computational linguistics and intelligent text processing
Shipping departments vs. shipping pacemakers: using thematic analysis to improve tagging accuracy

AAAI'92 Proceedings of the tenth national conference on Artificial intelligence
Part-of-speech tagging using parallel weighted finite-state transducers

IceTAL'10 Proceedings of the 7th international conference on Advances in natural language processing
Adding morphological information to a connectionist part-of-speech tagger

CAEPIA'09 Proceedings of the Current topics in artificial intelligence, and 13th conference on Spanish association for artificial intelligence
Apertium: a free/open-source platform for rule-based machine translation

Machine Translation
Speeding up target-language driven part-of-speech tagger training for machine translation

MICAI'06 Proceedings of the 5th Mexican international conference on Artificial Intelligence
Text mining for medical documents using a hidden markov model

AIRS'06 Proceedings of the Third Asia conference on Information Retrieval Technology
An efficient multi-agent system combining POS-Taggers for arabic texts

CICLing'06 Proceedings of the 7th international conference on Computational Linguistics and Intelligent Text Processing
Inductive improvement of part-of-speech tagging and its effect on a terminology of molecular biology

AI'05 Proceedings of the 18th Canadian Society conference on Advances in Artificial Intelligence
Data driven approaches to speech and language processing

Nonlinear Speech Modeling and Applications
Open-Source portuguese–spanish machine translation

PROPOR'06 Proceedings of the 7th international conference on Computational Processing of the Portuguese Language
Hierarchical web resources retrieval by exploiting Fuzzy Formal Concept Analysis

Information Processing and Management: an International Journal
Using a shallow linguistic kernel for drug-drug interaction extraction

Journal of Biomedical Informatics
Knowledge sources for constituent parsing of german, a morphologically rich and less-configurational language

Computational Linguistics
A hidden Markov model for collaborative filtering

MIS Quarterly
Knowledge discovery in inspection reports of marine structures

Expert Systems with Applications: An International Journal
Generation of compound words in statistical machine translation into compounding languages

Computational Linguistics
Multi-label automatic indexing of music by cascade classifiers

Web Intelligence and Agent Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

We present an implementation of a part-of-speech tagger based on a hidden Markov model. The methodology enables robust and accurate tagging with few resource requirements. Only a lexicon and some unlabeled training text are required. Accuracy exceeds 96%. We describe implementation strategies and optimizations which result in high-speed operation. Three applications for tagging are described: phrase recognition; word sense disambiguation; and grammatical function assignment.