Rapid development of Hindi named entity recognition using conditional random fields and feature induction

Authors:
Wei Li;Andrew McCallum
Affiliations:
University of Massachusetts, Amherst, MA;University of Massachusetts, Amherst, MA
Venue:
ACM Transactions on Asian Language Information Processing (TALIP)
Year:
2003

Citing 6
Cited 18

Inducing Features of Random Fields

IEEE Transactions on Pattern Analysis and Machine Intelligence
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data

ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
Table extraction using conditional random fields

Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval
Surprise! What's in a Cebuano or Hindi Name?

ACM Transactions on Asian Language Information Processing (TALIP)
Shallow parsing with conditional random fields

NAACL '03 Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology - Volume 1
Efficiently inducing features of conditional random fields

UAI'03 Proceedings of the Nineteenth conference on Uncertainty in Artificial Intelligence

Improving the Performance of a NER System by Post-processing and Voting

SSPR & SPR '08 Proceedings of the 2008 Joint IAPR International Workshop on Structural, Syntactic, and Statistical Pattern Recognition
Improving the Performance of a NER System by Post-processing, Context Patterns and Voting

ICCPOL '09 Proceedings of the 22nd International Conference on Computer Processing of Oriental Languages. Language Technology for the Knowledge-based Economy
3D terrestrial LIDAR classifications with super-voxels and multi-scale Conditional Random Fields

Computer-Aided Design
A Semi-supervised Approach for Maximum Entropy Based Hindi Named Entity Recognition

PReMI '09 Proceedings of the 3rd International Conference on Pattern Recognition and Machine Intelligence
Voted NER system using appropriate unlabeled data

NEWS '09 Proceedings of the 2009 Named Entities Workshop: Shared Task on Transliteration
A hidden Markov model based named entity recognition system: Bengali and Hindi as case studies

PReMI'07 Proceedings of the 2nd international conference on Pattern recognition and machine intelligence
A composite kernel for named entity recognition

Pattern Recognition Letters
Classifier Ensemble using Multiobjective Optimization for Named Entity Recognition

Proceedings of the 2010 conference on ECAI 2010: 19th European Conference on Artificial Intelligence
Think globally, apply locally: using distributional characteristics for Hindi named entity identification

NEWS '10 Proceedings of the 2010 Named Entities Workshop
Rule-based named entity recognition in Urdu

NEWS '10 Proceedings of the 2010 Named Entities Workshop
Weighted vote based classifier ensemble selection using genetic algorithm for named entity recognition

NLDB'10 Proceedings of the Natural language processing and information systems, and 15th international conference on Applications of natural language to information systems
Weighted Vote-Based Classifier Ensemble for Named Entity Recognition: A Genetic Algorithm-Based Approach

ACM Transactions on Asian Language Information Processing (TALIP)
Classifier Ensemble Selection Using Genetic Algorithm for Named Entity Recognition

Research on Language and Computation
A comparative study on feature reduction approaches in Hindi and Bengali named entity recognition

Knowledge-Based Systems
Extreme extraction: machine reading in a week

EMNLP '11 Proceedings of the Conference on Empirical Methods in Natural Language Processing
A multiobjective simulated annealing approach for classifier ensemble: Named entity recognition in Indian languages as case studies

Expert Systems with Applications: An International Journal
Active machine learning technique for named entity recognition

Proceedings of the International Conference on Advances in Computing, Communications and Informatics
Combining multiple classifiers using vote based classifier ensemble technique for named entity recognition

Data & Knowledge Engineering

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper describes our application of conditional random fields with feature induction to a Hindi named entity recognition task. With only five days development time and little knowledge of this language, we automatically discover relevant features by providing a large array of lexical tests and using feature induction to automatically construct the features that most increase conditional likelihood. In an effort to reduce overfitting, we use a combination of a Gaussian prior and early stopping based on the results of 10-fold cross validation.