Design, implementation, and evaluation of a methodology for automatic stemmer generation

Authors:
Massimo Melucci;Nicola Orio
Affiliations:
Dipartimento di Ingegneria dell'informazione, Via Gradenigo n. 6/a, 35131, Padova, Italy;Dipartimento di Ingegneria dell'informazione, Via Gradenigo n. 6/a, 35131, Padova, Italy
Venue:
Journal of the American Society for Information Science and Technology
Year:
2007

Citing 21
Cited 0

Automatic text processing

Automatic text processing
Another stemmer

ACM SIGIR Forum
Stemming algorithms

Information retrieval
Fundamentals of speech recognition

Fundamentals of speech recognition
Viewing morphology as an inference process

SIGIR '93 Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval
Stemming algorithms: a case study for detailed evaluation

Journal of the American Society for Information Science - Special issue: evaluation of information retrieval systems
Viewing stemming as recall enhancement

SIGIR '96 Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval
Corpus-based stemming using cooccurrence of word variants

ACM Transactions on Information Systems (TOIS)
Readings in information retrieval

Readings in information retrieval
Stemming methodologies over individual query words for an Arabic information retrieval system

Journal of the American Society for Information Science
Foundations of statistical natural language processing

Foundations of statistical natural language processing
A stemming procedure and stopword list for general French corpora

Journal of the American Society for Information Science
Authoritative sources in a hyperlinked environment

Journal of the ACM (JACM)
Improving stemming for Arabic information retrieval: light stemming and co-occurrence analysis

SIGIR '02 Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval
Introduction to Modern Information Retrieval

Introduction to Modern Information Retrieval
On arabic search: improving the retrieval effectiveness via a light stemming approach

Proceedings of the eleventh international conference on Information and knowledge management
The Effectiveness of a Graph-Based Algorithm for Stemming

ICADL '02 Proceedings of the 5th International Conference on Asian Digital Libraries: Digital Libraries: People, Knowledge, and Technology
Off-Line Character Recognition Using HMM by Multiple Directional Feature Extraction and Voting with Bagging Algorithm

ICDAR '99 Proceedings of the Fifth International Conference on Document Analysis and Recognition
A novel method for stemmer generation based on hidden markov models

CIKM '03 Proceedings of the twelfth international conference on Information and knowledge management
How Effective is Stemming and Decompounding for German Text Retrieval?

Information Retrieval
Unsupervised learning of the morphology of a natural language

Computational Linguistics

Quantified Score

Hi-index	0.00

Visualization

Abstract

The authors describe a statistical approach based on hidden Markov models (HMMs), for generating stemmers automatically. The proposed approach requires little effort to insert new languages in the system even if minimal linguistic knowledge is available. This is a key advantage especially for digital libraries, which are often developed for a specific institution or government because the program can manage a great amount of documents written in local languages. The evaluation described in the article shows that the stemmers implemented by means of HMMs are as effective as those based on linguistic rules. © 2007 Wiley Periodicals, Inc.