The effect of named entities on effectiveness in cross-language information retrieval evaluation

Authors:
Thomas Mandl;Christa Womser-Hacker
Affiliations:
University of Hildesheim, Marienburger Platz, Hildesheim, Germany;University of Hildesheim, Marienburger Platz, Hildesheim, Germany
Venue:
Proceedings of the 2005 ACM symposium on Applied computing
Year:
2005

Citing 8
Cited 20

Reflections on TREC

TREC-2 Proceedings of the second conference on Text retrieval conference
How reliable are the results of large-scale information retrieval experiments?

Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval
Using part-of-speech patterns to reduce query ambiguity

SIGIR '02 Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval
The effect of topic set size on retrieval experiment error

SIGIR '02 Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval
Research to Improve Cross-Language Retrieval - Position Paper for CLEF

CLEF '00 Revised Papers from the Workshop of Cross-Language Evaluation Forum on Cross-Language Information Retrieval and Evaluation
The Effect of Bilingual Term List Size on Dictionary-Based Cross-Language Information Retrieval

HICSS '03 Proceedings of the 36th Annual Hawaii International Conference on System Sciences (HICSS'03) - Track 4 - Volume 4
Transliteration of proper names in cross-language applications

Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval
Monolingual Document Retrieval for European Languages

Information Retrieval

Proximity-based document representation for named entity retrieval

Proceedings of the sixteenth ACM conference on Conference on information and knowledge management
How robust are multilingual information retrieval systems?

Proceedings of the 2008 ACM symposium on Applied computing
Mining named entity transliteration equivalents from comparable corpora

Proceedings of the 17th ACM conference on Information and knowledge management
A study of using an out-of-box commercial MT system for query translation in CLIR

Proceedings of the 2nd ACM workshop on Improving non english web searching
"They Are Out There, If You Know Where to Look": Mining Transliterations of OOV Query Terms for Cross-Language Information Retrieval

ECIR '09 Proceedings of the 31th European Conference on IR Research on Advances in Information Retrieval
MINT: a method for effective and scalable mining of named entity transliterations from large comparable corpora

EACL '09 Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics
Report of NEWS 2009 machine transliteration shared task

NEWS '09 Proceedings of the 2009 Named Entities Workshop: Shared Task on Transliteration
Semantic annotation based exploratory search for information analysts

Information Processing and Management: an International Journal
Compositional Machine Transliteration

ACM Transactions on Asian Language Information Processing (TALIP)
Report of NEWS 2010 transliteration generation shared task

NEWS '10 Proceedings of the 2010 Named Entities Workshop
Assessing the challenge of fine-grained named entity recognition and classification

NEWS '10 Proceedings of the 2010 Named Entities Workshop
Enhancing query translation with relevance feedback in translingual information retrieval

Information Processing and Management: an International Journal
Weighted Vote-Based Classifier Ensemble for Named Entity Recognition: A Genetic Algorithm-Based Approach

ACM Transactions on Asian Language Information Processing (TALIP)
Domain specific mono- and bilingual english to german retrieval experiments with a social science document corpus

CLEF'05 Proceedings of the 6th international conference on Cross-Language Evalution Forum: accessing Multilingual Information Repositories
A fast forward approach to cross-lingual question answering for english and german

CLEF'05 Proceedings of the 6th international conference on Cross-Language Evalution Forum: accessing Multilingual Information Repositories
Transliteration equivalence using canonical correlation analysis

ECIR'2010 Proceedings of the 32nd European conference on Advances in Information Retrieval
Regularized interlingual projections: evaluation on multilingual transliteration

EMNLP-CoNLL '12 Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning
Report of NEWS 2012 machine transliteration shared task

NEWS '12 Proceedings of the 4th Named Entity Workshop
Robust ad-hoc retrieval experiments with french and english at the University of Hildesheim

CLEF'06 Proceedings of the 7th international conference on Cross-Language Evaluation Forum: evaluation of multilingual and multi-modal information retrieval
Combining multiple classifiers using vote based classifier ensemble technique for named entity recognition

Data & Knowledge Engineering

Quantified Score

Hi-index	0.01

Visualization

Abstract

The large number of experiments carried out within evaluation initiatives for information retrieval has led to an invaluable source for further research and meta-analysis. In this study, an analysis of the results of the Cross Language Evaluation Forum (CLEF) campaigns for the years 2000 to 2003 is presented. This study considers the performance of the systems for each individual topic. It is dedicated to the influence of named entities on retrieval performance. Named entities in topics lead to significant improvement of the retrieval quality in general and for most systems and tasks. The performance of systems varies for topics without, with one or two and with three or more named entities. This knowledge gained by data mining on the evaluation results can be exploited for the improvement of retrieval systems as well as for the design of topics for future CLEF campaigns.