Combining probability models and web mining models: a framework for proper name transliteration

  • Authors:

  • Yilu Zhou;Feng Huang;Hsinchun Chen

  • Affiliations:

  • Department of Information Systems and Management, George Washington University, Washington, USA 20052;Consumer Electronic Group, Handheld Division, Advanced Micro Devices, Inc., Sunnyvale, USA;Department of Management Information Systems, The University of Arizona, Tucson, USA

  • Venue:
  • Information Technology and Management
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

The rapid growth of the Internet has created a tremendous number of multilingual resources. However, language boundaries prevent information sharing and discovery across countries. Proper names play an important role in search queries and knowledge discovery. When foreign names are involved, proper names are often translated phonetically which is referred to as transliteration. In this research we propose a generic transliteration framework, which incorporates an enhanced Hidden Markov Model (HMM) and a Web mining model. We improved the traditional statistical-based transliteration in three areas: (1) incorporated a simple phonetic transliteration knowledge base; (2) incorporated a bigram and a trigram HMM; (3) incorporated a Web mining model that uses word frequency of occurrence information from the Web. We evaluated the framework on an English---Arabic back transliteration. Experiments showed that when using HMM alone, a combination of the bigram and trigram HMM approach performed the best for English---Arabic transliteration. While the bigram model alone achieved fairly good performance, the trigram model alone did not. The Web mining approach boosted the performance by 79.05%. Overall, our framework achieved a precision of 0.72 when the eight best transliterations were considered. Our results show promise for using transliteration techniques to improve multilingual Web retrieval.