An EM Based Training Algorithm for Cross-Language Text Categorization

  • Authors:
  • Leonardo Rigutini;Marco Maggini;Bing Liu

  • Affiliations:
  • Università di Siena;Università di Siena;University of Illinois at Chicago

  • Venue:
  • WI '05 Proceedings of the 2005 IEEE/WIC/ACM International Conference on Web Intelligence
  • Year:
  • 2005

Quantified Score

Hi-index 0.00

Visualization

Abstract

Due to the globalization on the Web, many companies and institutions need to efficiently organize and search repositories containing multilingual documents. The management of these heterogeneous text collections increases the costs significantly because experts of different languages are required to organize these collections. Cross-Language Text Categorization can provide techniques to extend existing automatic classification systems in one language to new languages without requiring additional intervention of human experts. In this paper we propose a learning algorithm based on the EM scheme which can be used to train text classifiers in a multilingual environment. In particular, in the proposed approach, we assume that a predefined category set and a collection of labeled training data is available for a given language L驴. A classifier for a different language L驴 is trained by translating the available labeled training set for L驴 to L驴 and by using an additional set of unlabeled documents from L驴. This technique allows us to extract correct statistical properties of the language L驴 which are not completely available in automatically translated examples, because of the different characteristics of language L驴 and of the approximation of the translation process. Our experimental results show that the performance of the proposed method is very promising when applied on a test document set extracted from newsgroups in English and Italian.