A phonetic-based approach to Chinese chat text normalization

Authors:
Yunqing Xia;Kam-Fai Wong;Wenjie Li
Affiliations:
The Chinese University of Hong Kong, Shatin, Hong Kong;The Chinese University of Hong Kong, Shatin, Hong Kong;The Hong Kong Polytechnic University, Kowloon, Hong Kong
Venue:
ACL-44 Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics
Year:
2006

Citing 2
Cited 1

A statistical approach to machine translation

Computational Linguistics
Modified Kneser-Ney Smoothing of n-gram Models

Modified Kneser-Ney Smoothing of n-gram Models

Pretreatment for speech machine translation

ICCCI'10 Proceedings of the Second international conference on Computational collective intelligence: technologies and applications - Volume Part II

Quantified Score

Hi-index	0.00

Visualization

Abstract

Chatting is a popular communication media on the Internet via ICQ, chat rooms, etc. Chat language is different from natural language due to its anomalous and dynamic natures, which renders conventional NLP tools inapplicable. The dynamic problem is enormously troublesome because it makes static chat language corpus outdated quickly in representing contemporary chat language. To address the dynamic problem, we propose the phonetic mapping models to present mappings between chat terms and standard words via phonetic transcription, i.e. Chinese Pinyin in our case. Different from character mappings, the phonetic mappings can be constructed from available standard Chinese corpus. To perform the task of dynamic chat language term normalization, we extend the source channel model by incorporating the phonetic mapping models. Experimental results show that this method is effective and stable in normalizing dynamic chat language terms.