Experimental study on representing units in Chinese text categorization

Authors:
Li Baoli;Chen Yuzhong;Bai Xiaojing;Yu Shiwen
Affiliations:
Institute of Computational Linguistics, Department of Computer Science and Technology, Peking University, Beijing, P.R. China;Institute of Computational Linguistics, Department of Computer Science and Technology, Peking University, Beijing, P.R. China;Institute of Computational Linguistics, Department of Computer Science and Technology, Peking University, Beijing, P.R. China;Institute of Computational Linguistics, Department of Computer Science and Technology, Peking University, Beijing, P.R. China
Venue:
CICLing'03 Proceedings of the 4th international conference on Computational linguistics and intelligent text processing
Year:
2003

Citing 4
Cited 2

Foundations of statistical natural language processing

Foundations of statistical natural language processing
Learning to Classify Text Using Support Vector Machines: Methods, Theory and Algorithms

Learning to Classify Text Using Support Vector Machines: Methods, Theory and Algorithms
A Comparative Study on Feature Selection in Text Categorization

ICML '97 Proceedings of the Fourteenth International Conference on Machine Learning
Investigating the relationship between word segmentation performance and retrieval performance in Chinese IR

COLING '02 Proceedings of the 19th international conference on Computational linguistics - Volume 1

A comparison and semi-quantitative analysis of words and character-bigrams as features in Chinese text categorization

ACL-44 Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics
A comparative study on representing units in chinese text clustering

KSEM'06 Proceedings of the First international conference on Knowledge Science, Engineering and Management

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper is a comparative study on representing units in Chinese text categorization. Several kinds of representing units, including byte 3-gram, Chinese character, Chinese word, and Chinese word with part of speech tag, were investigated. Empirical evidence shows that when the size of training data is large enough, representations of higher-level or with larger feature spaces result in better performance than those of lower level or with smaller feature spaces, whereas when the training data is limited the conclusion may be the reverse. In general, representations of higher-level or with larger feature spaces need more training data to reach the best performance. But, as to a specific representation, the size of training data and the categorization performance are not always positively correlated.