Neural fuzzy systems: a neuro-fuzzy synergism to intelligent systems
Neural fuzzy systems: a neuro-fuzzy synergism to intelligent systems
Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval
Translation of web queries using anchor text mining
ACM Transactions on Asian Language Information Processing (TALIP)
Chinese word segmentation and its effect on information retrieval
Information Processing and Management: an International Journal
Word identification for Mandarin Chinese sentences
COLING '92 Proceedings of the 14th conference on Computational linguistics - Volume 1
Mining events and new name translations from online daily news
Proceedings of the 4th ACM/IEEE-CS joint conference on Digital libraries
A Chinese word segmentation based on language situation in processing ambiguous words
Information Sciences: an International Journal
A bottom-up merging algorithm for Chinese unknown word extraction
SIGHAN '03 Proceedings of the second SIGHAN workshop on Chinese language processing - Volume 17
Introduction to Modern Information Retrieval, Third Edition
Introduction to Modern Information Retrieval, Third Edition
Survey of clustering algorithms
IEEE Transactions on Neural Networks
Mining term networks from text collections for crime investigation
Expert Systems with Applications: An International Journal
Unknown Chinese word extraction based on variety of overlapping strings
Information Processing and Management: an International Journal
SADINA: Adaptive System for Academic News Dissemination
Proceedings of International Conference on Information Integration and Web-based Applications & Services
Hi-index | 12.05 |
Chinese word segmentation is an essential step in a processing of Chinese natural language because it is beneficial to the Chinese text mining and information retrieval. Currently, the lexicon-based Chinese word segmentation scheme is widely adopted, which can correctly identify Chinese sentences as distinct words from Chinese language texts in real-word applications. However, the word identification ability of the lexicon-based scheme is highly dependent with a well prepared lexicon with sufficient amount of lexical entries which covers all of the Chinese words. In particular, this scheme cannot perform Chinese word segmentation process well for highly changeable texts with time, such as newspaper articles and web documents. This is because highly changeable documents often contain many new words that cannot be identified by a lexicon-based Chinese word segmentation system with a constant lexicon. Moreover, to maintain a lexicon by manpower is an inefficient and time-consuming job. Therefore, this study proposes a novel statistics-based scheme for extraction of new words based on the categorized corpora of Google News retrieved automatically from the Google News site to promote the word identification ability for lexicon-based Chinese word segmentation systems. Since corpora of news almost contain all words used in daily life, to extract news words from corpora of news and to incrementally add them into lexicon for lexicon-based Chinese word segmentation systems provide benefits in terms of automatically constructing a professional lexicon and enhancing word identification capability. Compared to another proposed scheme of new word extraction, the experimental results indicated that the proposed extraction scheme of new words not only more correctly retrieves new words from the categorized corpora of Google News, but also obtains larger amount of new words. Moreover, the proposed scheme of new word extraction has been applied to automatically expand the lexicon of the Chinese word segmentation system ECScanner (A Chinese Lexicon Scanner with Lexicon Extension). Currently, the ECScanner has been published on the Web to provide Chinese word segmentation service based on Web service. Experimental results also confirmed that ECScanner is superior to CKIP (Chinese knowledge information processing) in identifying meaningful Chinese words.