Chinese new word detection from query logs

Authors:
Yan Zhang;Maosong Sun;Yang Zhang
Affiliations:
State Key Laboratory on Intelligent Technology and Systems Technology, Deptment of Computer Science and Technology, Tsinghua University, Beijing, China;State Key Laboratory on Intelligent Technology and Systems Technology, Deptment of Computer Science and Technology, Tsinghua University, Beijing, China;Sohu Inc. R&D Center, Beijing, China
Venue:
ADMA'10 Proceedings of the 6th international conference on Advanced data mining and applications - Volume Part II
Year:
2010

Citing 8
Cited 0

A web-based kernel function for measuring the similarity of short text snippets

Proceedings of the 15th international conference on World Wide Web
Event detection from evolution of click-through data

Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining
Exact indexing of dynamic time warping

VLDB '02 Proceedings of the 28th international conference on Very Large Data Bases
Using subspace analysis for event detection from web click-through data

Proceedings of the 17th international conference on World Wide Web
Automatic online news topic ranking using media focus and user attention based on aging theory

Proceedings of the 17th ACM conference on Information and knowledge management
DECK: Detecting Events from Web Click-Through Data

ICDM '08 Proceedings of the 2008 Eighth IEEE International Conference on Data Mining
On burstiness-aware search for document sequences

Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining
Incorporating user behaviors in new word detection

IJCAI'09 Proceedings of the 21st international jont conference on Artifical intelligence

Quantified Score

Hi-index	0.00

Visualization

Abstract

Existing works in literature mostly resort to the web pages or other author-centric resources to detect new words, which require highly complex text processing. This paper exploits the visitor-centric resources, specifically, query logs from the commercial search engine, to detect new words. Since query logs are generated by the search engine users, and are segmented naturally, the complex text processing work can be avoided. By dynamic time warping, a new word detection algorithm based on the trajectory similarity is proposed to distinguish new words from the query logs. Experiments based on real world data sets show the effectiveness and efficiency of the proposed algorithm.