ACM SIGIR Forum
Online Information Retrieval: Concepts, Principles, and Techniques
Online Information Retrieval: Concepts, Principles, and Techniques
When stopword lists make the difference
Journal of the American Society for Information Science and Technology
SMS based interface for FAQ retrieval
ACL '09 Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2 - Volume 2
Automatic construction of Chinese stop word list
ACOS'06 Proceedings of the 5th WSEAS international conference on Applied computer science
Evaluating bad query abandonment in an iterative SMS-based FAQ retrieval system
Proceedings of the 10th Conference on Open Research Areas in Information Retrieval
| Hi-index | 0.00 |
This paper investigates the effects of stopword removal in different stages of a system for SMS-based FAQ retrieval. Experiments are performed on the FIRE 2011 monolingual English data. The FAQ system comprises several stages, including normalization and correction of SMS, retrieval of FAQs potentially containing answers using the BM25 retrieval model, and detection of out-of-domain queries based on a k nearest-neighbor classifier. Both retrieval and OOD detection are tested with different stopword lists. Results indicate that i) retrieval performance is highest when stopwords are not removed and decreases when longer stopword lists are employed, ii) OOD detection accuracy decreases when trained on features collected during retrieval using no stopwords, iii) a combination of retrieval using no stopwords and OOD detection trained using the SMART stopwords yields the best results: 75.1% in-domain queries are answered correctly and 85.6% OOD queries are detected correctly.