Managing irrelevant knowledge in CBR models for unsolicited e-mail classification

Authors:
J. R. Méndez;D. Glez-Peña;F. Fdez-Riverola;F. Díaz;J. M. Corchado
Affiliations:
Dept. Informática, University of Vigo, Escuela Superior de Ingeniería Informática, Edificio Politécnico, Campus Universitario As Lagoas s/n, 32004 Ourense, Spain;Dept. Informática, University of Vigo, Escuela Superior de Ingeniería Informática, Edificio Politécnico, Campus Universitario As Lagoas s/n, 32004 Ourense, Spain;Dept. Informática, University of Vigo, Escuela Superior de Ingeniería Informática, Edificio Politécnico, Campus Universitario As Lagoas s/n, 32004 Ourense, Spain;Dept. Informática, University of Valladolid, Escuela Universitaria de Informática, Plaza Santa Eulalia, 9-11, 40005 Segovia, Spain;Dept. Informática y Automática, University of Salamanca, Plaza de la Merced s/n, 37008 Salamanca, Spain
Venue:
Expert Systems with Applications: An International Journal
Year:
2009

Citing 14
Cited 2

Learning in the presence of concept drift and hidden contexts

Machine Learning
Performance standards and evaluations in IR test collections: cluster-based retrieval models

Information Processing and Management: an International Journal
An Evaluation of Statistical Approaches to Text Categorization

Information Retrieval
The Utility Problem Analysed: A Case-Based Reasoning Perspective

EWCBR '96 Proceedings of the Third European Workshop on Advances in Case-Based Reasoning
Modelling the Competence of Case-Bases

EWCBR '98 Proceedings of the 4th European Workshop on Advances in Case-Based Reasoning
Diagnosis and Decision Support

Case-Based Reasoning Technology, From Foundations to Applications
Case-Base Maintenance

IEA/AIE '98 Proceedings of the 11th International Conference on Industrial and Engineering Applications of Artificial In telligence and Expert Systems: Tasks and Methods in Applied Artificial Intelligence
Applying lazy learning algorithms to tackle concept drift in spam filtering

Expert Systems with Applications: An International Journal
SpamHunting: An instance-based reasoning system for spam labelling and filtering

Decision Support Systems
Remembering to forget: a competence-preserving case deletion policy for case-based reasoning systems

IJCAI'95 Proceedings of the 14th international joint conference on Artificial intelligence - Volume 1
A study of cross-validation and bootstrap for accuracy estimation and model selection

IJCAI'95 Proceedings of the 14th international joint conference on Artificial intelligence - Volume 2
A case-based technique for tracking concept drift in spam filtering

Knowledge-Based Systems
Tracking concept drift at feature selection stage in spamhunting: an anti-spam instance-based reasoning system

ECCBR'06 Proceedings of the 8th European conference on Advances in Case-Based Reasoning
Support vector machines for spam categorization

IEEE Transactions on Neural Networks

Review: A review of machine learning approaches to Spam filtering

Expert Systems with Applications: An International Journal
Grindstone4Spam: An optimization toolkit for boosting e-mail classification

Journal of Systems and Software

Quantified Score

Hi-index	12.05

Visualization

Abstract

The problem of unsolicited e-mail has been increasing during recent years. Fortunately, some advanced technologies have been successfully applied to spam filtering, achieving promising results. Recently, we have introduced SpamHunting, a successful spam filter able to address the concept drift problem by combining a relevant term identification technique with an evolving sliding window strategy. Several successful spam filtering techniques use continuous learning strategies to achieve better adaptation capabilities and address concept drift issues. Nevertheless, due to the presence of concept drift and hidden changes in the environment, the presence of obsolete and irrelevant knowledge becomes a serious drawback. Soon after the launch of the filter, many decisions are made based on irrelevant and/or obsolete knowledge. Therefore, in such a situation, the use of forgetting strategies is as important as the implementation of continuous learning approaches. In this paper we introduce a novel technique designed for identifying and removing the obsolete and irrelevant knowledge that has accumulated over to the passage of time. We have carried out several experiments to test for the suitability of our proposal showing the results obtained and its applicability.