Assessing Classification Accuracy in the Revision Stage of a CBR Spam Filtering System

Authors:
José Ramón Méndez;Carlos González;Daniel Glez-Peña;Florentino Fdez-Riverola;Fernando Díaz;Juan Manuel Corchado
Affiliations:
Dept. Informática, University of Vigo, Escuela Superior de Ingeniería Informática, Edificio Politécnico, Campus Universitario As Lagoas s/n, 32004, Ourense, Spain;GFI Informatique, C/ Salvatierra 5, 28034, Madrid, Spain;Dept. Informática, University of Vigo, Escuela Superior de Ingeniería Informática, Edificio Politécnico, Campus Universitario As Lagoas s/n, 32004, Ourense, Spain;Dept. Informática, University of Vigo, Escuela Superior de Ingeniería Informática, Edificio Politécnico, Campus Universitario As Lagoas s/n, 32004, Ourense, Spain;Dept. Informática, University of Valladolid, Escuela Universitaria de Informática, Plaza Santa Eulalia, 9-11, 40005, Segovia, Spain;Dept. Informática y Automática, University of Salamanca, Plaza de la Merced s/n, 37008, Salamanca, Spain
Venue:
ICCBR '07 Proceedings of the 7th international conference on Case-Based Reasoning: Case-Based Reasoning Research and Development
Year:
2007

Citing 14
Cited 1

The nature of statistical learning theory

The nature of statistical learning theory
Learning in the presence of concept drift and hidden contexts

Machine Learning
The State of the Art in Text Filtering

User Modeling and User-Adapted Interaction
Applying lazy learning algorithms to tackle concept drift in spam filtering

Expert Systems with Applications: An International Journal
SpamHunting: An instance-based reasoning system for spam labelling and filtering

Decision Support Systems
ECUE: A Spam Filter that Uses Machine Learning to Track Concept Drift

Proceedings of the 2006 conference on ECAI 2006: 17th European Conference on Artificial Intelligence August 29 -- September 1, 2006, Riva del Garda, Italy
WikiRelate! computing semantic relatedness using wikipedia

AAAI'06 proceedings of the 21st national conference on Artificial intelligence - Volume 2
Computing semantic relatedness using Wikipedia-based explicit semantic analysis

IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
A study of cross-validation and bootstrap for accuracy estimation and model selection

IJCAI'95 Proceedings of the 14th international joint conference on Artificial intelligence - Volume 2
Estimating continuous distributions in Bayesian classifiers

UAI'95 Proceedings of the Eleventh conference on Uncertainty in artificial intelligence
Tokenising, stemming and stopword removal on anti-spam filtering domain

CAEPIA'05 Proceedings of the 11th Spanish association conference on Current Topics in Artificial Intelligence
Generating estimates of classification confidence for a case-based spam filter

ICCBR'05 Proceedings of the 6th international conference on Case-Based Reasoning Research and Development
A comparative performance study of feature selection methods for the anti-spam filtering domain

ICDM'06 Proceedings of the 6th Industrial Conference on Data Mining conference on Advances in Data Mining: applications in Medicine, Web Mining, Marketing, Image and Signal Mining
Tracking concept drift at feature selection stage in spamhunting: an anti-spam instance-based reasoning system

ECCBR'06 Proceedings of the 8th European conference on Advances in Case-Based Reasoning

SDAI: An integral evaluation methodology for content-based spam filtering models

Expert Systems with Applications: An International Journal

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper we introduce a quality metric for characterizing the solutions generated by a successful CBR spam filtering system called SpamHunting. The proposal is denoted as relevant information amount rateand it is based on combining estimations about relevance and amount of information recovered during the retrieve stage of a CBR system. The results obtained from experimentation show how this measure can successfully be used as a suitable complement for the classifications computed by our SpamHuntingsystem. In order to evaluate the performance of the quality estimation index, we have designed a formal benchmark procedure that can be used to evaluate any accuracy metric. Finally, following the designed test procedure, we show the behaviour of the proposed measure using two well-known publicly available corpus.