On the exponential value of labeled samples
Pattern Recognition Letters
Text Classification from Labeled and Unlabeled Documents using EM
Machine Learning - Special issue on information retrieval
Building cost functions minimizing to some summary statistics
IEEE Transactions on Neural Networks
ICANN '02 Proceedings of the International Conference on Artificial Neural Networks
Local sparsity control for naive Bayes with extreme misclassification costs
Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining
Quantifying trends accurately despite classifier error and class imbalance
Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining
Estimating class priors in domain adaptation for word sense disambiguation
ACL-44 Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics
Expert Systems with Applications: An International Journal
Transductive Methods for the Distributed Ensemble Classification Problem
Neural Computation
Minimax Regret Classifier for Imprecise Class Distributions
The Journal of Machine Learning Research
Non-stationary data sequence classification using online class priors estimation
Pattern Recognition
SBIM '07 Proceedings of the 4th Eurographics workshop on Sketch-based interfaces and modeling
Quantifying counts and costs via classification
Data Mining and Knowledge Discovery
ICIAR '08 Proceedings of the 5th international conference on Image Analysis and Recognition
Quantifying the proportion of damaged sperm cells based on image analysis and neural networks
SMO'08 Proceedings of the 8th conference on Simulation, modelling and optimization
Targeting spam control on middleboxes: Spam detection based on layer-3 e-mail content classification
Computer Networks: The International Journal of Computer and Telecommunications Networking
Quantification and semi-supervised classification methods for handling changes in class distribution
Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining
Improving Classification under Changes in Class and Within-Class Distributions
IWANN '09 Proceedings of the 10th International Work-Conference on Artificial Neural Networks: Part I: Bio-Inspired Systems: Computational and Ambient Intelligence
Learning when training data are costly: the effect of class distribution on tree induction
Journal of Artificial Intelligence Research
Word sense disambiguation with distribution estimation
IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
Minimax classifiers based on neural networks
Pattern Recognition
Assessing the impact of changing environments on classifier performance
Canadian AI'08 Proceedings of the Canadian Society for computational studies of intelligence, 21st conference on Advances in artificial intelligence
Transfer estimation of evolving class priors in data stream classification
Pattern Recognition
Classifying documents with link-based bibliometric measures
Information Retrieval
Monitoring and backtesting churn models
Expert Systems with Applications: An International Journal
Estimating class proportions in boar semen analysis using the hellinger distance
IEA/AIE'10 Proceedings of the 23rd international conference on Industrial engineering and other applications of applied intelligent systems - Volume Part I
A unifying view on dataset shift in classification
Pattern Recognition
Handling concept drift via ensemble and class distribution estimation technique
ADMA'11 Proceedings of the 7th international conference on Advanced Data Mining and Applications - Volume Part II
Drift mining in data: A framework for addressing drift in classification
Computational Statistics & Data Analysis
Class distribution estimation based on the Hellinger distance
Information Sciences: an International Journal
Neural Computation
Hi-index | 0.00 |
It sometimes happens (for instance in case control studies) that a classifier is trained on a data set that does not reflect the true a priori probabilities of the target classes on real-world data. This may have a negative effect on the classification accuracy obtained on the real-world data set, especially when the classifier's decisions are based on the a posteriori probabilities of class membership. Indeed, in this case, the trained classifier provides estimates of the a posteriori probabilities that are not valid for this real-world data set (they rely on the a priori probabilities of the training set). Applying the classifier as is (without correcting its outputs with respect to these new conditions) on this new data set may thus be suboptimal. In this note, we present a simple iterative procedure for adjusting the outputs of the trained classifier with respect to these new a priori probabilities without having to refit the model, even when these probabilities are not known in advance. As a by-product, estimates of the new a priori probabilities are also obtained. This iterative algorithm is a straightforward instance of the expectation-maximization (EM) algorithm and is shown to maximize the likelihood of the new data. Thereafter, we discuss a statistical test that can be applied to decide if the a priori class probabilities have changed from the training set to the real-world data. The procedure is illustrated on different classification problems involving a multilayer neural network, and comparisons with a standard procedure for a priori probability estimation are provided. Our original method, based on the EM algorithm, is shown to be superior to the standard one for a priori probability estimation. Experimental results also indicate that the classifier with adjusted outputs always performs better than the original one in terms of classification accuracy, when the a priori probability conditions differ from the training set to the real-world data. The gain in classification accuracy can be significant.