A decision-theoretic generalization of on-line learning and an application to boosting
Journal of Computer and System Sciences - Special issue: 26th annual ACM symposium on the theory of computing & STOC'94, May 23–25, 1994, and second annual Europe an conference on computational learning theory (EuroCOLT'95), March 13–15, 1995
Boosting in the limit: maximizing the margin of learned ensembles
AAAI '98/IAAI '98 Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence
Improved Boosting Algorithms Using Confidence-rated Predictions
Machine Learning - The Eleventh Annual Conference on computational Learning Theory
Proceedings of the 1998 conference on Advances in neural information processing systems II
Improved Generalization Through Explicit Optimization of Margins
Machine Learning
Machine Learning
On Bias, Variance, 0/1—Loss, and the Curse-of-Dimensionality
Data Mining and Knowledge Discovery
An Adaptive Version of the Boost by Majority Algorithm
Machine Learning
A Unified Bias-Variance Decomposition for Zero-One and Squared Loss
Proceedings of the Seventeenth National Conference on Artificial Intelligence and Twelfth Conference on Innovative Applications of Artificial Intelligence
MadaBoost: A Modification of AdaBoost
COLT '00 Proceedings of the Thirteenth Annual Conference on Computational Learning Theory
Identifying and Handling Mislabelled Instances
Journal of Intelligent Information Systems
Smooth boosting and learning with malicious noise
The Journal of Machine Learning Research
Pruning Training Sets for Learning of Object Categories
CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 1 - Volume 01
Learning a Similarity Metric Discriminatively, with Application to Face Verification
CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 1 - Volume 01
Boosted discriminant projections for nearest neighbor classification
Pattern Recognition
Robust Loss Functions for Boosting
Neural Computation
Boosting and other ensemble methods
Neural Computation
Class-switching neural network ensembles
Neurocomputing
Avoiding Boosting Overfitting by Removing Confusing Samples
ECML '07 Proceedings of the 18th European conference on Machine Learning
Distance Metric Learning for Large Margin Nearest Neighbor Classification
The Journal of Machine Learning Research
Boosting by weighting critical and erroneous samples
Neurocomputing
Using a genetic algorithm for editing k-nearest neighbor classifiers
IDEAL'07 Proceedings of the 8th international conference on Intelligent data engineering and automated learning
Nearest neighbor pattern classification
IEEE Transactions on Information Theory
Large margin nearest neighbor classifiers
IEEE Transactions on Neural Networks
A novel two-level nearest neighbor classification algorithm using an adaptive distance metric
Knowledge-Based Systems
A noise-detection based AdaBoost algorithm for mislabeled data
Pattern Recognition
Reducing overfitting of AdaBoost by clustering-based pruning of hard examples
Proceedings of the 7th International Conference on Ubiquitous Information Management and Communication
Projected-prototype based classifier for text categorization
Knowledge-Based Systems
| Hi-index | 0.01 |
Any realistic model of learning from samples must address the issue of noisy data. AdaBoost is known as an effective method for improving the performance of base classifiers both theoretically and empirically. However, previous studies have shown that AdaBoost is prone to overfitting, especially in noisy domains. On the other hand, the kNN rule is one of the oldest and simplest methods for pattern classification. Nevertheless, it often yields competitive results, and in certain domains, when cleverly combined with prior knowledge, it has significantly advanced the state-of-the-art. In this paper, an edited AdaBoost by weighted kNN (EAdaBoost ) is designed where AdaBoost and kNN naturally complement each other. First, AdaBoost is run on the training data to capitalize on some statistical regularity in the data. Then, a weighted kNN algorithm is run on the feature space composed of classifiers produced by AdaBoost to achieve competitive results. AdaBoost is then used to enhance the classification accuracy and avoid overfitting by editing the data sets using the weighted kNN algorithm for improving the quality of training data. Experiments performed on ten different UCI data sets show that the new Boosting algorithm almost always achieves considerably better classification accuracy than AdaBoost. Furthermore, experiments on data with artificially controlled noise indicate that the new Boosting algorithm is robust to noise.