Cost-sensitive boosting for classification of imbalanced data

Authors:
Yanmin Sun;Mohamed S. Kamel;Andrew K. C. Wong;Yang Wang
Affiliations:
Electrical and Computer Engineering Department, University of Waterloo, Waterloo, Ontario, Canada N2L 3G1;Electrical and Computer Engineering Department, University of Waterloo, Waterloo, Ontario, Canada N2L 3G1;Systems Design Engineering Department, University of Waterloo, Waterloo, Ontario, Canada N2L 3G1;Pattern Discovery Technologies Inc., Waterloo, Ontario, Canada, N2L 5Z4
Venue:
Pattern Recognition
Year:
2007

Citing 36
Cited 80

Improved Estimates for the Accuracy of Small Disjuncts

Machine Learning
Bagging predictors

Machine Learning
A decision-theoretic generalization of on-line learning and an application to boosting

Journal of Computer and System Sciences - Special issue: 26th annual ACM symposium on the theory of computing & STOC'94, May 23–25, 1994, and second annual Europe an conference on computational learning theory (EuroCOLT'95), March 13–15, 1995
On Combining Classifiers

IEEE Transactions on Pattern Analysis and Machine Intelligence
Machine Learning for the Detection of Oil Spills in Satellite Radar Images

Machine Learning - Special issue on applications of machine learning and the knowledge discovery process
MetaCost: a general method for making classifiers cost-sensitive

KDD '99 Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining
Mining association rules with multiple minimum supports

KDD '99 Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining
Improved Boosting Algorithms Using Confidence-rated Predictions

Machine Learning - The Eleventh Annual Conference on computational Learning Theory
Supervised versus unsupervised binary-learning by feedforward neural networks

Machine Learning
Learning and making decisions when costs and probabilities are both unknown

Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining
Random Forests

Machine Learning
Adaptive Fraud Detection

Data Mining and Knowledge Discovery
Support Vector Machines for Classification in Nonstandard Situations

Machine Learning
High-Order Pattern Discovery from Discrete-Valued Data

IEEE Transactions on Knowledge and Data Engineering
Pruning Decision Trees with Misclassification Costs

ECML '98 Proceedings of the 10th European Conference on Machine Learning
Improving Minority Class Prediction Using Case-Specific Feature Weights

ICML '97 Proceedings of the Fourteenth International Conference on Machine Learning
AdaCost: Misclassification Cost-Sensitive Boosting

ICML '99 Proceedings of the Sixteenth International Conference on Machine Learning
A Comparative Study of Cost-Sensitive Boosting Algorithms

ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Evaluating Boosting Algorithms to Classify Rare Classes: Comparison and Improvements

ICDM '01 Proceedings of the 2001 IEEE International Conference on Data Mining
From Association to Classification: Inference Using Weight of Evidence

IEEE Transactions on Knowledge and Data Engineering
Learning classifier models for predicting rare phenomena

Learning classifier models for predicting rare phenomena
One-class svms for document classification

The Journal of Machine Learning Research
Cost-Sensitive Learning by Cost-Proportionate Example Weighting

ICDM '03 Proceedings of the Third IEEE International Conference on Data Mining
Editorial: special issue on learning from imbalanced data sets

ACM SIGKDD Explorations Newsletter - Special issue on learning from imbalanced datasets
Mining with rarity: a unifying framework

ACM SIGKDD Explorations Newsletter - Special issue on learning from imbalanced datasets
A study of the behavior of several methods for balancing machine learning training data

ACM SIGKDD Explorations Newsletter - Special issue on learning from imbalanced datasets
Learning from imbalanced data sets with boosting and data generation: the DataBoost-IM approach

ACM SIGKDD Explorations Newsletter - Special issue on learning from imbalanced datasets
An iterative method for multi-class cost-sensitive learning

Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining
Decision trees with minimal costs

ICML '04 Proceedings of the twenty-first international conference on Machine learning
Introduction to Data Mining, (First Edition)

Introduction to Data Mining, (First Edition)
Training Cost-Sensitive Neural Networks with Methods Addressing the Class Imbalance Problem

IEEE Transactions on Knowledge and Data Engineering
The class imbalance problem: A systematic study

Intelligent Data Analysis
Cost-sensitive boosting for classification of imbalanced data

Cost-sensitive boosting for classification of imbalanced data
SMOTE: synthetic minority over-sampling technique

Journal of Artificial Intelligence Research
The foundations of cost-sensitive learning

IJCAI'01 Proceedings of the 17th international joint conference on Artificial intelligence - Volume 2
Data dependence in combining classifiers

MCS'03 Proceedings of the 4th international conference on Multiple classifier systems

Using Cost-Sensitive Learning to Determine Gene Conversions

ICIC '08 Proceedings of the 4th international conference on Intelligent Computing: Advanced Intelligent Computing Theories and Applications - with Aspects of Artificial Intelligence
Hierarchical fuzzy rule based classification systems with genetic rule selection for imbalanced data-sets

International Journal of Approximate Reasoning
A closed-form reduction of multi-class cost-sensitive learning to weighted multi-class learning

Pattern Recognition
On the influence of an adaptive inference system in fuzzy rule based classification systems for imbalanced data-sets

Expert Systems with Applications: An International Journal
Knowledge discovery from imbalanced and noisy data

Data & Knowledge Engineering
Cost-Sensitive Learning Vector Quantization for Financial Distress Prediction

EPIA '09 Proceedings of the 14th Portuguese Conference on Artificial Intelligence: Progress in Artificial Intelligence
Cost-Sensitive Boosting: Fitting an Additive Asymmetric Logistic Regression Model

ACML '09 Proceedings of the 1st Asian Conference on Machine Learning: Advances in Machine Learning
Support vector self-organizing learning for imbalanced medical data

IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
Ensemble-based classifiers

Artificial Intelligence Review
On the 2-tuples based genetic tuning performance for fuzzy rule based classification systems in imbalanced data-sets

Information Sciences: an International Journal
Improving software-quality predictions with data sampling and boosting

IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans
Boosting with pairwise constraints

Neurocomputing
Cost-sensitive boosting neural networks for software defect prediction

Expert Systems with Applications: An International Journal
Study on customer churn prediction methods based on multiple classifiers combination

IITA'09 Proceedings of the 3rd international conference on Intelligent information technology application
An asymmetric classifier based on partial least squares

Pattern Recognition
Large margin cost-sensitive learning of conditional random fields

Pattern Recognition
Sensitivity versus accuracy in multiclass problems using memetic Pareto evolutionary neural networks

IEEE Transactions on Neural Networks
An unsupervised self-organizing learning with support vector ranking for imbalanced datasets

Expert Systems with Applications: An International Journal
Analysis of an evolutionary RBFN design algorithm, CO2RBFN, for imbalanced data sets

Pattern Recognition Letters
Induction and pruning of classification rules for prediction of microseismic hazards in coal mines

Expert Systems with Applications: An International Journal
Letters: Error back-propagation algorithm for classification of imbalanced data

Neurocomputing
Class imbalance methods for translation initiation site recognition

IEA/AIE'10 Proceedings of the 23rd international conference on Industrial engineering and other applications of applied intelligent systems - Volume Part I
Audio tag annotation and retrieval using tag count information

MMM'11 Proceedings of the 17th international conference on Advances in multimedia modeling - Volume Part I
A genetic algorithm-based approach to cost-sensitive bankruptcy prediction

Expert Systems with Applications: An International Journal
Determination of relative agrarian technical efficiency by a dynamic over-sampling procedure guided by minimum sensitivity

Expert Systems with Applications: An International Journal
Addressing the classification with imbalanced data: open problems and new challenges on class distribution

HAIS'11 Proceedings of the 6th international conference on Hybrid artificial intelligent systems - Volume Part I
Class imbalance methods for translation initiation site recognition in DNA sequences

Knowledge-Based Systems
Feature selection for translation initiation site recognition

IEA/AIE'11 Proceedings of the 24th international conference on Industrial engineering and other applications of applied intelligent systems conference on Modern approaches in applied intelligence - Volume Part II
Translation initiation site recognition by means of evolutionary response surfaces

IEA/AIE'11 Proceedings of the 24th international conference on Industrial engineering and other applications of applied intelligent systems conference on Modern approaches in applied intelligence - Volume Part II
An evolutionary algorithm for gene structure prediction

IEA/AIE'11 Proceedings of the 24th international conference on Industrial engineering and other applications of applied intelligent systems conference on Modern approaches in applied intelligence - Volume Part II
A unifying view on dataset shift in classification

Pattern Recognition
Adaptive boosting for transfer learning using dynamic updates

ECML PKDD'11 Proceedings of the 2011 European conference on Machine learning and knowledge discovery in databases - Volume Part I
Metric anomaly detection via asymmetric risk minimization

SIMBAD'11 Proceedings of the First international conference on Similarity-based pattern recognition
QBoost: Predicting quantiles with boosting for regression and binary classification

Expert Systems with Applications: An International Journal
Clustering based bagging algorithm on imbalanced data sets

IUKM'11 Proceedings of the 2011 international conference on Integrated uncertainty in knowledge modelling and decision making
Gradient boosting trees for auto insurance loss cost modeling and prediction

Expert Systems with Applications: An International Journal
Dynamic classifier ensemble model for customer classification with imbalanced class distribution

Expert Systems with Applications: An International Journal
Instance selection for class imbalanced problems by means of selecting instances more than once

CAEPIA'11 Proceedings of the 14th international conference on Advances in artificial intelligence: spanish association for artificial intelligence
On Equivalence Relationships Between Classification and Ranking Algorithms

The Journal of Machine Learning Research
Shedding light on the asymmetric learning capability of AdaBoost

Pattern Recognition Letters
Analysis of preprocessing vs. cost-sensitive learning for imbalanced classification. Open problems on intrinsic data characteristics

Expert Systems with Applications: An International Journal
Asymmetric constraint optimization based adaptive boosting for cascade face detector

ICIC'11 Proceedings of the 7th international conference on Advanced Intelligent Computing Theories and Applications: with aspects of artificial intelligence
An empirical study of bagging predictors for imbalanced data with different levels of class distribution

AI'11 Proceedings of the 24th international conference on Advances in Artificial Intelligence
Editorial: Large scale instance selection by means of federal instance selection

Data & Knowledge Engineering
Inverse random under sampling for class imbalance problem and its application to multi-label classification

Pattern Recognition
Learning SVM with weighted maximum margin criterion for classification of imbalanced data

Mathematical and Computer Modelling: An International Journal
Segmentation of cervical cell nuclei in high-resolution microscopic images: A new algorithm and a web-based software framework

Computer Methods and Programs in Biomedicine
A simple methodology for soft cost-sensitive classification

Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining
Predicting seminal quality with artificial intelligence methods

Expert Systems with Applications: An International Journal
A noise-detection based AdaBoost algorithm for mislabeled data

Pattern Recognition
Exploratory class-imbalanced and non-identical data distribution in automatic keyphrase extraction

ISNN'12 Proceedings of the 9th international conference on Advances in Neural Networks - Volume Part II
DBFS: An effective Density Based Feature Selection scheme for small sample size and high dimensional imbalanced data sets

Data & Knowledge Engineering
ACOSampling: An ant colony optimization-based undersampling method for classifying imbalanced DNA microarray data

Neurocomputing
Over-Sampling from an auxiliary domain

ICONIP'12 Proceedings of the 19th international conference on Neural Information Processing - Volume Part I
Oversampling methods for classification of imbalanced breast cancer malignancy data

ICCVG'12 Proceedings of the 2012 international conference on Computer Vision and Graphics
Performance of corporate bankruptcy prediction models on imbalanced dataset: The effect of sampling methods

Knowledge-Based Systems
Analysing the classification of imbalanced data-sets with multiple classes: Binarization techniques and ad-hoc approaches

Knowledge-Based Systems
Cost-Sensitive Learning via Priority Sampling to Improve the Return on Marketing and CRM Investment

Journal of Management Information Systems
Fuzzy and hard clustering analysis for thyroid disease

Computer Methods and Programs in Biomedicine
Handling imbalanced data sets with synthetic boundary data generation using bootstrap re-sampling and AdaBoost techniques

Pattern Recognition Letters
Double-base asymmetric AdaBoost

Neurocomputing
Early prediction on imbalanced multivariate time series

Proceedings of the 22nd ACM international conference on Conference on information & knowledge management
EUSBoost: Enhancing ensembles for highly imbalanced data-sets by evolutionary undersampling

Pattern Recognition
Certainty-based active learning for sampling imbalanced datasets

Neurocomputing
Classification model for detecting and managing credit loan fraud based on individual-level utility concept

ACM SIGMIS Database
Classification model for detecting and managing credit loan fraud based on individual-level utility concept

ACM SIGMIS Database
Risk prediction of femoral neck osteoporosis using machine learning and conventional methods

IWANN'13 Proceedings of the 12th international conference on Artificial Neural Networks: advences in computational intelligence - Volume Part II
Editorial: Parameter-free classification in multi-class imbalanced data sets

Data & Knowledge Engineering
Integrated Fisher linear discriminants: An empirical study

Pattern Recognition
Training and assessing classification rules with imbalanced data

Data Mining and Knowledge Discovery
Boosted SVM for extracting rules from imbalanced data in application to prediction of the post-operative life expectancy in the lung cancer patients

Applied Soft Computing
Cost-sensitive decision tree ensembles for effective imbalanced classification

Applied Soft Computing
Applying Ant Colony Optimization to configuring stacking ensembles for data mining

Expert Systems with Applications: An International Journal
Multi-class boosting with asymmetric binary weak-learners

Pattern Recognition
Imbalanced data classification using second-order cone programming support vector machines

Pattern Recognition
Boosting weighted ELM for imbalanced learning

Neurocomputing
Cost-sensitive ordinal regression for fully automatic facial beauty assessment

Neurocomputing
Imbalanced evolving self-organizing learning

Neurocomputing
Influence of class distribution on cost-sensitive learning: A case study of bankruptcy analysis

Intelligent Data Analysis
Robust classification of imbalanced data using one-class and two-class SVM-based multiclassifiers

Intelligent Data Analysis - Business Analytics and Intelligent Optimization

Quantified Score

Hi-index	0.02

Visualization

Abstract

Classification of data with imbalanced class distribution has posed a significant drawback of the performance attainable by most standard classifier learning algorithms, which assume a relatively balanced class distribution and equal misclassification costs. The significant difficulty and frequent occurrence of the class imbalance problem indicate the need for extra research efforts. The objective of this paper is to investigate meta-techniques applicable to most classifier learning algorithms, with the aim to advance the classification of imbalanced data. The AdaBoost algorithm is reported as a successful meta-technique for improving classification accuracy. The insight gained from a comprehensive analysis of the AdaBoost algorithm in terms of its advantages and shortcomings in tacking the class imbalance problem leads to the exploration of three cost-sensitive boosting algorithms, which are developed by introducing cost items into the learning framework of AdaBoost. Further analysis shows that one of the proposed algorithms tallies with the stagewise additive modelling in statistics to minimize the cost exponential loss. These boosting algorithms are also studied with respect to their weighting strategies towards different types of samples, and their effectiveness in identifying rare cases through experiments on several real world medical data sets, where the class imbalance problem prevails.