Complexity Measures of Supervised Classification Problems

Authors:
T. K. Ho;M. Basu
Affiliations:
-;-
Venue:
IEEE Transactions on Pattern Analysis and Machine Intelligence
Year:
2002

Citing 13
Cited 82

A Test to Determine the Multivariate Normality of a Data Set

IEEE Transactions on Pattern Analysis and Machine Intelligence
Small Sample Size Effects in Statistical Pattern Recognition: Recommendations for Practitioners

IEEE Transactions on Pattern Analysis and Machine Intelligence
An introduction to Kolmogorov complexity and its applications

An introduction to Kolmogorov complexity and its applications
An Evaluation of Intrinsic Dimensionality Estimators

IEEE Transactions on Pattern Analysis and Machine Intelligence
The nature of statistical learning theory

The nature of statistical learning theory
An alternative method of stochastic discrimination with applications to pattern recognition

An alternative method of stochastic discrimination with applications to pattern recognition
Large-Scale Simulation Studies in Image Pattern Recognition

IEEE Transactions on Pattern Analysis and Machine Intelligence
Pattern classification with compact distribution maps

Computer Vision and Image Understanding
A Theory of Program Size Formally Identical to Information Theory

Journal of the ACM (JACM)
Meta Analysis of Classification Algorithms for Pattern Recognition

IEEE Transactions on Pattern Analysis and Machine Intelligence
Statistical Pattern Recognition: A Review

IEEE Transactions on Pattern Analysis and Machine Intelligence
Pretopological Approach for Supervised Learning

ICPR '96 Proceedings of the International Conference on Pattern Recognition (ICPR '96) Volume IV-Volume 7472 - Volume 7472
On the Nonlinearity of Pattern Classifiers

ICPR '96 Proceedings of the International Conference on Pattern Recognition (ICPR '96) Volume IV-Volume 7472 - Volume 7472

Feature subset selection using a new definition of classifiability

Pattern Recognition Letters
Multiresolution Estimates of Classification Complexity

IEEE Transactions on Pattern Analysis and Machine Intelligence
A classification paradigm for distributed vertically partitioned data

Neural Computation
Bias-Variance Analysis of Support Vector Machines for the Development of SVM-Based Ensemble Methods

The Journal of Machine Learning Research
Classifier geometrical characteristic comparison and its application in classifier selection

Pattern Recognition Letters
Voice-based gender identification in multimedia applications

Journal of Intelligent Information Systems - Special issue: Intelligent multimedia applications
Data complexity assessment in undersampled classification of high-dimensional biomedical data

Pattern Recognition Letters
Multiresolution estimates of classification complexity and multiple subspace classifiers for understanding and solving complex recognition tasks

SPPRA'06 Proceedings of the 24th IASTED international conference on Signal processing, pattern recognition, and applications
An Evaluation of the Robustness of MTS for Imbalanced Data

IEEE Transactions on Knowledge and Data Engineering
Random Forest for Gene Expression Based Cancer Classification: Overlooked Issues

IbPRIA '07 Proceedings of the 3rd Iberian conference on Pattern Recognition and Image Analysis, Part II
Measuring the Applicability of Self-organization Maps in a Case-Based Reasoning System

IbPRIA '07 Proceedings of the 3rd Iberian conference on Pattern Recognition and Image Analysis, Part II
Dataset Complexity and Gene Expression Based Cancer Classification

WILF '07 Proceedings of the 7th international workshop on Fuzzy Logic and Applications: Applications of Fuzzy Sets Theory
When Similar Problems Don't Have Similar Solutions

ICCBR '07 Proceedings of the 7th international conference on Case-Based Reasoning: Case-Based Reasoning Research and Development
A Methodology for Analyzing Case Retrieval from a Clustered Case Memory

ICCBR '07 Proceedings of the 7th international conference on Case-Based Reasoning: Case-Based Reasoning Research and Development
Information-Theoretic Measures for Meta-learning

HAIS '08 Proceedings of the 3rd international workshop on Hybrid Artificial Intelligence Systems
Data Complexity Analysis: Linkage between Context and Solution in Classification

SSPR & SPR '08 Proceedings of the 2008 Joint IAPR International Workshop on Structural, Syntactic, and Statistical Pattern Recognition
Dataset complexity in gene expression based cancer classification using ensembles of k-nearest neighbors

Artificial Intelligence in Medicine
Data dependency in multiple classifier systems

Pattern Recognition
Mining extremely small data sets with application to software reuse

Software—Practice & Experience
2009 Special Issue: Evolving neural networks for strategic decision-making problems

Neural Networks
On using prototype reduction schemes to enhance the computation of volume-based inter-class overlap measures

Pattern Recognition
Modeling Problem Transformations based on Data Complexity

Proceedings of the 2007 conference on Artificial Intelligence Research and Development
On the Dimensions of Data Complexity through Synthetic Data Sets

Proceedings of the 2008 conference on Artificial Intelligence Research and Development: Proceedings of the 11th International Conference of the Catalan Association for Artificial Intelligence
Domains of Competence of Artificial Neural Networks Using Measures of Separability of Classes

IWANN '09 Proceedings of the 10th International Work-Conference on Artificial Neural Networks: Part I: Bio-Inspired Systems: Computational and Ambient Intelligence
An Information Theoretic Perspective on Multiple Classifier Systems

MCS '09 Proceedings of the 8th International Workshop on Multiple Classifier Systems
Beyond Homemade Artificial Data Sets

HAIS '09 Proceedings of the 4th International Conference on Hybrid Artificial Intelligence Systems
The Role of Biomedical Dataset in Classification

AIME '09 Proceedings of the 12th Conference on Artificial Intelligence in Medicine: Artificial Intelligence in Medicine
Using Supervised Complexity Measures in the Analysis of Cancer Gene Expression Data Sets

BSB '09 Proceedings of the 4th Brazilian Symposium on Bioinformatics: Advances in Bioinformatics and Computational Biology
Evolutionary undersampling for classification with imbalanced datasets: Proposals and taxonomy

Evolutionary Computation
Domains of competence of fuzzy rule based classification systems with data complexity measures: A case of study using a fuzzy hybrid genetic based machine learning method

Fuzzy Sets and Systems
Feature Selection via Maximizing Neighborhood Soft Margin

ACML '09 Proceedings of the 1st Asian Conference on Machine Learning: Advances in Machine Learning
Estimation of classification complexity

IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
A multiobjective evolutionary approach to concurrently learn rule and data bases of linguistic fuzzy-rule-based systems

IEEE Transactions on Fuzzy Systems
Selecting discrete and continuous features based on neighborhood decision error minimization

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
SVM decision boundary based discriminative subspace induction

Pattern Recognition
IFS-CoCo: Instance and feature selection based on cooperative coevolution with nearest neighbor rule

Pattern Recognition
Classification Using Geometric Level Sets

The Journal of Machine Learning Research
A fast computation of inter-class overlap measures using prototype reduction schemes

Canadian AI'08 Proceedings of the Canadian Society for computational studies of intelligence, 21st conference on Advances in artificial intelligence
Feature evaluation and selection based on neighborhood soft margin

Neurocomputing
In search of targeted-complexity problems

Proceedings of the 12th annual conference on Genetic and evolutionary computation
Sensitivity versus accuracy in multiclass problems using memetic Pareto evolutionary neural networks

IEEE Transactions on Neural Networks
Mining data with random forests: A survey and results of new tests

Pattern Recognition
Building binary-tree-based multiclass classifiers using separability measures

Neurocomputing
CODE: a data complexity framework for imbalanced datasets

PAKDD'09 Proceedings of the 13th Pacific-Asia international conference on Knowledge discovery and data mining: new frontiers in applied data mining
NDPMine: efficiently mining discriminative numerical features for pattern-based classification

ECML PKDD'10 Proceedings of the 2010 European conference on Machine learning and knowledge discovery in databases: Part II
Mapping classifiers and datasets

Expert Systems with Applications: An International Journal
Fusion of local and global detection systems to detect tuberculosis in chest radiographs

MICCAI'10 Proceedings of the 13th international conference on Medical image computing and computer-assisted intervention: Part III
The landscape contest at ICPR 2010

ICPR'10 Proceedings of the 20th International conference on Recognizing patterns in signals, speech, images, and videos
Support Vector Machine incorporated with feature discrimination

Expert Systems with Applications: An International Journal
Predicting problem difficulty for genetic programming applied to data classification

Proceedings of the 13th annual conference on Genetic and evolutionary computation
How many neurons?: a genetic programming answer

Proceedings of the 13th annual conference companion on Genetic and evolutionary computation
Towards the automatic design of decision tree induction algorithms

Proceedings of the 13th annual conference companion on Genetic and evolutionary computation
Estimating classifier performance with genetic programming

EuroGP'11 Proceedings of the 14th European conference on Genetic programming
Addressing the classification with imbalanced data: open problems and new challenges on class distribution

HAIS'11 Proceedings of the 6th international conference on Hybrid artificial intelligent systems - Volume Part I
A unifying view on dataset shift in classification

Pattern Recognition
GRASP forest: a new ensemble method for trees

MCS'11 Proceedings of the 10th international conference on Multiple classifier systems
Analysis of complexity indices for classification problems: Cancer gene expression data

Neurocomputing
Shared domains of competence of approximate learning models using measures of separability of classes

Information Sciences: an International Journal
Classifier selection based on data complexity measures

CIARP'05 Proceedings of the 10th Iberoamerican Congress conference on Progress in Pattern Recognition, Image Analysis and Applications
Data characterization for effective prototype selection

IbPRIA'05 Proceedings of the Second Iberian conference on Pattern Recognition and Image Analysis - Volume Part II
A Kolmogorov-Smirnov statistic based segmentation approach to learning from imbalanced datasets: With application in property refinance prediction

Expert Systems with Applications: An International Journal
Analysis of preprocessing vs. cost-sensitive learning for imbalanced classification. Open problems on intrinsic data characteristics

Expert Systems with Applications: An International Journal
Automatic recommendation of classification algorithms based on data set characteristics

Pattern Recognition
On-line multi-stage sorting algorithm for agriculture products

Pattern Recognition
Linear separability and classification complexity

Expert Systems with Applications: An International Journal
A multi-objective neural network based method for cover crop identification from remote sensed data

Expert Systems with Applications: An International Journal
On dataset complexity for case base maintenance

ICCBR'11 Proceedings of the 19th international conference on Case-Based Reasoning Research and Development
Evolving data sets to highlight the performance differences between machine learning classifiers

Proceedings of the 14th annual conference companion on Genetic and evolutionary computation
Predicting noise filtering efficacy with data complexity measures for nearest neighbor classification

Pattern Recognition
Neighborhood effective information ratio for hybrid feature subset evaluation and selection

Neurocomputing
Learner excellence biased by data set selection: A case for data characterisation and artificial data sets

Pattern Recognition
Searching for novel clustering programs

Proceedings of the 15th annual conference on Genetic and evolutionary computation
Adaptive artificial datasets through learning classifier systems for classification tasks

Proceedings of the 15th annual conference companion on Genetic and evolutionary computation
Exhaustive comparison of colour texture features and classification methods to discriminate cells categories in histological images of fish ovary

Pattern Recognition
Analysis of data complexity measures for classification

Expert Systems with Applications: An International Journal
A behavior-based analysis of modal problems

Proceedings of the 15th annual conference companion on Genetic and evolutionary computation
An n-spheres based synthetic data generator for supervised classification

IWANN'13 Proceedings of the 12th international conference on Artificial Neural Networks: advances in computational intelligence - Volume Part I
A unified view of class-selection with probabilistic classifiers

Pattern Recognition
A GA-based feature selection approach with an application to handwritten character recognition

Pattern Recognition Letters
Towards UCI+: A mindful repository design

Information Sciences: an International Journal
Domains of competence of the semi-naive Bayesian network classifiers

Information Sciences: an International Journal
On the use of meta-learning for instance selection: An architecture and an experimental study

Information Sciences: an International Journal

Quantified Score

Hi-index	0.15

Visualization

Abstract

We studied a number of measures that characterize the difficulty of a classification problem, focusing on the geometrical complexity of the class boundary. We compared a set of real-world problems to random labelings of points and found that real problems contain structures in this measurement space that are significantly different from the random sets. Distributions of problems in this space show that there exist at least two independent factors affecting a problem's difficulty. We suggest using this space to describe a classifier's domain of competence. This can guide static and dynamic selection of classifiers for specific problems as well as subproblems formed by confinement, projection, and transformations of the feature vectors.