A Comparative Analysis of Methods for Pruning Decision Trees

Authors:
Floriana Esposito;Donato Malerba;Giovanni Semeraro
Affiliations:
-;-;-
Venue:
IEEE Transactions on Pattern Analysis and Machine Intelligence
Year:
1997

Citing 1
Cited 64

An Iterative Growing and Pruning Algorithm for Classification Tree Design

IEEE Transactions on Pattern Analysis and Machine Intelligence

Comments on Esposito et al.

IEEE Transactions on Pattern Analysis and Machine Intelligence
Globally Optimal Fuzzy Decision Trees for Classification and Regression

IEEE Transactions on Pattern Analysis and Machine Intelligence
Learning and making decisions when costs and probabilities are both unknown

Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining
Symbolic Learning Techniques in Paper Document Processing

MLDM '99 Proceedings of the First International Workshop on Machine Learning and Data Mining in Pattern Recognition
Boosted Tree Ensembles for Solving Multiclass Problems

MCS '02 Proceedings of the Third International Workshop on Multiple Classifier Systems
An Empirical Comparison of Pruning Methods for Ensemble Classifiers

IDA '01 Proceedings of the 4th International Conference on Advances in Intelligent Data Analysis
Evaluation of decision trees: a multi-criteria approach

Computers and Operations Research
Selective Rademacher Penalization and Reduced Error Pruning of Decision Trees

The Journal of Machine Learning Research
Optimization study with ligand-design interval rules

Journal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology
Evolutionary stratified training set selection for extracting classification rules with trade off precision-interpretability

Data & Knowledge Engineering
Post-pruning in decision tree induction using multiple performance measures

Computers and Operations Research
A web-based GIS Decision Support System for managing and planning USDA's Conservation Reserve Program (CRP)

Environmental Modelling & Software
Decision trees for mining data streams

Intelligent Data Analysis
Anytime Learning of Decision Trees

The Journal of Machine Learning Research
Multiobjective Optimization in Bioinformatics and Computational Biology

IEEE/ACM Transactions on Computational Biology and Bioinformatics (TCBB)
Predicting Metastasis in Breast Cancer: Comparing a Decision Tree with Domain Experts

Journal of Medical Systems
Decision trees using model ensemble-based nodes

Pattern Recognition
A comprehensive review of recursive Naïve Bayes Classifiers

Intelligent Data Analysis
Hybrid systems of local basis functions

Intelligent Data Analysis
Experiments with an innovative tree pruning algorithm

AIAP'07 Proceedings of the 25th conference on Proceedings of the 25th IASTED International Multi-Conference: artificial intelligence and applications
Invariant optimal feature selection: A distance discriminant and feature ranking based solution

Pattern Recognition
Dynamic modular fuzzy neural classifier with tree-based structure identification

Neurocomputing
A k-norm pruning algorithm for decision tree classifiers based on error rate estimation

Machine Learning
Parallel learning using decision trees: a novel approach

AMCOS'05 Proceedings of the 4th WSEAS International Conference on Applied Mathematics and Computer Science
Learning in Environments with Unknown Dynamics: Towards more Robust Concept Learners

The Journal of Machine Learning Research
Maximizing classifier utility when there are data acquisition and modeling costs

Data Mining and Knowledge Discovery
Bayesian Model of Recognition on a Finite Set of Events

SETN '08 Proceedings of the 5th Hellenic conference on Artificial Intelligence: Theories, Models and Applications
An experimental comparison of performance measures for classification

Pattern Recognition Letters
A similarity measure to assess the stability of classification trees

Computational Statistics & Data Analysis
Applying enhanced data mining approaches in predicting bank performance: A case of Taiwanese commercial banks

Expert Systems with Applications: An International Journal
An analysis of reduced error pruning

Journal of Artificial Intelligence Research
Ranking cases with decision trees: a geometric method that preserves intelligibility

IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
k-norm misclassification rate estimation for decision trees

ASC '07 Proceedings of The Eleventh IASTED International Conference on Artificial Intelligence and Soft Computing
Optimisation of the decision tree technique applied to simulated sow herd datasets

Computers and Electronics in Agriculture
Using mobile phones to determine transportation modes

ACM Transactions on Sensor Networks (TOSN)
CSNL: A cost-sensitive non-linear decision tree algorithm

ACM Transactions on Knowledge Discovery from Data (TKDD)
Simplification methods for model trees with regression and splitting nodes

MLDM'03 Proceedings of the 3rd international conference on Machine learning and data mining in pattern recognition
A memetic algorithm for global induction of decision trees

SOFSEM'08 Proceedings of the 34th conference on Current trends in theory and practice of computer science
An overview of AI research in Italy

Artificial intelligence
Handling over-fitting in test cost-sensitive decision tree learning by feature selection, smoothing and pruning

Journal of Systems and Software
Implementation of a scalable decision forest model based on information theory

Expert Systems with Applications: An International Journal
Morphological perceptrons with competitive learning: Lattice-theoretical framework and constructive learning algorithm

Information Sciences: an International Journal
A data mining approach to guide students through the enrollment process based on academic performance

User Modeling and User-Adapted Interaction
Towards the automatic design of decision tree induction algorithms

Proceedings of the 13th annual conference companion on Genetic and evolutionary computation
Construction of an event tree on the basis of expert knowledge and time series

KONT'07/KPP'07 Proceedings of the First international conference on Knowledge processing and data analysis
ACE-Cost: acquisition cost efficient classifier by hybrid decision tree with local SVM leaves

MLDM'11 Proceedings of the 7th international conference on Machine learning and data mining in pattern recognition
Discrete decision tree induction to avoid overfitting on categorical data

MAMECTIS/NOLASC/CONTROL/WAMUS'11 Proceedings of the 13th WSEAS international conference on mathematical methods, computational techniques and intelligent systems, and 10th WSEAS international conference on non-linear analysis, non-linear systems and chaos, and 7th WSEAS international conference on dynamical systems and control, and 11th WSEAS international conference on Wavelet analysis and multirate systems: recent researches in computational techniques, non-linear systems and control
Mixed decision trees: an evolutionary approach

DaWaK'06 Proceedings of the 8th international conference on Data Warehousing and Knowledge Discovery
Jmax-pruning: A facility for the information theoretic pruning of modular classification rules

Knowledge-Based Systems
A data pre-processing method to increase efficiency and accuracy in data mining

AIME'05 Proceedings of the 10th conference on Artificial Intelligence in Medicine
Approximate boolean reasoning: foundations and applications in data mining

Transactions on Rough Sets V
Evolutionary learning of linear trees with embedded feature selection

ICAISC'06 Proceedings of the 8th international conference on Artificial Intelligence and Soft Computing
Evaluation of decision tree pruning with subadditive penalties

IDEAL'06 Proceedings of the 7th international conference on Intelligent Data Engineering and Automated Learning
Generalised bottom-up pruning: A model level combination of decision trees

Expert Systems with Applications: An International Journal
A hybrid particle swarm optimization based fuzzy expert system for the diagnosis of coronary artery disease

Expert Systems with Applications: An International Journal
A hyper-heuristic evolutionary algorithm for automatically designing decision-tree algorithms

Proceedings of the 14th annual conference on Genetic and evolutionary computation
Making a Shallow Network Deep: Conversion of a Boosting Classifier into a Decision Tree by Boolean Optimisation

International Journal of Computer Vision
Construction of decision trees by using feature importance value for improved learning performance

ICONIP'12 Proceedings of the 19th international conference on Neural Information Processing - Volume Part II
Global top-scoring pair decision tree for gene expression data analysis

EuroGP'13 Proceedings of the 16th European conference on Genetic Programming
Decision trees: a recent overview

Artificial Intelligence Review
Using decision tree for diagnosing heart disease patients

AusDM '11 Proceedings of the Ninth Australasian Data Mining Conference - Volume 121
Lazy overfitting control

MLDM'13 Proceedings of the 9th international conference on Machine Learning and Data Mining in Pattern Recognition
Automatic design of decision-tree algorithms with evolutionary algorithms

Evolutionary Computation
A hybrid decision tree classifier

Journal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology

Quantified Score

Hi-index	0.15

Visualization

Abstract

In this paper, we address the problem of retrospectively pruning decision trees induced from data, according to a top-down approach. This problem has received considerable attention in the areas of pattern recognition and machine learning, and many distinct methods have been proposed in literature. We make a comparative study of six well-known pruning methods with the aim of understanding their theoretical foundations, their computational complexity, and the strengths and weaknesses of their formulation. Comments on the characteristics of each method are empirically supported. In particular, a wide experimentation performed on several data sets leads us to opposite conclusions on the predictive accuracy of simplified trees from some drawn in the literature. We attribute this divergence to differences in experimental designs. Finally, we prove and make use of a property of the reduced error pruning method to obtain an objective evaluation of the tendency to overprune/underprune observed in each method.