An Empirical Comparison of Pruning Methods for Decision Tree Induction

Authors:
John Mingers
Affiliations:
School of Industrial and Business Studies, University of Warwick, Coventry CV4 7AL, England. BSRCD@CU.WARWICK.AC.UK
Venue:
Machine Learning
Year:
1989

Citing 0
Cited 58

Efficient agnostic PAC-learning with simple hypothesis

COLT '94 Proceedings of the seventh annual conference on Computational learning theory
The Multiscale Classifier

IEEE Transactions on Pattern Analysis and Machine Intelligence
Pruning Algorithms for Rule Learning

Machine Learning
A Teaching Strategy for Memory-Based Control

Artificial Intelligence Review - Special issue on lazy learning
An Exact Probability Metric for Decision Tree Splitting and Stopping

Machine Learning
New Techniques for Data Reduction in a Database System for Knowledge Discovery Applications

Journal of Intelligent Information Systems
Multiple Comparisons in Induction Algorithms

Machine Learning
Efficient Construction of Regression Trees with Range and Region Splitting

Machine Learning
Automatic Construction of Decision Trees from Data: A Multi-Disciplinary Survey

Data Mining and Knowledge Discovery
The Role of Occam‘s Razor in Knowledge Discovery

Data Mining and Knowledge Discovery
Generalized Version Space Learning Algorithm for Noisy and Uncertain Data

IEEE Transactions on Knowledge and Data Engineering
Sequential Decision Models for Expert System Optimization

IEEE Transactions on Knowledge and Data Engineering
Probability Bounds for Goal Directed Queries in Bayesian Networks

IEEE Transactions on Knowledge and Data Engineering
Goal-Directed Classification Using Linear Machine Decision Trees

IEEE Transactions on Pattern Analysis and Machine Intelligence
Possibilistic Induction in Decision-Tree Learning

ECML '02 Proceedings of the 13th European Conference on Machine Learning
Experimental Evaluation of a Trainable Scribble Recognizer for Calligraphic Interfaces

GREC '01 Selected Papers from the Fourth International Workshop on Graphics Recognition Algorithms and Applications
Bloomy Decision Tree for Multi-objective Classification

PKDD '01 Proceedings of the 5th European Conference on Principles of Data Mining and Knowledge Discovery
Pre-pruning Classification Trees to Reduce Overfitting in Noisy Domains

IDEAL '02 Proceedings of the Third International Conference on Intelligent Data Engineering and Automated Learning
Feature Selection as Retrospective Pruning in Hierarchical Clustering

IDA '99 Proceedings of the Third International Symposium on Advances in Intelligent Data Analysis
Using J-Pruning to Reduce Overfitting of Classification Rules in Noisy Domains

DEXA '02 Proceedings of the 13th International Conference on Database and Expert Systems Applications
The Difficulty of Reduced Error Pruning of Leveled Branching Programs

Annals of Mathematics and Artificial Intelligence
Simplifying decision trees: A survey

The Knowledge Engineering Review
A flexible POS tagger using an automatically acquired language model

ACL '98 Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics
Selective Rademacher Penalization and Reduced Error Pruning of Decision Trees

The Journal of Machine Learning Research
A scalable decision tree system and its application in pattern recognition and intrusion detection

Decision Support Systems
A comparison of machine learning with human judgment

Journal of Management Information Systems - Special section: Research in integrating learning capabilities into information systems
A Comparison of Classification Methods for Predicting Deception in Computer-Mediated Communication

Journal of Management Information Systems
Classification in the presence of class noise using a probabilistic Kernel Fisher method

Pattern Recognition
A similarity measure to assess the stability of classification trees

Computational Statistics & Data Analysis
Classification algorithm sensitivity to training data with non representative attribute noise

Decision Support Systems
Robust supervised classification with mixture models: Learning from data with uncertain labels

Pattern Recognition
Estimation of conditional probabilities with decision trees and an application to fine-grained POS tagging

COLING '08 Proceedings of the 22nd International Conference on Computational Linguistics - Volume 1
Iterative optimization and simplification of hierarchical clusterings

Journal of Artificial Intelligence Research
A system for induction of oblique decision trees

Journal of Artificial Intelligence Research
An analysis of reduced error pruning

Journal of Artificial Intelligence Research
Classifiers: a theoretical and empirical study

IJCAI'91 Proceedings of the 12th international joint conference on Artificial intelligence - Volume 2
Intermediate decision trees

IJCAI'95 Proceedings of the 14th international joint conference on Artificial intelligence - Volume 2
A scalable decision tree system and its application in pattern recognition and intrusion detection

Decision Support Systems
Relabeling algorithm for retrieval of noisy instances and improving prediction quality

Computers in Biology and Medicine
CSNL: A cost-sensitive non-linear decision tree algorithm

ACM Transactions on Knowledge Discovery from Data (TKDD)
Inductive learning in probabilistic domain

AAAI'90 Proceedings of the eighth National conference on Artificial intelligence - Volume 2
Sparse data and the effect of overfitting avoidance in decision tree induction

AAAI'92 Proceedings of the tenth national conference on Artificial intelligence
OC1: randomized induction of oblique decision trees

AAAI'93 Proceedings of the eleventh national conference on Artificial intelligence
Pattern discovery in distributed databases

AAAI'97/IAAI'97 Proceedings of the fourteenth national conference on artificial intelligence and ninth conference on Innovative applications of artificial intelligence
Performance Analysis of Class Noise Detection Algorithms

Proceedings of the 2010 conference on STAIRS 2010: Proceedings of the Fifth Starting AI Researchers' Symposium
A data mining approach to guide students through the enrollment process based on academic performance

User Modeling and User-Adapted Interaction
Discrete decision tree induction to avoid overfitting on categorical data

MAMECTIS/NOLASC/CONTROL/WAMUS'11 Proceedings of the 13th WSEAS international conference on mathematical methods, computational techniques and intelligent systems, and 10th WSEAS international conference on non-linear analysis, non-linear systems and chaos, and 7th WSEAS international conference on dynamical systems and control, and 11th WSEAS international conference on Wavelet analysis and multirate systems: recent researches in computational techniques, non-linear systems and control
Validated decision trees versus collective decisions

ICCCI'11 Proceedings of the Third international conference on Computational collective intelligence: technologies and applications - Volume Part II
Identifying mislabeled training data with the aid of unlabeled data

Applied Intelligence
Experimental evaluation of time-series decision tree

AM'03 Proceedings of the Second international conference on Active Mining
Using boolean differences for discovering ill-defined attributes in propositional machine learning

MICAI'05 Proceedings of the 4th Mexican international conference on Advances in Artificial Intelligence
VisTree: generic decision tree inducer and visualizer

DNIS'10 Proceedings of the 6th international conference on Databases in Networked Information Systems
Approximate boolean reasoning: foundations and applications in data mining

Transactions on Rough Sets V
Evaluation of decision tree pruning with subadditive penalties

IDEAL'06 Proceedings of the 7th international conference on Intelligent Data Engineering and Automated Learning
A statistical approach to growing a reliable honest tree

Computational Statistics & Data Analysis
An evidential reasoning based classification algorithm and its application for face recognition with class noise

Pattern Recognition
An ensemble of decision cluster crotches for classification of high dimensional data

Knowledge-Based Systems
Ensemble-based noise detection: noise ranking and visual performance evaluation

Data Mining and Knowledge Discovery

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper compares five methods for pruning decision trees, developed from sets of examples. When used with uncertain rather than deterministic data, decision-tree induction involves three main stages—creating a complete tree able to classify all the training examples, pruning this tree to give statistical reliability, and processing the pruned tree to improve understandability. This paper concerns the second stage—pruning. It presents empirical comparisons of the five methods across several domains. The results show that three methods—critical value, error complexity and reduced error—perform well, while the other two may cause problems. They also show that there is no significant interaction between the creation and pruning methods.