Incremental Induction of Decision Trees

Authors:
Paul E. Utgoff
Affiliations:
Department of Computer and Information Science, University of Massachusetts, Amherst, MA 01003. UTGOFF@CS.UMASS.EDU
Venue:
Machine Learning
Year:
1989

Citing 0
Cited 120

Coping with change

ICAIL '91 Proceedings of the 3rd international conference on Artificial intelligence and law
The HCV induction algorithm

CSC '93 Proceedings of the 1993 ACM conference on Computer science
Detecting change in legal concepts

ICAIL '95 Proceedings of the 5th international conference on Artificial intelligence and law
A Review and Empirical Evaluation of Feature Weighting Methods for aClass of Lazy Learning Algorithms

Artificial Intelligence Review - Special issue on lazy learning
Decision Tree Induction Based on Efficient Tree Restructuring

Machine Learning
Learning to remove Internet advertisements

Proceedings of the third annual conference on Autonomous Agents
BOAT—optimistic decision tree construction

SIGMOD '99 Proceedings of the 1999 ACM SIGMOD international conference on Management of data
Classification and regression: money *can* grow on trees

KDD '99 Tutorial notes of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining
Adaptive information filtering: detecting changes in text streams

Proceedings of the eighth international conference on Information and knowledge management
Using the ID3 symbolic classification algorithm to reduce data density

SAC '94 Proceedings of the 1994 ACM symposium on Applied computing
Mining high-speed data streams

Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining
MobiMine: monitoring the stock market from a PDA

ACM SIGKDD Explorations Newsletter
A Hybrid Architecture for Situated Learning of Reactive Sequential Decision Making

Applied Intelligence
Automatic Construction of Decision Trees from Data: A Multi-Disciplinary Survey

Data Mining and Knowledge Discovery
A Survey of Methods for Scaling Up Inductive Algorithms

Data Mining and Knowledge Discovery
On Issues of Instance Selection

Data Mining and Knowledge Discovery
Discretization: An Enabling Technique

Data Mining and Knowledge Discovery
Multi-Layer Hierarchical Rule Learning in Reactive Robot Control Using Incremental Decision Trees

Journal of Intelligent and Robotic Systems
Pasting Small Votes for Classification in Large Databases and On-Line

Machine Learning
An incremental learning algorithm for constructing boolean functions from positive and negative examples

Computers and Operations Research
Decision Trees: An Overview and Their Use in Medicine

Journal of Medical Systems
On the power of incremental learning

Theoretical Computer Science
An Efficient Inductive Learning Method for Object-Oriented Database Using Attribute Entropy

IEEE Transactions on Knowledge and Data Engineering
Symbolic Learning Techniques in Paper Document Processing

MLDM '99 Proceedings of the First International Workshop on Machine Learning and Data Mining in Pattern Recognition
Induction as Pre-processing

PAKDD '99 Proceedings of the Third Pacific-Asia Conference on Methodologies for Knowledge Discovery and Data Mining
Variants of iterative learning

Theoretical Computer Science
Mining and monitoring evolving data

Handbook of massive data sets
Cancer classification using gene expression data

Information Systems - Special issue: Data management in bioinformatics
Controlled Flux Results in Stable Decision Trees

ICTAI '99 Proceedings of the 11th IEEE International Conference on Tools with Artificial Intelligence
Mining dynamic databases by weighting

Acta Cybernetica
References

Intelligent data analysis
Online Ensemble Learning: An Empirical Study

Machine Learning
Post-mining: maintenance of association rules by wieghting

Information Systems
Mining concept-drifting data streams using ensemble classifiers

Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining
Learning systems and their engineering: a project proposal

Practicing software engineering in the 21st century
A Fourier Spectrum-Based Approach to Represent Decision Trees for Mining Data Streams in Mobile Environments

IEEE Transactions on Knowledge and Data Engineering
Efficient Multisplitting Revisited: Optima-Preserving Elimination of Partition Candidates

Data Mining and Knowledge Discovery
A review of machine learning

The Knowledge Engineering Review
Becoming incrementally reactive: on-line learning of an evolving decision tree array for robot navigation

Robotica
RRIA: a rough set and rule tree based incremental knowledge acquisition algorithm

Fundamenta Informaticae - Special issue on the 9th international conference on rough sets, fuzzy sets, data mining and granular computing (RSFDGrC 2003)
Clustering in Dynamic Spatial Databases

Journal of Intelligent Information Systems
Interruptible anytime algorithms for iterative improvement of decision trees

UBDM '05 Proceedings of the 1st international workshop on Utility-based data mining
Dynamic feature selection for hardware prediction

Journal of Systems Architecture: the EUROMICRO Journal
Learning the structure of Factored Markov Decision Processes in reinforcement learning problems

ICML '06 Proceedings of the 23rd international conference on Machine learning
On the Computational Complexity of Optimal Multisplitting

Fundamenta Informaticae - Intelligent Systems
Anytime Learning of Decision Trees

The Journal of Machine Learning Research
Machine learning based feature extraction for quality control in a production line

Integrated Computer-Aided Engineering
Constructing a Fuzzy Rule System from Examples

Integrated Computer-Aided Engineering
Learning consumer preferences using semantic similarity

Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems
Networked database builder and data-mining engine for electronic journal papers

International Journal of Computational Science and Engineering
Boosting classifiers for drifting concepts

Intelligent Data Analysis - Knowlegde Discovery from Data Streams
Efficient instance-based learning on data streams

Intelligent Data Analysis
Methodological Review: Evolution and challenges in the design of computational systems for triage assistance

Journal of Biomedical Informatics
Pessimistic cost-sensitive active learning of decision trees for profit maximizing targeting campaigns

Data Mining and Knowledge Discovery
An Efficient and Sensitive Decision Tree Approach to Mining Concept-Drifting Data Streams

Informatica
An Incremental Fuzzy Decision Tree Classification Method for Mining Data Streams

MLDM '07 Proceedings of the 5th international conference on Machine Learning and Data Mining in Pattern Recognition
CASE-BASED REASONING FOR PREDICTING MULTIPERIOD FINANCIAL PERFORMANCES OF TECHNOLOGY-BASED SMEs

Applied Artificial Intelligence
Mining decision rules on data streams in the presence of concept drifts

Expert Systems with Applications: An International Journal
Recognition of multi-interval rules in dataset with continuous-valued attributes

Expert Systems with Applications: An International Journal
Knowledge-based system for text classification using ID6NB algorithm

Knowledge-Based Systems
Toward an optimum combination of English teachers for objective teaching

Expert Systems with Applications: An International Journal
Hierarchical Classifiers for Complex Spatio-temporal Concepts

Transactions on Rough Sets IX
A Multi-partition Multi-chunk Ensemble Technique to Classify Concept-Drifting Data Streams

PAKDD '09 Proceedings of the 13th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining
NetPrints: diagnosing home network misconfigurations using shared knowledge

NSDI'09 Proceedings of the 6th USENIX symposium on Networked systems design and implementation
Decision Tree Learning Using a Bayesian Approach at Each Node

Canadian AI '09 Proceedings of the 22nd Canadian Conference on Artificial Intelligence: Advances in Artificial Intelligence
Anticipatory Learning Classifier Systems and Factored Reinforcement Learning

Anticipatory Behavior in Adaptive Learning Systems
Any time induction of decision trees: an iterative improvement approach

AAAI'06 Proceedings of the 21st national conference on Artificial intelligence - Volume 1
Effective solution for unhandled exception in decision tree induction algorithms

Expert Systems with Applications: An International Journal
Discovery of Significant Classification Rules from Incrementally Inducted Decision Tree Ensemble for Diagnosis of Disease

ADMA '09 Proceedings of the 5th International Conference on Advanced Data Mining and Applications
Considering Unseen States as Impossible in Factored Reinforcement Learning

ECML PKDD '09 Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases: Part I
Using AI for e-government automatic assessment of immigration application forms

IAAI'07 Proceedings of the 19th national conference on Innovative applications of artificial intelligence - Volume 2
Alignment algorithms for learning to read aloud

IJCAI'97 Proceedings of the Fifteenth international joint conference on Artifical intelligence - Volume 2
Integrating learning from examples into the search for diagnostic policies

Journal of Artificial Intelligence Research
Incremental learning in a fuzzy intelligent system

IJCAI'99 Proceedings of the 16th international joint conference on Artificial intelligence - Volume 2
Learning nested concept classes with limited storage

IJCAI'91 Proceedings of the 12th international joint conference on Artificial intelligence - Volume 2
Top-down induction of first-order logical decision trees

Artificial Intelligence
HE-Tree: a framework for detecting changes in clustering structure for categorical data streams

The VLDB Journal — The International Journal on Very Large Data Bases
Machine learning techniques for scheduling jobs with incompatible families and unequal ready times on parallel batch machines

Engineering Applications of Artificial Intelligence
Towards incremental classifier fusion

Intelligent Data Analysis
Mining distributed evolving data streams using fractal GP ensembles

EuroGP'07 Proceedings of the 10th European conference on Genetic programming
AttributeNets: an incremental learning method for interpretable classification

PAKDD'07 Proceedings of the 11th Pacific-Asia conference on Advances in knowledge discovery and data mining
A new fuzzy decision tree classification method for mining high-speed data streams based on binary search trees

FAW'07 Proceedings of the 1st annual international conference on Frontiers in algorithmics
Integrating decision tree learning into inductive databases

KDID'06 Proceedings of the 5th international conference on Knowledge discovery in inductive databases
A new decision tree classification method for mining high-speed data streams based on threaded binary search trees

PAKDD'07 Proceedings of the 2007 international conference on Emerging technologies in knowledge discovery and data mining
Similarities in fuzzy data mining: from a cognitive view to real-world applications

WCCI'08 Proceedings of the 2008 IEEE world conference on Computational intelligence: research frontiers
A neural tree and its application to spam e-mail detection

Expert Systems with Applications: An International Journal
Two case studies in cost-sensitive concept acquisition

AAAI'90 Proceedings of the eighth National conference on Artificial intelligence - Volume 2
Complementary discrimination learning with decision lists

AAAI'92 Proceedings of the tenth national conference on Artificial intelligence
Tree induction over perennial objects

SSDBM'10 Proceedings of the 22nd international conference on Scientific and statistical database management
Active learning from stream data using optimal weight classifier ensemble

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Incremental learning by heterogeneous bagging ensemble

ADMA'10 Proceedings of the 6th international conference on Advanced data mining and applications - Volume Part II
Incremental Gaussian Discriminant Analysis based on Graybill and Deal weighted combination of estimators for brain tumour diagnosis

Journal of Biomedical Informatics
Incremental rule induction based on rough set theory

ISMIS'11 Proceedings of the 19th international conference on Foundations of intelligent systems
Incremental semi-automatic correction of misclassified spatial objects

ICAIS'11 Proceedings of the Second international conference on Adaptive and intelligent systems
Incremental learning in AttributeNets with dynamic reduct and IQuickReduct

RSKT'11 Proceedings of the 6th international conference on Rough sets and knowledge technology
Avoiding order effects in incremental learning

AI*IA'05 Proceedings of the 9th conference on Advances in Artificial Intelligence
Online learning objectionable image filter based on SVM

PCM'04 Proceedings of the 5th Pacific Rim conference on Advances in Multimedia Information Processing - Volume Part I
A method for compiling general concepts with the use of temporal decision trees

Scientific and Technical Information Processing
Networked mining of atomic and molecular data from electronic journal databases on the internet

DNIS'05 Proceedings of the 4th international conference on Databases in Networked Information Systems
Self-reproduction for articulated behaviors with dual humanoid robots using on-line decision tree classification

Robotica
Incremental versus non-incremental rule induction for multicriteria classification

Transactions on Rough Sets II
Handling different categories of concept drifts in data streams using distributed GP

EuroGP'10 Proceedings of the 13th European conference on Genetic Programming
On the benefits of argumentation-derived evidence in learning policies

ArgMAS'10 Proceedings of the 7th international conference on Argumentation in Multi-Agent Systems
Evolutionary design of decision trees for medical application

Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery
Effect of data discretization on the classification accuracy in a high-dimensional framework

International Journal of Intelligent Systems
Hybrid active learning for reducing the annotation effort of operators in classification systems

Pattern Recognition
Learning rules of a card game from video

Artificial Intelligence Review
An aggressive margin-based algorithm for incremental learning

PAKDD'12 Proceedings of the 16th Pacific-Asia conference on Advances in Knowledge Discovery and Data Mining - Volume Part I
Rough Rule Extracting From Various Conditions: Incremental and Approximate Approaches for Inconsistent Data

Fundamenta Informaticae
Incremental rules induction based on rule layers

RSKT'12 Proceedings of the 7th international conference on Rough Sets and Knowledge Technology
RRIA: A Rough Set and Rule Tree Based Incremental Knowledge Acquisition Algorithm

Fundamenta Informaticae - The 9th International Conference on Rough Sets, Fuzzy Sets, Data Mining and Granular Conputing (RSFDGrC 2003)
On the Computational Complexity of Optimal Multisplitting

Fundamenta Informaticae - Intelligent Systems
Incremental rules induction method based on three rule layers

ISMIS'12 Proceedings of the 20th international conference on Foundations of Intelligent Systems
A selective Bayes classifier with meta-heuristics for incomplete data

Neurocomputing
When Is the Right Time to Refresh Knowledge Discovered from Data?

Operations Research
Which work-item updates need your response?

Proceedings of the 10th Working Conference on Mining Software Repositories
Hybrid decision tree and naïve Bayes classifiers for multi-class classification tasks

Expert Systems with Applications: An International Journal
An incremental decision tree algorithm based on rough sets and its application in intrusion detection

Artificial Intelligence Review
Incremental Induction of Medical Diagnostic Rules Based on Incremental Sampling Scheme and SubRule Layers

Fundamenta Informaticae - To Andrzej Skowron on His 70th Birthday
Building fast decision trees from large training sets

Intelligent Data Analysis

Quantified Score

Hi-index	0.01

Visualization

Abstract

This article presents an incremental algorithm for inducing decision trees equivalent to those formed by Quinlan's nonincremental ID3 algorithm, given the same training instances. The new algorithm, named ID5R, lets one apply the ID3 induction process to learning tasks in which training instances are presented serially. Although the basic tree-building algorithms differ only in how the decision trees are constructed, experiments show that incremental training makes it possible to select training instances more carefully, which can result in smaller decision trees. The ID3 algorithm and its variants are compared in terms of theoretical complexity and empirical behavior.