Comparison of Shannon, Renyi and Tsallis Entropy Used in Decision Trees

Authors:
Tomasz Maszczyk;Włodzisław Duch
Affiliations:
Department of Informatics, Nicolaus Copernicus University, Toruń, Poland 87-100;Department of Informatics, Nicolaus Copernicus University, Toruń, Poland 87-100
Venue:
ICAISC '08 Proceedings of the 9th international conference on Artificial Intelligence and Soft Computing
Year:
2006

Citing 5
Cited 2

C4.5: programs for machine learning

C4.5: programs for machine learning
A Mathematical Theory of Communication

A Mathematical Theory of Communication
Feature Extraction Using Information-Theoretic Learning

IEEE Transactions on Pattern Analysis and Machine Intelligence
Learning highly non-separable Boolean functions using constructive feedforward neural network

ICANN'07 Proceedings of the 17th international conference on Artificial neural networks
Generalized information potential criterion for adaptive system training

IEEE Transactions on Neural Networks

A hybrid system with regression trees in steel-making process

HAIS'11 Proceedings of the 6th international conference on Hybrid artificial intelligent systems - Volume Part I
Automatic sleep staging: A computer assisted approach for optimal combination of features and polysomnographic channels

Expert Systems with Applications: An International Journal

Quantified Score

Hi-index	0.00

Visualization

Abstract

Shannon entropy used in standard top-down decision trees does not guarantee the best generalization. Split criteria based on generalized entropies offer different compromise between purity of nodes and overall information gain. Modified C4.5 decision trees based on Tsallis and Renyi entropies have been tested on several high-dimensional microarray datasets with interesting results. This approach may be used in any decision tree and information selection algorithm.