Overcoming the Myopia of Inductive Learning Algorithms with RELIEFF

Authors:
Igor Kononenko;Edvard Šimec;Marko Robnik-Šikonja
Affiliations:
University of Ljubljana, Faculty of Computer and Information Science, Tržaška 25, SI-1001 Ljubljana, Slovenia E-mail: igor.kononenko@fri.uni-lj.si;University of Ljubljana, Faculty of Computer and Information Science, Tržaška 25, SI-1001 Ljubljana, Slovenia E-mail: igor.kononenko@fri.uni-lj.si;University of Ljubljana, Faculty of Computer and Information Science, Tržaška 25, SI-1001 Ljubljana, Slovenia E-mail: igor.kononenko@fri.uni-lj.si
Venue:
Applied Intelligence
Year:
1997

Citing 9
Cited 37

Learning decision rules in noisy domains

Proceedings of Expert Systems '86, The 6Th Annual Technical Conference on Research and development in expert systems III
Information-Based Evaluation Criterion for Classifier's Performance

Machine Learning
On estimating probabilities in tree pruning

EWSL-91 Proceedings of the European working session on learning on Machine learning
A practical approach to feature selection

ML92 Proceedings of the ninth international workshop on Machine learning
Estimating attributes: analysis and extensions of RELIEF

ECML-94 Proceedings of the European conference on machine learning on Machine Learning
Machine learning, neural and statistical classification

Machine learning, neural and statistical classification
Use of Contextual Information for Feature Ranking and Discretization

IEEE Transactions on Knowledge and Data Engineering
Induction of Decision Trees

Machine Learning
On biases in estimating multi-valued attributes

IJCAI'95 Proceedings of the 14th international joint conference on Artificial intelligence - Volume 2

A Machine Learning Approach to POS Tagging

Machine Learning
A Survey of Methods for Scaling Up Inductive Algorithms

Data Mining and Knowledge Discovery
A Formalism for Relevance and Its Application in Feature Subset Selection

Machine Learning
Use of Contextual Information for Feature Ranking and Discretization

IEEE Transactions on Knowledge and Data Engineering
Reliable Classifications with Machine Learning

ECML '02 Proceedings of the 13th European Conference on Machine Learning
Machine Learning for Sequential Data: A Review

Proceedings of the Joint IAPR International Workshop on Structural, Syntactic, and Statistical Pattern Recognition
Ensemble Feature Selection Based on Contextual Merit and Correlation Heuristics

ADBIS '01 Proceedings of the 5th East European Conference on Advances in Databases and Information Systems
Function Decomposition in Machine Learning

Machine Learning and Its Applications, Advanced Lectures
Evaluation of Prognostic Factors and Prediction of Chronic Wound Healing Rate by Machine Learning Tools

AIME '01 Proceedings of the 8th Conference on AI in Medicine in Europe: Artificial Intelligence Medicine
An application of machine learning in the diagnosis of ischaemic heart disease

CBMS '97 Proceedings of the 10th IEEE Symposium on Computer-Based Medical Systems (CBMS '97)
Theoretical and Empirical Analysis of ReliefF and RReliefF

Machine Learning
A selective sampling approach to active feature selection

Artificial Intelligence
Automatic web pages categorization with ReliefF and Hidden Naive Bayes

Proceedings of the 2007 ACM symposium on Applied computing
Anytime Learning of Decision Trees

The Journal of Machine Learning Research
From machine learning to knowledge discovery: Survey of preprocessing and postprocessing

Intelligent Data Analysis
FreeViz-An intelligent multivariate visualization approach to explorative analysis of biomedical data

Journal of Biomedical Informatics
Image statistics and data mining of anal intraepithelial neoplasia

Pattern Recognition Letters
Can feature information interaction help for information fusion in multimedia problems?

Multimedia Tools and Applications
On the value of combining feature subset selection with genetic algorithms: faster learning of coverage models

PROMISE '09 Proceedings of the 5th International Conference on Predictor Models in Software Engineering
Design of input vector for day-ahead price forecasting of electricity markets

Expert Systems with Applications: An International Journal
OWA rough set model for forecasting the revenues growth rate of the electronic industry

Expert Systems with Applications: An International Journal
Attribute selection with fuzzy decision reducts

Information Sciences: an International Journal
Generalized iterative RELIEF for supervised distance metric learning

Pattern Recognition
Traversability: A Case Study for Learning and Perceiving Affordances in Robots

Adaptive Behavior - Animals, Animats, Software Agents, Robots, Adaptive Systems
A new dataset evaluation method based on category overlap

Computers in Biology and Medicine
Implementing ReliefF filters to extract meaningful features from genetic lifetime datasets

Journal of Biomedical Informatics
Using reliable short rules to avoid unnecessary tests in decision trees

MICAI'06 Proceedings of the 5th Mexican international conference on Artificial Intelligence
A comprehensively sized decision tree generation method for interactive data mining of very large databases

ADMA'05 Proceedings of the First international conference on Advanced Data Mining and Applications
COW: a co-evolving memetic wrapper for herb-herb interaction analysis in TCM informatics

PAKDD'11 Proceedings of the 15th international conference on New Frontiers in Applied Data Mining
Analyzing Online Review Helpfulness Using a Regressional ReliefF-Enhanced Text Mining Method

ACM Transactions on Management Information Systems (TMIS)
Comprehensible evaluation of prognostic factors and prediction of wound healing

Artificial Intelligence in Medicine
Evolutionary feature selection for classification: a plug-in hybrid vehicle adoption application

Proceedings of the 14th annual conference on Genetic and evolutionary computation
An adaption of relief for redundant feature elimination

ISNN'12 Proceedings of the 9th international conference on Advances in Neural Networks - Volume Part II
Accurate Prediction of Coronary Artery Disease Using Reliable Diagnosis System

Journal of Medical Systems
A new gene selection method for microarray data based on PSO and informativeness metric

ICIC'13 Proceedings of the 9th international conference on Intelligent Computing Theories and Technology
Finding rough and fuzzy-rough set reducts with SAT

Information Sciences: an International Journal
Nonlinear feature for gait speed estimation using inertial sensors

BodyNets '13 Proceedings of the 8th International Conference on Body Area Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

Current inductive machine learning algorithms typically use greedy search with limited lookahead. This prevents them to detect significant conditional dependencies between the attributes that describe training objects. Instead of myopic impurity functions and lookahead, we propose to use RELIEFF, an extension ofRELIEF developed by Kira and Rendell [10, 11],for heuristic guidance of inductive learningalgorithms. We have reimplemented Assistant, a system for top down induction of decision trees, using RELIEFF as an estimator of attributes at each selection step. The algorithm is tested on several artificial and several real world problems and the results are compared with some other well known machine learning algorithms. Excellent results on artificial data sets and two real world problems show the advantageof the presented approach to inductivelearning.