A practical approach to feature selection
ML92 Proceedings of the ninth international workshop on Machine learning
Estimating attributes: analysis and extensions of RELIEF
ECML-94 Proceedings of the European conference on machine learning on Machine Learning
Feature Selection: Evaluation, Application, and Small Sample Performance
IEEE Transactions on Pattern Analysis and Machine Intelligence
Artificial Intelligence Review - Special issue on lazy learning
Artificial Intelligence Review - Special issue on lazy learning
A decision-theoretic generalization of on-line learning and an application to boosting
Journal of Computer and System Sciences - Special issue: 26th annual ACM symposium on the theory of computing & STOC'94, May 23–25, 1994, and second annual Europe an conference on computational learning theory (EuroCOLT'95), March 13–15, 1995
Wrappers for feature subset selection
Artificial Intelligence - Special issue on relevance
Neural Networks for Pattern Recognition
Neural Networks for Pattern Recognition
Linear Programming Boosting via Column Generation
Machine Learning
Novel Methods for Subset Selection with Respect to Problem Knowledge
IEEE Intelligent Systems
An introduction to variable and feature selection
The Journal of Machine Learning Research
Pattern Classification (2nd Edition)
Pattern Classification (2nd Edition)
Margin based feature selection - theory and algorithms
ICML '04 Proceedings of the twenty-first international conference on Machine learning
Unifying the error-correcting and output-code AdaBoost within the margin framework
ICML '05 Proceedings of the 22nd international conference on Machine learning
On-line EM Algorithm for the Normalized Gaussian Network
Neural Computation
Iterative RELIEF for feature weighting
ICML '06 Proceedings of the 23rd international conference on Machine learning
Solving multiclass learning problems via error-correcting output codes
Journal of Artificial Intelligence Research
Discriminatory Data Mapping by Matrix-Based Supervised Learning Metrics
ANNPR '08 Proceedings of the 3rd IAPR workshop on Artificial Neural Networks in Pattern Recognition
Online Feature Selection Algorithm with Bayesian l 1 Regularization
PAKDD '09 Proceedings of the 13th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining
Matrix Metric Adaptation Linear Discriminant Analysis of Biomedical Data
IWANN '09 Proceedings of the 10th International Work-Conference on Artificial Neural Networks: Part I: Bio-Inspired Systems: Computational and Ambient Intelligence
Design of input vector for day-ahead price forecasting of electricity markets
Expert Systems with Applications: An International Journal
Optimal feature selection for support vector machines
Pattern Recognition
Selecting discrete and continuous features based on neighborhood decision error minimization
IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Gaussian kernel based fuzzy rough sets: Model, uncertainty measures and applications
International Journal of Approximate Reasoning
Efficient feature selection in the presence of outliers and noises
AIRS'08 Proceedings of the 4th Asia information retrieval conference on Information retrieval technology
Feature selection by nonparametric Bayes error minimization
PAKDD'08 Proceedings of the 12th Pacific-Asia conference on Advances in knowledge discovery and data mining
Sparse kernel-based feature weighting
PAKDD'08 Proceedings of the 12th Pacific-Asia conference on Advances in knowledge discovery and data mining
Generalized iterative RELIEF for supervised distance metric learning
Pattern Recognition
Collaborative optimization of clustering by fuzzy c-means and weight determination by ReliefF
FSKD'09 Proceedings of the 6th international conference on Fuzzy systems and knowledge discovery - Volume 1
Feature selection for Bayesian network classifiers using the MDL-FS score
International Journal of Approximate Reasoning
Energy Supervised Relevance Neural Gas for Feature Ranking
Neural Processing Letters
Robust relief-feature weighting, margin maximization, and fuzzy optimization
IEEE Transactions on Fuzzy Systems
Soft fuzzy rough sets for robust feature evaluation and selection
Information Sciences: an International Journal
Review Article: Stable feature selection for biomarker discovery
Computational Biology and Chemistry
Journal of Biomedical Informatics
A new prediction strategy for price spike forecasting of day-ahead electricity markets
Applied Soft Computing
Improvements over adaptive local hyperplane to achieve better classification
ICDM'11 Proceedings of the 11th international conference on Advances in data mining: applications and theoretical aspects
Feature selection for support vector machines with RBF kernel
Artificial Intelligence Review
TAKES: a fast method to select features in the kernel space
Proceedings of the 20th ACM international conference on Information and knowledge management
Feature evaluation and selection with cooperative game theory
Pattern Recognition
Large-margin feature selection for monotonic classification
Knowledge-Based Systems
Financial ratio selection for business crisis prediction
Expert Systems with Applications: An International Journal
A boosting approach for supervised Mahalanobis distance metric learning
Pattern Recognition
A RELIEF-based modality weighting approach for multimodal information retrieval
Proceedings of the 2nd ACM International Conference on Multimedia Retrieval
Feature weighting by RELIEF based on local hyperplane approximation
PAKDD'12 Proceedings of the 16th Pacific-Asia conference on Advances in Knowledge Discovery and Data Mining - Volume Part II
On the evolutionary optimization of k-NN by label-dependent feature weighting
Pattern Recognition Letters
Feature selection for link prediction
Proceedings of the 5th Ph.D. workshop on Information and knowledge
Minimum-maximum local structure information for feature selection
Pattern Recognition Letters
RPCA: a novel preprocessing method for PCA
Advances in Artificial Intelligence
Journal of Information Science
Class dependent feature weighting and k-nearest neighbor classification
PRIB'13 Proceedings of the 8th IAPR international conference on Pattern Recognition in Bioinformatics
Robust feature selection based on regularized brownboost loss
Knowledge-Based Systems
Expert Systems with Applications: An International Journal
PLS-based recursive feature elimination for high-dimensional small sample
Knowledge-Based Systems
Joint Laplacian feature weights learning
Pattern Recognition
Novel feature selection methods to financial distress prediction
Expert Systems with Applications: An International Journal
White box radial basis function classifiers with component selection for clinical prediction models
Artificial Intelligence in Medicine
MaskedPainter: Feature selection for microarray data analysis
Intelligent Data Analysis
Feature ranking fusion for text classifier
Intelligent Data Analysis
| Hi-index | 0.15 |
RELIEF is considered one of the most successful algorithms for assessing the quality of features. In this paper, we propose a set of new feature weighting algorithms that perform significantly better than RELIEF, without introducing a large increase in computational complexity. Our work starts from a mathematical interpretation of the seemingly heuristic RELIEF algorithm as an online method solving a convex optimization problem with a margin-based objective function. This interpretation explains the success of RELIEF in real application and enables us to identify and address its following weaknesses. RELIEF makes an implicit assumption that the nearest neighbors found in the original feature space are the ones in the weighted space and RELIEF lacks a mechanism to deal with outlier data. We propose an iterative RELIEF (I-RELIEF) algorithm to alleviate the deficiencies of RELIEF by exploring the framework of the Expectation-Maximization algorithm. We extend I-RELIEF to multiclass settings by using a new multiclass margin definition. To reduce computational costs, an online learning algorithm is also developed. Convergence analysis of the proposed algorithms is presented. The results of large-scale experiments on the UCI and microarray data sets are reported, which demonstrate the effectiveness of the proposed algorithms, and verify the presented theoretical results.