The feature selection problem: traditional methods and a new algorithm

Authors:
Kenji Kira;Larry A. Rendell
Affiliations:
Computer & Information Systems Laboratory, Mitsubishi Electric Corporation, Kanagawa, Japan;Beckman Institute and Department of Computer Science, University of Illinois at Urbana-Champaign, Urbana, IL
Venue:
AAAI'92 Proceedings of the tenth national conference on Artificial intelligence
Year:
1992

Citing 13
Cited 82

Exemplar based knowledge acquisition: a unified approach to concept representati on, classification, and learning

Exemplar based knowledge acquisition: a unified approach to concept representati on, classification, and learning
Improving the design of similarity-based rule-learning systems

International Journal of Expert Systems
Concept learning and heuristic classification in weak-theory domains

Artificial Intelligence
Incremental, instance-based learning of independent and graded concept descriptions

Proceedings of the sixth international workshop on Machine learning
Instance-Based Learning Algorithms

Machine Learning
The replication problem: a constructive induction approach

EWSL-91 Proceedings of the European working session on learning on Machine learning
Learning hard concepts through constructive induction: framework and rationale

Computational Intelligence
A practical approach to feature selection

ML92 Proceedings of the ninth international workshop on Machine learning
Incremental Learning from Noisy Data

Machine Learning
Learning DNF by decision trees

IJCAI'89 Proceedings of the 11th international joint conference on Artificial intelligence - Volume 1
Constructive induction on decision trees

IJCAI'89 Proceedings of the 11th international joint conference on Artificial intelligence - Volume 1
CABOT: an adaptive approach to case-based search

IJCAI'91 Proceedings of the 12th international joint conference on Artificial intelligence - Volume 2
Learning and representation change

AAAI'87 Proceedings of the sixth National conference on Artificial intelligence - Volume 2

Automatic Construction of Decision Trees from Data: A Multi-Disciplinary Survey

Data Mining and Knowledge Discovery
A hybrid approach for feature subset selection using neural networks and ant colony optimization

Expert Systems with Applications: An International Journal
Integrating support vector machines and neural networks

Neural Networks
Machine learning methods for microbial source tracking

Environmental Modelling & Software
Incremental GRLVQ: Learning relevant features for 3D object recognition

Neurocomputing
Review: Dimensionality reduction based on rough set theory: A review

Applied Soft Computing
Performance of feature-selection methods in the classification of high-dimension data

Pattern Recognition
Evolutionary-based feature selection approaches with new criteria for data mining: A case study of credit approval data

Expert Systems with Applications: An International Journal
Machine learning in prognosis of the femoral neck fracture recovery

Artificial Intelligence in Medicine
Exact and approximate discrete optimization algorithms for finding useful disjunctions of categorical predicates in data analysis

Discrete Applied Mathematics
Positive approximation: An accelerator for attribute reduction in rough set theory

Artificial Intelligence
Optimizing reservoir features in oil exploration management based on fusion of soft computing

Applied Soft Computing
Correlation based feature selection method

International Journal of Bio-Inspired Computation
A new dataset evaluation method based on category overlap

Computers in Biology and Medicine
Multi-objective semi-supervised feature selection and model selection based on Pearson's correlation coefficient

CIARP'10 Proceedings of the 15th Iberoamerican congress conference on Progress in pattern recognition, image analysis, computer vision, and applications
Feature extraction for novelty detection as applied to fault detection in machinery

Pattern Recognition Letters
An efficient accelerator for attribute reduction from incomplete data in rough set framework

Pattern Recognition
Application of wrapper methods for feature selection in modelling ripening process of a viticulture crop

ACACOS'11 Proceedings of the 10th WSEAS international conference on Applied computer and applied computational science
Pattern recognition in wireless sensor networks in presence of sensor failures

NNECFSIC'12 Proceedings of the 12th WSEAS international conference on Neural networks, fuzzy systems, evolutionary computing & automation
Classification of infectious diseases based on chemiluminescent signatures of phagocytes in whole blood

Artificial Intelligence in Medicine
Use of the FRiS-function for taxonomy, attribute selection and decision rule construction

KONT'07/KPP'07 Proceedings of the First international conference on Knowledge processing and data analysis
Minimizing calibration time for brain reading

DAGM'11 Proceedings of the 33rd international conference on Pattern recognition
Sequential classifier combination for pattern recognition in wireless sensor networks

MCS'11 Proceedings of the 10th international conference on Multiple classifier systems
Feature selection for interpatient supervised heart beat classification

Computational Intelligence and Neuroscience - Special issue on Selected Papers from the 4th International Conference on Bioinspired Systems and Cognitive Signal Processing
Predict on-shelf product availability in grocery retailing with classification methods

Expert Systems with Applications: An International Journal
Test-retest reliability and feature selection in physiological time series classification

Computer Methods and Programs in Biomedicine
Sequential support vector machine classification for small-grain weed species discrimination with special regard to Cirsium arvense and Galium aparine

Computers and Electronics in Agriculture
Orthogonal relief algorithm for feature selection

ICIC'06 Proceedings of the 2006 international conference on Intelligent Computing - Volume Part I
Robust SVM-based biomarker selection with noisy mass spectrometric proteomic data

EuroGP'06 Proceedings of the 2006 international conference on Applications of Evolutionary Computing
Using ensemble feature selection approach in selecting subset with relevant features

ISNN'06 Proceedings of the Third international conference on Advances in Neural Networks - Volume Part I
A method for feature selection on microarray data using support vector machine

DaWaK'06 Proceedings of the 8th international conference on Data Warehousing and Knowledge Discovery
Effective feature preprocessing for time series forecasting

ADMA'06 Proceedings of the Second international conference on Advanced Data Mining and Applications
Entropy on covers

Data Mining and Knowledge Discovery
Feature selection based on relative attribute dependency: an experimental study

RSFDGrC'05 Proceedings of the 10th international conference on Rough Sets, Fuzzy Sets, Data Mining, and Granular Computing - Volume Part I
Efficient case based feature construction

ECML'05 Proceedings of the 16th European conference on Machine Learning
Evaluate with confidence estimation: machine ranking of translation outputs using grammatical features

WMT '11 Proceedings of the Sixth Workshop on Statistical Machine Translation
Feature selection method using preferences aggregation

MLDM'05 Proceedings of the 4th international conference on Machine Learning and Data Mining in Pattern Recognition
E-Coli promoter recognition using neural networks with feature selection

ICIC'05 Proceedings of the 2005 international conference on Advances in Intelligent Computing - Volume Part II
Supporting generalized cases in conversational CBR

MICAI'05 Proceedings of the 4th Mexican international conference on Advances in Artificial Intelligence
Personalized implicit learning in a music recommender system

UMAP'10 Proceedings of the 18th international conference on User Modeling, Adaptation, and Personalization
A study of applying dimensionality reduction to restrict the size of a hypothesis space

ILP'05 Proceedings of the 15th international conference on Inductive Logic Programming
A life-long learning vector quantization approach for interactive learning of multiple categories

Neural Networks
Temporal data mining for smart homes

Designing Smart Homes
Analysis of feature weighting methods based on feature ranking methods for classification

ICONIP'11 Proceedings of the 18th international conference on Neural Information Processing - Volume Part II
Feature selection for dimensionality reduction

SLSFS'05 Proceedings of the 2005 international conference on Subspace, Latent Structure and Feature Selection
An unsupervised feature selection framework based on clustering

PAKDD'11 Proceedings of the 15th international conference on New Frontiers in Applied Data Mining
Large-margin feature selection for monotonic classification

Knowledge-Based Systems
Feature selection for MAUC-oriented classification systems

Neurocomputing
An efficient rough feature selection algorithm with a multi-granulation view

International Journal of Approximate Reasoning
Application of global optimization methods to model and feature selection

Pattern Recognition
Inferring disease-related metabolite dependencies with a bayesian optimization algorithm

EvoBIO'12 Proceedings of the 10th European conference on Evolutionary Computation, Machine Learning and Data Mining in Bioinformatics
Machine learning for medical diagnosis: history, state of the art and perspective

Artificial Intelligence in Medicine
Efficient feature selection filters for high-dimensional data

Pattern Recognition Letters
A global-ranking local feature selection method for text categorization

Expert Systems with Applications: An International Journal
An adaption of relief for redundant feature elimination

ISNN'12 Proceedings of the 9th international conference on Advances in Neural Networks - Volume Part II
Rough Rule Extracting From Various Conditions: Incremental and Approximate Approaches for Inconsistent Data

Fundamenta Informaticae
Selecting feature subset for high dimensional data via the propositional FOIL rules

Pattern Recognition
A New Rough Sets Model Based on Database Systems

Fundamenta Informaticae - The 9th International Conference on Rough Sets, Fuzzy Sets, Data Mining and Granular Conputing (RSFDGrC 2003)
Feature selection on node statistics based embedding of graphs

Pattern Recognition Letters
Sensor selection to support practical use of health-monitoring smart environments

Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery
Improving neural networks classification through chaining

ICANN'12 Proceedings of the 22nd international conference on Artificial Neural Networks and Machine Learning - Volume Part II
Assisted descriptor selection based on visual comparative data analysis

EuroVis'11 Proceedings of the 13th Eurographics / IEEE - VGTC conference on Visualization
Improving fuzzy multilevel graph embedding through feature selection technique

SSPR'12/SPR'12 Proceedings of the 2012 Joint IAPR international conference on Structural, Syntactic, and Statistical Pattern Recognition
A genetic programming approach to hyper-heuristic feature selection

SEAL'12 Proceedings of the 9th international conference on Simulated Evolution and Learning
Intelligent Decision Support System for Osteoporosis Prediction

International Journal of Intelligent Information Technologies
Studying User Footprints in Different Online Social Networks

ASONAM '12 Proceedings of the 2012 International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2012)
2013 Special Issue: Methods for pattern selection, class-specific feature selection and classification for automated learning

Neural Networks
Human action recognition optimization based on evolutionary feature subset selection

Proceedings of the 15th annual conference on Genetic and evolutionary computation
Optimal joint selection for skeletal data from RGB-D devices using a genetic algorithm

MICAI'12 Proceedings of the 11th Mexican international conference on Advances in Computational Intelligence - Volume Part II
Feature selection using misclassification counts

AusDM '11 Proceedings of the Ninth Australasian Data Mining Conference - Volume 121
A generic adaptive simulation algorithm for component-based simulation systems

Proceedings of the 2013 ACM SIGSIM conference on Principles of advanced discrete simulation
Discovering health-related knowledge in social media using ensembles of heterogeneous features

Proceedings of the 22nd ACM international conference on Conference on information & knowledge management
On fuzzy-rough attribute selection: Criteria of Max-Dependency, Max-Relevance, Min-Redundancy, and Max-Significance

Applied Soft Computing
Class dependent feature weighting and k-nearest neighbor classification

PRIB'13 Proceedings of the 8th IAPR international conference on Pattern Recognition in Bioinformatics
Simultaneous sample and gene selection using t-score and approximate support vectors

PRIB'13 Proceedings of the 8th IAPR international conference on Pattern Recognition in Bioinformatics
Evolutionary joint selection to improve human action recognition with RGB-D devices

Expert Systems with Applications: An International Journal
Feature selection with test cost constraint

International Journal of Approximate Reasoning
Feature selection for high-dimensional multi-category data using PLS-based local recursive feature elimination

Expert Systems with Applications: An International Journal
PLS-based recursive feature elimination for high-dimensional small sample

Knowledge-Based Systems
A survey on feature selection methods

Computers and Electrical Engineering
A real-time transportation prediction system

Applied Intelligence
A novel feature subset selection algorithm based on association rule mining

Intelligent Data Analysis

Quantified Score

Hi-index	0.01

Visualization

Abstract

For real-world concept learning problems, feature selection is important to speed up learning and to improve concept quality. We review and analyze past approaches to feature selection and note their strengths and weaknesses. We then introduce and theoretically examine a new algorithm Rellef which selects relevant features using a statistical method. Relief does not depend on heuristics, is accurate even if features interact, and is noise-tolerant. It requires only linear time in the number of given features and the number of training instances, regardless of the target concept complexity. The algorithm also has certain limitations such as nonoptimal feature set size. Ways to overcome the limitations are suggested. We also report the test results of comparison between Relief and other feature selection algorithms. The empirical results support the theoretical analysis, suggesting a practical approach to feature selection for real-world problems.