Locally Adaptive Metric Nearest-Neighbor Classification

Authors:
Carlotta Domeniconi;Jing Peng;Dimitrios Gunopulos
Affiliations:
-;-;-
Venue:
IEEE Transactions on Pattern Analysis and Machine Intelligence
Year:
2002

Citing 7
Cited 67

The multi-class metric problem in nearest neighbour discrimination rules

Pattern Recognition
C4.5: programs for machine learning

C4.5: programs for machine learning
Local learning algorithms

Neural Computation
Similarity metric learning for a variable-kernel classifier

Neural Computation
Bagging predictors

Machine Learning
Discriminant Adaptive Nearest Neighbor Classification

IEEE Transactions on Pattern Analysis and Machine Intelligence
Locally Weighted Learning

Artificial Intelligence Review - Special issue on lazy learning

Adaptive Quasiconformal Kernel Nearest Neighbor Classification

IEEE Transactions on Pattern Analysis and Machine Intelligence
Document clustering via adaptive subspace iteration

Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval
Locally linear metric adaptation for semi-supervised clustering

ICML '04 Proceedings of the twenty-first international conference on Machine learning
Query-sensitive embeddings

Proceedings of the 2005 ACM SIGMOD international conference on Management of data
A general model for clustering binary data

Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining
Boosting the distance estimation

Pattern Recognition Letters
Learning Weighted Metrics to Minimize Nearest-Neighbor Classification Error

IEEE Transactions on Pattern Analysis and Machine Intelligence
Improving nearest neighbor classification with cam weighted distance

Pattern Recognition
Locally linear metric adaptation with application to semi-supervised clustering and image retrieval

Pattern Recognition
Relaxational metric adaptation and its application to semi-supervised clustering and content-based image retrieval

Pattern Recognition
Learning prototypes and distances: A prototype reduction technique based on nearest neighbor error minimization

Pattern Recognition
Prototype selection for dissimilarity-based classifiers

Pattern Recognition
Improving nearest neighbor rule with a simple adaptive distance measure

Pattern Recognition Letters
Simultaneous feature selection and feature weighting using Hybrid Tabu Search/K-nearest neighbor classifier

Pattern Recognition Letters
Query-sensitive embeddings

ACM Transactions on Database Systems (TODS)
Credible Case-Based Inference Using Similarity Profiles

IEEE Transactions on Knowledge and Data Engineering
Neighbor search with global geometry: a minimax message passing algorithm

Proceedings of the 24th international conference on Machine learning
Metric learning by discriminant neighborhood embedding

Pattern Recognition
An efficient weighted nearest neighbour classifier using vertical data representation

International Journal of Business Intelligence and Data Mining
Locally linear reconstruction for instance-based learning

Pattern Recognition
IKNN: Informative K-Nearest Neighbor Pattern Classification

PKDD 2007 Proceedings of the 11th European conference on Principles and Practice of Knowledge Discovery in Databases
Weighted locally linear embedding for dimension reduction

Pattern Recognition
Cluster-based nearest-neighbour classifier and its application on the lightning classification

Journal of Computer Science and Technology
Boosting k-nearest neighbor classifier by means of input space projection

Expert Systems with Applications: An International Journal
A method of learning weighted similarity function to improve the performance of nearest neighbor

Information Sciences: an International Journal
Imbalanced Training Set Reduction and Feature Selection Through Genetic Optimization

Proceedings of the 2005 conference on Artificial Intelligence Research and Development
Graph-Based Discrete Differential Geometry for Critical Instance Filtering

ECML PKDD '09 Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases: Part II
A proposed method of local feature-weighting to improve predictions of basic nearest neighbor rule

ASC '07 Proceedings of The Eleventh IASTED International Conference on Artificial Intelligence and Soft Computing
Coupled Metric Learning for Face Recognition with Degraded Images

ACML '09 Proceedings of the 1st Asian Conference on Machine Learning: Advances in Machine Learning
Kernel Difference-Weighted k-Nearest Neighbors Classification

ICIC '07 Proceedings of the 3rd International Conference on Intelligent Computing: Advanced Intelligent Computing Theories and Applications. With Aspects of Artificial Intelligence
Hierarchical Multi-view Fisher Discriminant Analysis

ICONIP '09 Proceedings of the 16th International Conference on Neural Information Processing: Part II
Application of combined support vector machines in process fault diagnosis

ACC'09 Proceedings of the 2009 conference on American Control Conference
A new fuzzy rule-based initialization method for K-nearest neighbor classifier

FUZZ-IEEE'09 Proceedings of the 18th international conference on Fuzzy Systems
Exploring margin setting for good generalization in multiple class discrimination

Pattern Recognition
Kernel-based metric learning for semi-supervised clustering

Neurocomputing
An efficient nearest neighbor classifier using an adaptive distance measure

CAIP'07 Proceedings of the 12th international conference on Computer analysis of images and patterns
Generalizing dissimilarity representations using feature lines

CIARP'07 Proceedings of the Congress on pattern recognition 12th Iberoamerican conference on Progress in pattern recognition, image analysis and applications
SubClass: classification of multidimensional noisy data using subspace clusters

PAKDD'08 Proceedings of the 12th Pacific-Asia conference on Advances in knowledge discovery and data mining
A novel algorithm based on conditional entropy established by clustering for feature selection

FSKD'09 Proceedings of the 6th international conference on Fuzzy systems and knowledge discovery - Volume 1
Creating diverse nearest-neighbour ensembles using simultaneous metaheuristic feature selection

Pattern Recognition Letters
Semi-supervised learning based on nearest neighbor rule and cut edges

Knowledge-Based Systems
Sparse ensembles using weighted combination methods based on linear programming

Pattern Recognition
Local class boundaries for support vector machine

LSMS/ICSEE'10 Proceedings of the 2010 international conference on Life system modeling and simulation and intelligent computing, and 2010 international conference on Intelligent computing for sustainable energy and environment: Part II
From classifiers to discriminators: A nearest neighbor rule induced discriminant analysis

Pattern Recognition
Learning low-rank kernel matrices for constrained clustering

Neurocomputing
On the use of diagonal and class-dependent weighted distances for the probabilistic k-nearest neighbor

IbPRIA'11 Proceedings of the 5th Iberian conference on Pattern recognition and image analysis
Improving nearest neighbor rule with a simple adaptive distance measure

ICNC'06 Proceedings of the Second international conference on Advances in Natural Computation - Volume Part I
Adaptive nearest neighbor classifier based on supervised ellipsoid clustering

FSKD'06 Proceedings of the Third international conference on Fuzzy Systems and Knowledge Discovery
Identification of highly similar 3d objects using model saliency

ECCV'06 Proceedings of the 9th European conference on Computer Vision - Volume Part IV
An adaptive nearest neighbor classification algorithm for data streams

PKDD'05 Proceedings of the 9th European conference on Principles and Practice of Knowledge Discovery in Databases
An affinity-based new local distance function and similarity measure for kNN algorithm

Pattern Recognition Letters
A statistical confidence-based adaptive nearest neighbor algorithm for pattern classification

ICMLC'05 Proceedings of the 4th international conference on Advances in Machine Learning and Cybernetics
Cluster-based adaptive metric classification

Neurocomputing
Nearest neighbor classification using cam weighted distance

FSKD'05 Proceedings of the Second international conference on Fuzzy Systems and Knowledge Discovery - Volume Part II
Locally determining the number of neighbors in the k-nearest neighbor rule based on statistical confidence

ICNC'05 Proceedings of the First international conference on Advances in Natural Computation - Volume Part I
Noisy data elimination using mutual k-nearest neighbor for classification mining

Journal of Systems and Software
Phishing detection with popular search engines: simple and effective

FPS'11 Proceedings of the 4th Canada-France MITACS conference on Foundations and Practice of Security
Feature weighting by RELIEF based on local hyperplane approximation

PAKDD'12 Proceedings of the 16th Pacific-Asia conference on Advances in Knowledge Discovery and Data Mining - Volume Part II
Perceptual relativity-based local hyperplane classification

Neurocomputing
Measuring image distances via embedding in a semantic manifold

ECCV'12 Proceedings of the 12th European conference on Computer Vision - Volume Part IV
A Semi-Supervised Metric Learning for Content-Based Image Retrieval

International Journal of Computer Vision and Image Processing
Adaptive distance metrics for nearest neighbour classification based on genetic programming

EuroGP'13 Proceedings of the 16th European conference on Genetic Programming
Local discriminative distance metrics ensemble learning

Pattern Recognition
Random subspace evidence classifier

Neurocomputing
Cognitive gravitation model for classification on small noisy data

Neurocomputing
Linear reconstruction measure steered nearest neighbor classification framework

Pattern Recognition
A fast prototype reduction method based on template reduction and visualization-induced self-organizing map for nearest neighbor algorithm

Applied Intelligence

Quantified Score

Hi-index	0.15

Visualization

Abstract

Nearest-neighbor classification assumes locally constant class conditional probabilities. This assumption becomes invalid in high dimensions with finite samples due to the curse of dimensionality. Severe bias can be introduced under these conditions when using the nearest-neighbor rule. We propose a locally adaptive nearest-neighbor classification method to try to minimize bias. We use a Chi-squared distance analysis to compute a flexible metric for producing neighborhoods that are highly adaptive to query locations. Neighborhoods are elongated along less relevant feature dimensions and constricted along most influential ones. As a result, the class conditional probabilities are smoother in the modified neighborhoods, whereby better classification performance can be achieved. The efficacy of our method is validated and compared against other techniques using both simulated and real-world data.