Learning a kernel matrix for nonlinear dimensionality reduction

Authors:
Kilian Q. Weinberger;Fei Sha;Lawrence K. Saul
Affiliations:
University of Pennsylvania, Philadelphia, PA;University of Pennsylvania, Philadelphia, PA;University of Pennsylvania, Philadelphia, PA
Venue:
ICML '04 Proceedings of the twenty-first international conference on Machine learning
Year:
2004

Citing 11
Cited 67

Semidefinite programming

SIAM Review
Nonlinear component analysis as a kernel eigenvalue problem

Neural Computation
Geometry and invariance in kernel based methods

Advances in kernel methods
Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond

Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond
A Database for Handwritten Text Recognition Research

IEEE Transactions on Pattern Analysis and Machine Intelligence
Kernel Matrix Completion by Semidefinite Programming

ICANN '02 Proceedings of the International Conference on Artificial Neural Networks
Laplacian Eigenmaps for dimensionality reduction and data representation

Neural Computation
Think globally, fit locally: unsupervised learning of low dimensional manifolds

The Journal of Machine Learning Research
Learning the Kernel Matrix with Semidefinite Programming

The Journal of Machine Learning Research
A kernel view of the dimensionality reduction of manifolds

ICML '04 Proceedings of the twenty-first international conference on Machine learning
Unsupervised learning of image manifolds by semidefinite programming

CVPR'04 Proceedings of the 2004 IEEE computer society conference on Computer vision and pattern recognition

Action respecting embedding

ICML '05 Proceedings of the 22nd international conference on Machine learning
Harmonic mixtures: combining mixture models and graph-based methods for inductive and scalable semi-supervised learning

ICML '05 Proceedings of the 22nd international conference on Machine learning
Unsupervised Learning of Image Manifolds by Semidefinite Programming

International Journal of Computer Vision
Learning low-rank kernel matrices

ICML '06 Proceedings of the 23rd international conference on Machine learning
Local distance preservation in the GP-LVM through back constraints

ICML '06 Proceedings of the 23rd international conference on Machine learning
On the Nyström Method for Approximating a Gram Matrix for Improved Kernel-Based Learning

The Journal of Machine Learning Research
Robust non-linear dimensionality reduction using successive 1-dimensional Laplacian Eigenmaps

Proceedings of the 24th international conference on Machine learning
Transductive regression piloted by inter-manifold relations

Proceedings of the 24th international conference on Machine learning
Rank minimization via online learning

Proceedings of the 25th international conference on Machine learning
Topologically-constrained latent variable models

Proceedings of the 25th international conference on Machine learning
Semi-definite Manifold Alignment

ECML '07 Proceedings of the 18th European conference on Machine Learning
Learning a Kernel Matrix for Time Series Data from DTW Distances

Neural Information Processing
Ambiguity Modeling in Latent Spaces

MLMI '08 Proceedings of the 5th international workshop on Machine Learning for Multimodal Interaction
Quality assessment of dimensionality reduction: Rank-based criteria

Neurocomputing
Nonlinear dimensionality reduction with relative distance comparison

Neurocomputing
Gaussian kernel optimization for pattern classification

Pattern Recognition
Local Dimensionality Reduction for Non-Parametric Regression

Neural Processing Letters
Nonlinear process monitoring based on maximum variance unfolding projections

Expert Systems with Applications: An International Journal
Learning linear dynamical systems without sequence information

ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
Geometry-aware metric learning

ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
Partial order embedding with multiple kernels

ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
Toward a perceptual space for gloss

ACM Transactions on Graphics (TOG)
Variant of Gaussian kernel and parameter setting method for nonlinear SVM

Neurocomputing
An introduction to nonlinear dimensionality reduction by maximum variance unfolding

AAAI'06 proceedings of the 21st national conference on Artificial intelligence - Volume 2
Variational Graph Embedding for Globally and Locally Consistent Feature Extraction

ECML PKDD '09 Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases: Part II
Transfer learning via dimensionality reduction

AAAI'08 Proceedings of the 23rd national conference on Artificial intelligence - Volume 2
Feature selection and kernel design via linear programming

IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Constrained many-objective optimization: a way forward

CEC'09 Proceedings of the Eleventh conference on Congress on Evolutionary Computation
Features and Metric from a Classifier Improve Visualizations with Dimension Reduction

ICANN '09 Proceedings of the 19th International Conference on Artificial Neural Networks: Part II
Kernel extrapolation

Neurocomputing
Dimensionality Estimation, Manifold Learning and Function Approximation using Tensor Voting

The Journal of Machine Learning Research
Non-linear dimensionality reduction procedures for certain large-dimensional multi-objective optimization problems: employing correntropy and a novel maximum variance unfolding

EMO'07 Proceedings of the 4th international conference on Evolutionary multi-criterion optimization
Embedding new data points for manifold learning via coordinate propagation

PAKDD'07 Proceedings of the 11th Pacific-Asia conference on Advances in knowledge discovery and data mining
Nonlinear dimensionality reduction with hybrid distance for trajectory representation of dynamic texture

Signal Processing
Approximately harmonic projection: Theoretical analysis and an algorithm

Pattern Recognition
Scale-independent quality criteria for dimensionality reduction

Pattern Recognition Letters
Exploiting tag and word correlations for improved webpage clustering

SMUC '10 Proceedings of the 2nd international workshop on Search and mining user-generated contents
Manifold learning for object tracking with multiple motion dynamics

ECCV'10 Proceedings of the 11th European conference on computer vision conference on Computer vision: Part III
Robust Positive semidefinite L-Isomap Ensemble

Pattern Recognition Letters
Regression on Fixed-Rank Positive Semidefinite Matrices: A Riemannian Approach

The Journal of Machine Learning Research
A new scheme to learn a kernel in regularization networks

Neurocomputing
Nonlinear dimensionality reduction using a temporal coherence principle

Information Sciences: an International Journal
A Family of Simple Non-Parametric Kernel Learning Algorithms

The Journal of Machine Learning Research
Supervised semi-definite embedding for email data cleaning and visualization

APWeb'05 Proceedings of the 7th Asia-Pacific web conference on Web Technologies Research and Development
Generalized foley-sammon transform with kernels

ISNN'05 Proceedings of the Second international conference on Advances in Neural Networks - Volume Part I
Nonrigid embeddings for dimensionality reduction

ECML'05 Proceedings of the 16th European conference on Machine Learning
Approximating a gram matrix for improved kernel-based learning

COLT'05 Proceedings of the 18th annual conference on Learning Theory
Metric Learning for Estimating Psychological Similarities

ACM Transactions on Intelligent Systems and Technology (TIST)
Dimensionality reduction by Mixed Kernel Canonical Correlation Analysis

Pattern Recognition
Distance metric learning with eigenvalue optimization

The Journal of Machine Learning Research
Leveraging Social Bookmarks from Partially Tagged Corpus for Improved Web Page Clustering

ACM Transactions on Intelligent Systems and Technology (TIST)
A unifying probabilistic perspective for spectral dimensionality reduction: insights and new models

The Journal of Machine Learning Research
Orthogonal projection analysis

IScIDE'11 Proceedings of the Second Sino-foreign-interchange conference on Intelligent Science and Intelligent Data Engineering
Modeling and monitoring of multimodes process

ISNN'12 Proceedings of the 9th international conference on Advances in Neural Networks - Volume Part I
Self-taught dimensionality reduction on the high-dimensional small-sized data

Pattern Recognition
From sBoW to dCoT marginalized encoders for text representation

Proceedings of the 21st ACM international conference on Information and knowledge management
A kernel-based framework for image collection exploration

Journal of Visual Languages and Computing
Image Dimensionality Reduction Based on the Intrinsic Dimension and Parallel Genetic Algorithm

International Journal of Cognitive Informatics and Natural Intelligence
Multiple kernel local Fisher discriminant analysis for face recognition

Signal Processing
Editor's choice article: Canonical locality preserving Latent Variable Model for discriminative pose inference

Image and Vision Computing
Inductive manifold learning using structured support vector machine

Pattern Recognition
Learning social network embeddings for predicting information diffusion

Proceedings of the 7th ACM international conference on Web search and data mining
Isometric sliced inverse regression for nonlinear manifold learning

Statistics and Computing
BILGO: Bilateral greedy optimization for large scale semidefinite programming

Neurocomputing
On the convergence of maximum variance unfolding

The Journal of Machine Learning Research
Parallel vector field embedding

The Journal of Machine Learning Research
Supervised Distance Preserving Projections

Neural Processing Letters

Quantified Score

Hi-index	0.00

Visualization

Abstract

We investigate how to learn a kernel matrix for high dimensional data that lies on or near a low dimensional manifold. Noting that the kernel matrix implicitly maps the data into a nonlinear feature space, we show how to discover a mapping that "unfolds" the underlying manifold from which the data was sampled. The kernel matrix is constructed by maximizing the variance in feature space subject to local constraints that preserve the angles and distances between nearest neighbors. The main optimization involves an instance of semidefinite programming---a fundamentally different computation than previous algorithms for manifold learning, such as Isomap and locally linear embedding. The optimized kernels perform better than polynomial and Gaussian kernels for problems in manifold learning, but worse for problems in large margin classification. We explain these results in terms of the geometric properties of different kernels and comment on various interpretations of other manifold learning algorithms as kernel methods.