Kernel Matching Pursuit

Authors:
Pascal Vincent;Yoshua Bengio
Affiliations:
Department of IRO, Université de Montréal, C.P. 6128, Montreal, Qc, H3C 3J7, Canada. Vincentp@iro.umontreal.ca (http://www.iro.umontreal.ca/~vincent);Department of IRO, Université de Montréal, C.P. 6128, Montreal, Qc, H3C 3J7, Canada. bengioy@iro.umontreal.ca
Venue:
Machine Learning
Year:
2002

Citing 12
Cited 57

A training algorithm for optimal margin classifiers

COLT '92 Proceedings of the fifth annual workshop on Computational learning theory
The nature of statistical learning theory

The nature of statistical learning theory
Sample Compression, Learnability, and the Vapnik-Chervonenkis Dimension

Machine Learning
Large margin classification using the perceptron algorithm

COLT' 98 Proceedings of the eleventh annual conference on Computational learning theory
A sparse representation for function approximation

Neural Computation
Support vector density estimation

Advances in kernel methods
Semiparametric support vector and linear programming machines

Proceedings of the 1998 conference on Advances in neural information processing systems II
Structural Modelling with Sparse Kernels

Machine Learning
Sparse Greedy Matrix Approximation for Machine Learning

ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Effiicient BackProp

Neural Networks: Tricks of the Trade, this book is an outgrowth of a 1996 NIPS workshop
Generalisation Error Bounds for Sparse Linear Classifiers

COLT '00 Proceedings of the Thirteenth Annual Conference on Computational Learning Theory
Comparing support vector machines with Gaussian kernels to radialbasis function classifiers

IEEE Transactions on Signal Processing

Some greedy learning algorithms for sparse regression and classification with mercer kernels

The Journal of Machine Learning Research
Ranking a random feature for variable and feature selection

The Journal of Machine Learning Research
Sparse Representation for Coarse and Fine Object Recognition

IEEE Transactions on Pattern Analysis and Machine Intelligence
Constructing a speculative kernel machine for pattern classification

Neural Networks
Kernel matching pursuit classifier ensemble

Pattern Recognition
Building Support Vector Machines with Reduced Classifier Complexity

The Journal of Machine Learning Research
A sequential algorithm for feed-forward neural networks with optimal coefficients and interacting frequencies

Neurocomputing
Functional dissipation microarrays for classification

Pattern Recognition
Heuristics for the selection of weights in sequential feed-forward neural networks: An experimental study

Neurocomputing
Nonlinear estimation of subpixel proportion via kernel least square regression

International Journal of Remote Sensing
A Kernel Matching Pursuit Approach to Man-Made Objects Detection in Aerial Images

IbPRIA '07 Proceedings of the 3rd Iberian conference on Pattern Recognition and Image Analysis, Part II
Sparse Least Squares Support Vector Machines by Forward Selection Based on Linear Discriminant Analysis

ANNPR '08 Proceedings of the 3rd IAPR workshop on Artificial Neural Networks in Pattern Recognition
Example-Based Learning for Single-Image Super-Resolution

Proceedings of the 30th DAGM symposium on Pattern Recognition
Online Manifold Regularization: A New Learning Setting and Empirical Study

ECML PKDD '08 Proceedings of the 2008 European Conference on Machine Learning and Knowledge Discovery in Databases - Part I
A fast linear-in-the-parameters classifier construction algorithm using orthogonal forward selection to minimize leave-one-out misclassification rate

International Journal of Systems Science
Orthogonal-least-squares regression: A unified approach for data modelling

Neurocomputing
An experimental study on methods for the selection of basis functions in regression

Neurocomputing
Sparse kernel SVMs via cutting-plane training

Machine Learning
Kernel Polytope Faces Pursuit

ECML PKDD '09 Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases: Part I
Construction of tunable radial basis function networks using orthogonal forward selection

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Kernel-matching pursuits with arbitrary loss functions

IEEE Transactions on Neural Networks
Selecting features of linear-chain conditional random fields via greedy stage-wise algorithms

Pattern Recognition Letters
Using Kernel Basis with Relevance Vector Machine for Feature Selection

ICANN '09 Proceedings of the 19th International Conference on Artificial Neural Networks: Part II
Twin Gaussian Processes for Structured Prediction

International Journal of Computer Vision
Weight-decay regularization in reproducing Kernel Hilbert spaces by variable-basis schemes

WSEAS Transactions on Mathematics
Stagewise weak gradient pursuits

IEEE Transactions on Signal Processing
Regularization techniques and suboptimal solutions to optimization problems in learning from data

Neural Computation
Gaussian kernel based fuzzy rough sets: Model, uncertainty measures and applications

International Journal of Approximate Reasoning
Similarity-based classification of sequences using hidden Markov models

Pattern Recognition
Kernel matching pursuit for large datasets

Pattern Recognition
On the sparseness of 1-norm support vector machines

Neural Networks
MRA Kernel matching pursuit machine

PRICAI'06 Proceedings of the 9th Pacific Rim international conference on Artificial intelligence
Selecting a reduced set for building sparse support vector regression in the primal

PAKDD'07 Proceedings of the 11th Pacific-Asia conference on Advances in knowledge discovery and data mining
Sparse approximation and the pursuit of meaningful signal models with interference adaptation

IEEE Transactions on Audio, Speech, and Language Processing
Selection of basis functions guided by the L2 soft margin

ICANN'07 Proceedings of the 17th international conference on Artificial neural networks
Kernel matching reduction algorithms for classification

RSKT'08 Proceedings of the 3rd international conference on Rough sets and knowledge technology
Active learning and basis selection for kernel-based linear models: a Bayesian perspective

IEEE Transactions on Signal Processing
Sparse approximation through boosting for learning large scale kernel machines

IEEE Transactions on Neural Networks
Particle swarm optimization aided orthogonal forward regression for unified data modeling

IEEE Transactions on Evolutionary Computation
Visual tracking via online manifold regularization

ICIMCS '10 Proceedings of the Second International Conference on Internet Multimedia Computing and Service
Constructing sparse KFDA using pre-image reconstruction

ICONIP'10 Proceedings of the 17th international conference on Neural information processing: models and applications - Volume Part II
Greedy optimization classifiers ensemble based on diversity

Pattern Recognition
A high-order feature synthesis and selection algorithm applied to insurance risk modelling

International Journal of Business Intelligence and Data Mining
Comparing error minimized extreme learning machines and support vector sequential feed-forward neural networks

Neural Networks
A new dictionary learning method for kernel matching pursuit

FSKD'06 Proceedings of the Third international conference on Fuzzy Systems and Knowledge Discovery
Kernel matching pursuit based on immune clonal algorithm for image recognition

SEAL'06 Proceedings of the 6th international conference on Simulated Evolution And Learning
Kernel discriminant analysis for regression problems

Pattern Recognition
Base vector selection for kernel matching pursuit

ADMA'06 Proceedings of the Second international conference on Advanced Data Mining and Applications
Kernel basis pursuit

ECML'05 Proceedings of the 16th European conference on Machine Learning
Orthogonal forward selection for constructing the radial basis function network with tunable nodes

ICIC'05 Proceedings of the 2005 international conference on Advances in Intelligent Computing - Volume Part I
Refining kernel matching pursuit

ISNN'10 Proceedings of the 7th international conference on Advances in Neural Networks - Volume Part II
Selection of weights for sequential feed-forward neural networks: an experimental study

IWANN'05 Proceedings of the 8th international conference on Artificial Neural Networks: computational Intelligence and Bioinspired Systems
Wavelet kernel matching pursuit machine

AI'06 Proceedings of the 19th Australian joint conference on Artificial Intelligence: advances in Artificial Intelligence
Comparing support vector machines and feed-forward neural networks with similar parameters

IDEAL'06 Proceedings of the 7th international conference on Intelligent Data Engineering and Automated Learning
Parameterized attribute reduction with Gaussian kernel based fuzzy rough sets

Information Sciences: an International Journal
Learning non-linear classifiers with a sparsity constraint using L1 regularization

Proceedings of the 28th Annual ACM Symposium on Applied Computing
Fast sparse approximation of extreme learning machine

Neurocomputing

Quantified Score

Hi-index	0.01

Visualization

Abstract

Matching Pursuit algorithms learn a function that is a weighted sum of basis functions, by sequentially appending functions to an initially empty basis, to approximate a target function in the least-squares sense. We show how matching pursuit can be extended to use non-squared error loss functions, and how it can be used to build kernel-based solutions to machine learning problems, while keeping control of the sparsity of the solution. We present a version of the algorithm that makes an optimal choice of both the next basis and the weights of all the previously chosen bases. Finally, links to boosting algorithms and RBF training procedures, as well as an extensive experimental comparison with SVMs for classification are given, showing comparable results with typically much sparser models.