Learning a kernel function for classification with small training samples

Authors:
Tomer Hertz;Aharon Bar Hillel;Daphna Weinshall
Affiliations:
The Hebrew University of Jerusalem, Jerusalem, Israel;The Hebrew University of Jerusalem, Jerusalem, Israel;The Hebrew University of Jerusalem, Jerusalem, Israel
Venue:
ICML '06 Proceedings of the 23rd international conference on Machine learning
Year:
2006

Citing 12
Cited 19

A Bayesian/Information Theoretic Model of Learning to Learn viaMultiple Task Sampling

Machine Learning - Special issue on inductive transfer
Learning to learn

Learning to learn
Improved Boosting Algorithms Using Confidence-rated Predictions

Machine Learning - The Eleventh Annual Conference on computational Learning Theory
Learning the Kernel Matrix with Semi-Definite Programming

ICML '02 Proceedings of the Nineteenth International Conference on Machine Learning
From Few to Many: Generative Models for Recognition Under Variable Pose and Illumination

FG '00 Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition 2000
Recognizing Novel 3-D Objects under New Illumination and Viewing Position Using a Small Number of Example Views or Even a Single View

ICCV '98 Proceedings of the Sixth International Conference on Computer Vision
Boosting margin based distance functions for clustering

ICML '04 Proceedings of the twenty-first international conference on Machine learning
Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories

CVPRW '04 Proceedings of the 2004 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'04) Volume 12 - Volume 12
Building a Classification Cascade for Visual Identification from One Example

ICCV '05 Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 - Volume 01
Learning Gaussian processes from multiple tasks

ICML '05 Proceedings of the 22nd international conference on Machine learning
Model-based transductive learning of the kernel matrix

Machine Learning
Solving multiclass learning problems via error-correcting output codes

Journal of Artificial Intelligence Research

BoostCluster: boosting clustering by pairwise constraints

Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining
A kernel optimization method based on the localized kernel Fisher criterion

Pattern Recognition
Pairwise Rayleigh quotient classifier with application to the analysis of breast tumors

SPPR'07 Proceedings of the Fourth conference on IASTED International Conference: Signal Processing, Pattern Recognition, and Applications
Optimizing the data-dependent kernel under a unified kernel optimization framework

Pattern Recognition
Automatic kernel clustering with a Multi-Elitist Particle Swarm Optimization Algorithm

Pattern Recognition Letters
Adaptive p-posterior mixture-model kernels for multiple instance learning

Proceedings of the 25th international conference on Machine learning
Learning Similarity Measures from Pairwise Constraints with Neural Networks

ICANN '08 Proceedings of the 18th international conference on Artificial Neural Networks, Part II
Kernel-Based Inductive Transfer

ECML PKDD '08 Proceedings of the European conference on Machine Learning and Knowledge Discovery in Databases - Part II
Semi-supervised Edge Learning for Building Detection in Aerial Images

ISVC '08 Proceedings of the 4th International Symposium on Advances in Visual Computing, Part II
An adaptive middleware for supporting time-critical event response

Cluster Computing
Semi-supervised fuzzy clustering: A kernel-based approach

Knowledge-Based Systems
Semi-supervised clustering using similarity neural networks

IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
Pairwise Rayleigh quotient classifier with application to the analysis of breast tumors

SPPRA '07 Proceedings of the Fourth IASTED International Conference on Signal Processing, Pattern Recognition, and Applications
Kernel-induced fuzzy clustering of image pixels with an improved differential evolution algorithm

Information Sciences: an International Journal
Image classification from small sample, with distance learning and feature selection

ISVC'07 Proceedings of the 3rd international conference on Advances in visual computing - Volume Part II
Adapting visual category models to new domains

ECCV'10 Proceedings of the 11th European conference on Computer vision: Part IV
A multi-scale learning framework for visual categorization

ACCV'10 Proceedings of the 10th Asian conference on Computer vision - Volume Part I
Aggregated distance metric learning (ADM) for image classification in presence of limited training data

MICCAI'11 Proceedings of the 14th international conference on Medical image computing and computer-assisted intervention - Volume Part III
A boosting approach for supervised Mahalanobis distance metric learning

Pattern Recognition

Quantified Score

Hi-index	0.00

Visualization

Abstract

When given a small sample, we show that classification with SVM can be considerably enhanced by using a kernel function learned from the training data prior to discrimination. This kernel is also shown to enhance retrieval based on data similarity. Specifically, we describe KernelBoost - a boosting algorithm which computes a kernel function as a combination of 'weak' space partitions. The kernel learning method naturally incorporates domain knowledge in the form of unlabeled data (i.e. in a semi-supervised or transductive settings), and also in the form of labeled samples from relevant related problems (i.e. in a learning-to-learn scenario). The latter goal is accomplished by learning a single kernel function for all classes. We show comparative evaluations of our method on datasets from the UCI repository. We demonstrate performance enhancement on two challenging tasks: digit classification with kernel SVM, and facial image retrieval based on image similarity as measured by the learnt kernel.