Kernel Factory: An ensemble of kernel machines

Authors:
Michel Ballings;Dirk Van Den Poel
Affiliations:
Faculty of Economics and Business Administration, Department of Marketing, Ghent University, Tweekerkenstraat 2, B-9000 Ghent, Belgium;Faculty of Economics and Business Administration, Department of Marketing, Ghent University, Tweekerkenstraat 2, B-9000 Ghent, Belgium
Venue:
Expert Systems with Applications: An International Journal
Year:
2013

Citing 19
Cited 0

MultiBoosting: A Technique for Combining Boosting and Wagging

Machine Learning
Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond

Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond
Random Forests

Machine Learning
Ensembling neural networks: many could be better than all

Artificial Intelligence
The Case against Accuracy Estimation for Comparing Induction Algorithms

ICML '98 Proceedings of the Fifteenth International Conference on Machine Learning
Crafting Papers on Machine Learning

ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
An Analysis of the Interacting Roles of Population Size and Crossover in Genetic Algorithms

PPSN I Proceedings of the 1st Workshop on Parallel Problem Solving from Nature
Kernel Methods for Pattern Analysis

Kernel Methods for Pattern Analysis
Combined 5 × 2 cv F Test for Comparing Supervised Classification Learning Algorithms

Neural Computation
Churn prediction in subscription services: An application of support vector machines while comparing two parameter-selection techniques

Expert Systems with Applications: An International Journal
Statistical Comparisons of Classifiers over Multiple Data Sets

The Journal of Machine Learning Research
Random Forests for multiclass classification: Random MultiNomial Logit

Expert Systems with Applications: An International Journal
Predicting customer retention and profitability by using random forests and regression forests techniques

Expert Systems with Applications: An International Journal
Data augmentation by predicting spending pleasure using commercially available external data

Journal of Intelligent Information Systems
Improved prediction of biomass composition for switchgrass using reproducing kernel methods with wavelet compressed FT-NIR spectra

Expert Systems with Applications: An International Journal
Making use of population information in evolutionary artificialneural networks

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Recognizing plankton images from the shadow image particle profiling evaluation recorder

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Including spatial interdependence in customer acquisition models: A cross-category comparison

Expert Systems with Applications: An International Journal
Predicting e-commerce company success by mining the text of its publicly-accessible website

Expert Systems with Applications: An International Journal

Quantified Score

Hi-index	12.05

Visualization

Abstract

We propose an ensemble method for kernel machines. The training data is randomly split into a number of mutually exclusive partitions defined by a row and column parameter. Each partition forms an input space and is transformed by an automatically selected kernel function into a kernel matrix K. Subsequently, each K is used as training data for a base binary classifier (Random Forest). This results in a number of predictions equal to the number of partitions. A weighted average combines the predictions into one final prediction. To optimize the weights, a genetic algorithm is used. This approach has the advantage of simultaneously promoting (1) diversity, (2) accuracy, and (3) computational speed. (1) Diversity is fostered because the individual K's are based on a subset of features and observations, (2) accuracy is sought by automatic kernel selection and the genetic algorithm, and (3) computational speed is obtained because the computation of each K can be parallelized. Using five times twofold cross validation we benchmark the classification performance of Kernel Factory against Random Forest and Kernel-Induced Random Forest (KIRF). We find that Kernel Factory has significantly better performance than Kernel-Induced Random Forest. When the right kernel is selected Kernel Factory is also significantly better than Random Forest. In addition, an open-source R-software package of the algorithm (kernelFactory) is available from CRAN.