Machine Learning
Computational Statistics & Data Analysis - Nonlinear methods and data mining
An empirical comparison of supervised learning algorithms
ICML '06 Proceedings of the 23rd international conference on Machine learning
Generalized Additive Models (Texts in Statistical Science)
Generalized Additive Models (Texts in Statistical Science)
A comparison of methods for the fitting of generalized additive models
Statistics and Computing
Additive Groves of Regression Trees
ECML '07 Proceedings of the 18th European conference on Machine Learning
Feature shaping for linear SVM classifiers
Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining
PLANET: massively parallel learning of tree ensembles with MapReduce
Proceedings of the VLDB Endowment
Parallel boosted regression trees for web search ranking
Proceedings of the 20th international conference on World wide web
Accurate intelligible models with pairwise interactions
Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining
| Hi-index | 0.00 |
Complex models for regression and classification have high accuracy, but are unfortunately no longer interpretable by users. We study the performance of generalized additive models (GAMs), which combine single-feature models called shape functions through a linear function. Since the shape functions can be arbitrarily complex, GAMs are more accurate than simple linear models. But since they do not contain any interactions between features, they can be easily interpreted by users. We present the first large-scale empirical comparison of existing methods for learning GAMs. Our study includes existing spline and tree-based methods for shape functions and penalized least squares, gradient boosting, and backfitting for learning GAMs. We also present a new method based on tree ensembles with an adaptive number of leaves that consistently outperforms previous work. We complement our experimental results with a bias-variance analysis that explains how different shape models influence the additive model. Our experiments show that shallow bagged trees with gradient boosting distinguish itself as the best method on low- to medium-dimensional datasets.