The cascade-correlation learning architecture
Advances in neural information processing systems 2
Bumptrees for efficient function, constraint, and classification learning
NIPS-3 Proceedings of the 1990 conference on Advances in neural information processing systems 3
Machine Learning
Local dimensionality reduction
NIPS '97 Proceedings of the 1997 conference on Advances in neural information processing systems 10
Mixtures of probabilistic principal component analyzers
Neural Computation
Comparison of approximate methods for handling hyperparameters
Neural Computation
An Algorithm for Finding Best Matches in Logarithmic Expected Time
ACM Transactions on Mathematical Software (TOMS)
Machine Learning
Pattern Recognition and Neural Networks
Pattern Recognition and Neural Networks
Bayesian parameter estimation via variational methods
Statistics and Computing
A Dynamic Adaptation of AD-trees for Efficient Machine Learning on Large Data Sets
ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Locally Weighted Projection Regression: Incremental Real Time Learning in High Dimensional Space
ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Variational Relevance Vector Machines
UAI '00 Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence
The Anchors Hierarchy: Using the Triangle Inequality to Survive High Dimensional Data
UAI '00 Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence
Bayesian learning for neural networks
Bayesian learning for neural networks
Sparse bayesian learning and the relevance vector machine
The Journal of Machine Learning Research
Feature selection, L1 vs. L2 regularization, and rotational invariance
ICML '04 Proceedings of the twenty-first international conference on Machine learning
Sparse Multinomial Logistic Regression: Fast Algorithms and Generalization Bounds
IEEE Transactions on Pattern Analysis and Machine Intelligence
Bayesian regression with input noise for high dimensional data
ICML '06 Proceedings of the 23rd international conference on Machine learning
Building Support Vector Machines with Reduced Classifier Complexity
The Journal of Machine Learning Research
EfficientL1regularized logistic regression
AAAI'06 Proceedings of the 21st national conference on Artificial intelligence - Volume 1
Cached sufficient statistics for efficient machine learning with large datasets
Journal of Artificial Intelligence Research
Assessing similarity of feature selection techniques in high-dimensional domains
Pattern Recognition Letters
| Hi-index | 0.00 |
We present a novel algorithm for efficient learning and feature selection in high-dimensional regression problems. We arrive at this model through a modification of the standard regression model, enabling us to derive a probabilistic version of the well-known statistical regression technique of backfitting. Using the expectation-maximization algorithm, along with variational approximation methods to overcome intractability, we extend our algorithm to include automatic relevance detection of the input features. This variational Bayesian least squares (VBLS) approach retains its simplicity as a linear model, but offers a novel statistically robust black-box approach to generalized linear regression with high-dimensional inputs. It can be easily extended to nonlinear regression and classification problems. In particular, we derive the framework of sparse Bayesian learning, the relevance vector machine, with VBLS at its core, offering significant computational and robustness advantages for this class of methods. The iterative nature of VBLS makes it most suitable for real-time incremental learning, which is crucial especially in the application domain of robotics, brain-machine interfaces, and neural prosthetics, where real-time learning of models for control is needed. We evaluate our algorithm on synthetic and neurophysiological data sets, as well as on standard regression and classification benchmark data sets, comparing it with other competitive statistical approaches and demonstrating its suitability as a drop-in replacement for other generalized linear regression techniques.