Bayesian regression with input noise for high dimensional data

Authors:
Jo-Anne Ting;Aaron D'Souza;Stefan Schaal
Affiliations:
University of Southern California, Los Angeles, CA;Google, Inc., Mountain View, CA;University of Southern California, Los Angeles, CA and ATR Computational Neuroscience Laboratories, Kyoto, Japan
Venue:
ICML '06 Proceedings of the 23rd international conference on Machine learning
Year:
2006

Citing 4
Cited 5

Model-based control of a robot manipulator

Model-based control of a robot manipulator
Locally Weighted Learning

Artificial Intelligence Review - Special issue on lazy learning
Bayesian learning for neural networks

Bayesian learning for neural networks
The Bayesian backfitting relevance vector machine

ICML '04 Proceedings of the twenty-first international conference on Machine learning

Learning an Outlier-Robust Kalman Filter

ECML '07 Proceedings of the 18th European conference on Machine Learning
Privacy-Preserving Data Publishing

Foundations and Trends in Databases
Dimensionality Estimation, Manifold Learning and Function Approximation using Tensor Voting

The Journal of Machine Learning Research
Efficient learning and feature selection in high-dimensional regression

Neural Computation
An approximate inference with Gaussian process to latent functions from uncertain data

Neurocomputing

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper examines high dimensional regression with noise-contaminated input and output data. Goals of such learning problems include optimal prediction with noiseless query points and optimal system identification. As a first step, we focus on linear regression methods, since these can be easily cast into nonlinear learning problems with locally weighted learning approaches. Standard linear regression algorithms generate biased regression estimates if input noise is present and suffer numerically when the data contains redundancy and irrelevancy. Inspired by Factor Analysis Regression, we develop a variational Bayesian algorithm that is robust to ill-conditioned data, automatically detects relevant features, and identifies input and output noise -- all in a computationally efficient way. We demonstrate the effectiveness of our techniques on synthetic data and on a system identification task for a rigid body dynamics model of a robotic vision head. Our algorithm performs 10 to 70% better than previously suggested methods.