Energy-based temporal neural networks for imputing missing values

Authors:
Philemon Brakel;Benjamin Schrauwen
Affiliations:
Department of Electronics and Information Systems, Ghent University, Gent, Belgium;Department of Electronics and Information Systems, Ghent University, Gent, Belgium
Venue:
ICONIP'12 Proceedings of the 19th international conference on Neural Information Processing - Volume Part II
Year:
2012

Citing 6
Cited 0

Training products of experts by minimizing contrastive divergence

Neural Computation
UNSUPERVISED LEARNING OF DISTRIBUTIONS ON BINARY VECTORS USING TWO LAYER NETWORKS

UNSUPERVISED LEARNING OF DISTRIBUTIONS ON BINARY VECTORS USING TWO LAYER NETWORKS
A fast learning algorithm for deep belief nets

Neural Computation
Extracting and composing robust features with denoising autoencoders

Proceedings of the 25th international conference on Machine learning
Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations

ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
Dynamic Factor Graphs for Time Series Modeling

ECML PKDD '09 Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases: Part II

Quantified Score

Hi-index	0.00

Visualization

Abstract

Imputing missing values in high dimensional time series is a difficult problem. There have been some approaches to the problem [11,8] where neural architectures were trained as probabilistic models of the data. However, we argue that this approach is not optimal. We propose to view temporal neural networks with latent variables as energy-based models and train them for missing value recovery directly. In this paper we introduce two energy-based models. The first model is based on a one dimensional convolution and the second model utilizes a recurrent neural network. We demonstrate how ideas from the energy-based learning framework can be used to train these models to recover missing values. The models are evaluated on a motion capture dataset.