An Information Theoretic Approach of Designing Sparse Kernel Adaptive Filters

Authors:
Weifeng Liu;Il Park;J. C. Principe
Affiliations:
Forecasting Team, Amazon.com, Seattle, WA, USA;-;-
Venue:
IEEE Transactions on Neural Networks
Year:
2009

Citing 0
Cited 6

Kernel Wiener filter and its application to pattern recognition

IEEE Transactions on Neural Networks
A sparse kernel algorithm for online time series data prediction

Expert Systems with Applications: An International Journal
Fixed budget quantized kernel least-mean-square algorithm

Signal Processing
Kernel minimum error entropy algorithm

Neurocomputing
A linear recurrent kernel online learning algorithm with sparse updates

Neural Networks
An information theoretic sparse kernel algorithm for online learning

Expert Systems with Applications: An International Journal

Quantified Score

Hi-index	0.01

Visualization

Abstract

This paper discusses an information theoretic approach of designing sparse kernel adaptive filters. To determine useful data to be learned and remove redundant ones, a subjective information measure called surprise is introduced. Surprise captures the amount of information a datum contains which is transferable to a learning system. Based on this concept, we propose a systematic sparsification scheme, which can drastically reduce the time and space complexity without harming the performance of kernel adaptive filters. Nonlinear regression, short term chaotic time-series prediction, and long term time-series forecasting examples are presented.