A regularized minimum cross-entropy algorithm on mixtures of experts for time series prediction

Authors:
Zhiwu Lu
Affiliations:
Institute of Computer Science & Technology of Peking University, Beijing, China
Venue:
ISNN'06 Proceedings of the Third international conference on Advnaces in Neural Networks - Volume Part II
Year:
2006

Citing 5
Cited 0

Convergence results for the EM approach to mixtures of experts architectures

Neural Networks
BYY harmony learning, structural RPCL, and topological self-organizing on mixture models

Neural Networks - New developments in self-organizing maps
Fast learning in networks of locally-tuned processing units

Neural Computation
A gradient BYY harmony learning algorithm on mixture of experts for curve detection

IDEAL'05 Proceedings of the 6th international conference on Intelligent Data Engineering and Automated Learning
An overview of statistical learning theory

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

The well-known mixtures of experts(ME) model is usually trained by expectation maximization(EM) algorithm for maximum likelihood learning. However, we have to first determine the number of experts, which is often hardly known. Derived from regularization theory, a regularized minimum cross-entropy(RMCE) algorithm is proposed to train ME model, which can automatically make model selection. When time series is modeled by ME, it is demonstrated by some climate prediction experiments that RMCE algorithm outperforms EM algorithm. We also compare RMCE algorithm with other regression methods such as back-propagation(BP) algorithm and normalized radial basis function(NRBF) network, and find that RMCE algorithm still shows promising results.