Adaptive and competitive committee machine architecture

Authors:
Jian Yang;Siwei Luo
Affiliations:
School of Computer and Information Technology, Beijing Jiaotong University, Beijing, China;School of Computer and Information Technology, Beijing Jiaotong University, Beijing, China
Venue:
ICNC'05 Proceedings of the First international conference on Advances in Natural Computation - Volume Part I
Year:
2005

Citing 8
Cited 0

Hierarchical mixtures of experts and the EM algorithm

Neural Computation
Optimal linear combinations of neural networks

Optimal linear combinations of neural networks
Information geometry of the EM and em algorithms for neural networks

Neural Networks
A Bayesian approach to model selection in hierarchical mixtures-of-experts architectures

Neural Networks
From data mining to knowledge discovery: an overview

Advances in knowledge discovery and data mining
Neural Network Ensembles

IEEE Transactions on Pattern Analysis and Machine Intelligence
Improving the accuracy of an artificial neural network using multiple differently trained networks

Neural Computation
Adaptive mixtures of local experts

Neural Computation

Quantified Score

Hi-index	0.00

Visualization

Abstract

Learning problem has three distinct phases, that is, model representation, learning criterion (target function) and implementation algorithm. This paper focuses on the close relation between the selection of learning criterion for committee machine and network approximation and competitive adaptation. By minimizing the KL deviation between posterior distributions, we give a general posterior modular architecture and the corresponding learning criterion form, which reflects remarkable adaptation and scalability. Besides this, we point out, from the generalized KL deviation defined on finite measure manifold in information geometry theory, that the proposed learning criterion reduces to so-called Mahalanobis deviation of which ordinary mean square error approximation is a special case, when each module is assumed Gaussian.