Mutual Learning with Many Linear Perceptrons: On-Line Learning Theory

Authors:
Kazuyuki Hara;Yoichi Nakayama;Seiji Miyoshi;Masato Okada
Affiliations:
Tokyo Metropolitan College of Industrial Technology, Tokyo, Japan 140-0011;Tokyo Metropolitan College of Technology, Tokyo, Japan 140-0011;Faculty of Engineering Science, Kansai University, Osaka, Japan 564-8680;Graduate School of Frontier Sciences, The University of Tokyo, Chiba, Japan 277-8561 and Brain Science Institute, Riken, Saitama, Japan 351-0198
Venue:
ICANN '09 Proceedings of the 19th International Conference on Artificial Neural Networks: Part I
Year:
2009

Citing 2
Cited 0

Bagging predictors

Machine Learning
A decision-theoretic generalization of on-line learning and an application to boosting

Journal of Computer and System Sciences - Special issue: 26th annual ACM symposium on the theory of computing & STOC'94, May 23–25, 1994, and second annual Europe an conference on computational learning theory (EuroCOLT'95), March 13–15, 1995

Quantified Score

Hi-index	0.00

Visualization

Abstract

We propose a new mutual learning using many weak learner (or student) which converges into the identical state of Bagging that is kind of ensemble learning, within the framework of on-line learning, and have analyzed its asymptotic property through the statistical mechanics method. Mutual learning involving more than three students is essential compares to two student case from a viewpoint of variety of selection of a student acting as teacher. The proposed model consists of two learning steps: many students independently learn from a teacher, and then the students learn from others through the mutual learning. In mutual learning, students learn from other students and the generalization error is improved even if the teacher has not taken part in the mutual learning. We demonstrate that the learning style of selecting a student to act as teacher randomly is superior to that of cyclic order by using principle component analysis.