Structuring Neural Networks through Bidirectional Clustering of Weights

Authors:
Kazumi Saito;Ryohei Nakano
Affiliations:
-;-
Venue:
DS '02 Proceedings of the 5th International Conference on Discovery Science
Year:
2002

Citing 8
Cited 4

Optimal brain damage

Advances in neural information processing systems 2
Simplifying neural networks by soft weight-sharing

Neural Computation
Extracting Refined Rules from Knowledge-Based Neural Networks

Machine Learning
Structural learning and rule discovery

Knowledge-based neurocomputing
Neural Networks: A Comprehensive Foundation

Neural Networks: A Comprehensive Foundation
Neural Networks for Pattern Recognition

Neural Networks for Pattern Recognition
Discovering Polynomials to Fit Multivariate Data Having Numeric and Nominal Variables

Progress in Discovery Science, Final Report of the Japanese Discovery Science Project
SMEM Algorithm for Mixture Models

Neural Computation

Mutual information neuro-evolutionary system (MINES)

CEC'09 Proceedings of the Eleventh conference on Congress on Evolutionary Computation
Finding nominally conditioned multivariate polynomials using a four-layer perceptron having shared weights

KES'06 Proceedings of the 10th international conference on Knowledge-Based Intelligent Information and Engineering Systems - Volume Part II
Model selection and weight sharing of multi-layer perceptrons

KES'05 Proceedings of the 9th international conference on Knowledge-Based Intelligent Information and Engineering Systems - Volume Part IV
An information-based approach towards neuro-evolution

Proceedings of the 14th annual conference companion on Genetic and evolutionary computation

Quantified Score

Hi-index	0.00

Visualization

Abstract

We present a method for succinctly structuring neural networks having a few thousands weights. Here structuring means weight sharing where weights in a network are divided into clusters and weights within the same cluster are constrained to have the same value. Our method employs a newly developed weight sharing technique called bidirectional clustering of weights (BCW), together with second-order optimal criteria for both cluster merge and split. Our experiments using two artificial data sets showed that the BCW method works well to find a succinct network structure from an original network having about two thousands weights in both regression and classification problems.