Network Optimization through Learning and Pruning in Neuromanifold

Authors:
Hyunjin Lee;Hyeyoung Park;Yillbyung Lee
Affiliations:
-;-;-
Venue:
PRICAI '02 Proceedings of the 7th Pacific Rim International Conference on Artificial Intelligence: Trends in Artificial Intelligence
Year:
2002

Citing 9
Cited 0

Natural gradient works efficiently in learning

Neural Computation
Neural classifier construction using regularization, pruning and test error estimation

Neural Networks
Pruning using parameter and neuronal metrics

Neural Computation
Local minima and plateaus in hierarchical structures of multilayer perceptions

Neural Networks
Adaptive natural gradient learning algorithms for various stochastic models

Neural Networks
Neural Networks for Pattern Recognition

Neural Networks for Pattern Recognition
Algebraic Analysis for Nonidentifiable Learning Machines

Neural Computation
Adaptive Method of Realizing Natural Gradient Learning for Multilayer Perceptrons

Neural Computation
On "Natural" Learning and Pruning in Multilayered Perceptrons

Neural Computation

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper, we propose an optimization method of neural networks based on the geometrical structure of neuromanifold. The optimizing process starts from the manifold of sufficiently large network model. In the manifold of the given network structure, we first find an optimal point, which achieves good generalization performance. To do this, we propose an extension of the adaptive natural gradient learning with regularization term. Using hierarchical structure of neuromanifold, we then try to optimize the network structure. To do this, we apply the natural pruning method starting from the current optimal parameter point. The whole optimization process can be explained from the geometrical point of view. We confirm the generalization performance of the optimized network by the proposed method through experiments on benchmark data sets.