Weight-decay regularization in reproducing Kernel Hilbert spaces by variable-basis schemes

Authors:
Giorgio Gnecco;Marcello Sanguineti
Affiliations:
Department of Computer and Information Science, University of Genoa, Genova, Italy;Department of Communications, Computer and System Sciences, University of Genoa, Genova, Italy
Venue:
WSEAS Transactions on Mathematics
Year:
2009

Citing 19
Cited 0

Representations and rates of approximation of real-valued Boolean functions by neural networks

Neural Networks
Kernel Matching Pursuit

Machine Learning
Approximation of functions over redundant dictionaries using coherence

SODA '03 Proceedings of the fourteenth annual ACM-SIAM symposium on Discrete algorithms
Sparse Greedy Matrix Approximation for Machine Learning

ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Kernel Methods for Pattern Analysis

Kernel Methods for Pattern Analysis
Learning with generalization capability by kernal methods of bounded complexity

Journal of Complexity
On the Nyström Method for Approximating a Gram Matrix for Improved Kernel-Based Learning

The Journal of Machine Learning Research
Algorithms for subset selection in linear regression

STOC '08 Proceedings of the fortieth annual ACM symposium on Theory of computing
Elastic-net regularization in learning theory

Journal of Complexity
Identification of non linear MISO process using RKHS and Volterra models

WSEAS TRANSACTIONS on SYSTEMS
Multilayer perceptron and neural networks

WSEAS Transactions on Circuits and Systems
Fixed-Point Continuation for $\ell_1$-Minimization: Methodology and Convergence

SIAM Journal on Optimization
Regularization techniques and suboptimal solutions to optimization problems in learning from data

Neural Computation
The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network

IEEE Transactions on Information Theory
Bounds on rates of variable-basis and neural-network approximation

IEEE Transactions on Information Theory
Sequential greedy approximation for certain convex optimization problems

IEEE Transactions on Information Theory
Greed is good: algorithmic results for sparse approximation

IEEE Transactions on Information Theory
On the exponential convergence of matching pursuits in quasi-incoherent dictionaries

IEEE Transactions on Information Theory
The generalized LASSO

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.03

Visualization

Abstract

The optimization problems associated with various regularization techniques for supervised learning from data (e.g., weight-decay and Tikhonov regularization) are described in the context of Reproducing Kernel Hilbert Spaces. Suboptimal solutions expressed by sparse kernel models with a given upper bound on the number of kernel computational units are investigated. Improvements of some estimates obtained in Comput. Manag. Sci., vol. 6, pp. 53-79, 2009 are derived. Relationships between sparseness and generalization are discussed.