Structural Modelling with Sparse Kernels
Machine Learning
| Hi-index | 0.00 |
Smoothing regularizers for radial basis functions have been studied extensively, but no general smoothing regularizers for PROJECTIVE BASIS FUNCTIONS (PBFs), such as the widely-used sigmoidal PBFs, have heretofore been proposed. We derive new classes of algebraically-simple m:th-order smoothing regularizers for networks of projective basis functions. Our simple algebraic forms enable the direct enforcement of smoothness without the need for e.g. costly Monte Carlo integrations of the smoothness functional. We show that our regularizers are highly correlated with the values of standard smoothness functionals, and thus suitable for enforcing smoothness constraints onto PBF networks. The regularizers are tested on illustrative sample problems and compared to quadratic weight decay. The new regularizers are shown to yield better generalization errors than weight decay when the implicit assumptions in the latter are wrong. Unlike weight decay, the new regularizers distinguish between the roles of the input and output weights and capture the interactions between them. (Short version of report, long version is CSE 96-006)