Twenty-one ML estimators for model selection
Automatica (Journal of IFAC)
System identification (2nd ed.): theory for the user
System identification (2nd ed.): theory for the user
A new kernel-based approach for linear system identification
Automatica (Journal of IFAC)
Prediction error identification of linear systems: A nonparametric Gaussian regression approach
Automatica (Journal of IFAC)
Uniformly Improving the CramÉr-Rao Bound and Maximum-Likelihood Estimation
IEEE Transactions on Signal Processing
Paper: On the estimation of transfer functions
Automatica (Journal of IFAC)
Non-stationary stochastic embedding for transfer function estimation
Automatica (Journal of IFAC)
L2 Model reduction and variance reduction
Automatica (Journal of IFAC)
Consistent identification of Wiener systems: A machine learning viewpoint
Automatica (Journal of IFAC)
Regularized spectrum estimation using stable spline kernels
Automatica (Journal of IFAC)
Hi-index | 22.15 |
Intrigued by some recent results on impulse response estimation by kernel and nonparametric techniques, we revisit the old problem of transfer function estimation from input-output measurements. We formulate a classical regularization approach, focused on finite impulse response (FIR) models, and find that regularization is necessary to cope with the high variance problem. This basic, regularized least squares approach is then a focal point for interpreting other techniques, like Bayesian inference and Gaussian process regression. The main issue is how to determine a suitable regularization matrix (Bayesian prior or kernel). Several regularization matrices are provided and numerically evaluated on a data bank of test systems and data sets. Our findings based on the data bank are as follows. The classical regularization approach with carefully chosen regularization matrices shows slightly better accuracy and clearly better robustness in estimating the impulse response than the standard approach-the prediction error method/maximum likelihood (PEM/ML) approach. If the goal is to estimate a model of given order as well as possible, a low order model is often better estimated by the PEM/ML approach, and a higher order model is often better estimated by model reduction on a high order regularized FIR model estimated with careful regularization. Moreover, an optimal regularization matrix that minimizes the mean square error matrix is derived and studied. The importance of this result lies in that it gives the theoretical upper bound on the accuracy that can be achieved for this classical regularization approach.