Time Series Analysis, Forecasting and Control
Time Series Analysis, Forecasting and Control
A neural network ensemble method with jittered training data for time series forecasting
Information Sciences: an International Journal
Time series forecasting with a non-linear model and the scatter search meta-heuristic
Information Sciences: an International Journal
Improving artificial neural networks' performance in seasonal time series forecasting
Information Sciences: an International Journal
Information Sciences: an International Journal
A locally linear RBF network-based state-dependent AR model for nonlinear time series modeling
Information Sciences: an International Journal
Information Sciences: an International Journal
Equivalences between neural-autoregressive time series models and fuzzy systems
IEEE Transactions on Neural Networks
Time-series forecasting using a system of ordinary differential equations
Information Sciences: an International Journal
Modern Applied Statistics with S
Modern Applied Statistics with S
Hi-index | 0.07 |
In time series predictor evaluation, we observe that with respect to the model selection procedure there is a gap between evaluation of traditional forecasting procedures, on the one hand, and evaluation of machine learning techniques on the other hand. In traditional forecasting, it is common practice to reserve a part from the end of each time series for testing, and to use the rest of the series for training. Thus it is not made full use of the data, but theoretical problems with respect to temporal evolutionary effects and dependencies within the data as well as practical problems regarding missing values are eliminated. On the other hand, when evaluating machine learning and other regression methods used for time series forecasting, often cross-validation is used for evaluation, paying little attention to the fact that those theoretical problems invalidate the fundamental assumptions of cross-validation. To close this gap and examine the consequences of different model selection procedures in practice, we have developed a rigorous and extensive empirical study. Six different model selection procedures, based on (i) cross-validation and (ii) evaluation using the series' last part, are used to assess the performance of four machine learning and other regression techniques on synthetic and real-world time series. No practical consequences of the theoretical flaws were found during our study, but the use of cross-validation techniques led to a more robust model selection. To make use of the ''best of both worlds'', we suggest that the use of a blocked form of cross-validation for time series evaluation became the standard procedure, thus using all available information and circumventing the theoretical problems.