The importance of complexity in model selection
Journal of Mathematical Psychology
Is my model too complex? Evaluating model formulation using model reduction
Environmental Modelling & Software
| Hi-index | 0.00 |
Model averaging is a group of methods for combining predictions from several models which have the benefit of considering model uncertainty in addition to parameter uncertainty. The aim of this paper is to introduce these methods in the context of mechanistic model development. In model averaging predictions are combined, by weighting with factors related to model performance, resulting in ensemble predictions. Bayesian Model Averaging (BMA) is model averaging in a Bayesian framework where the model weights are Posterior model probabilities (PMPs). We describe three approximation methods (AIC, BIC and Laplace) for calculating PMPs and to compare with a full Bayesian approach implemented using a Markov Chain Monte Carlo (MCMC) method (Metropolis-Hastings). We also describe a simplified BMA approach which is readily implemented, as it only requires the maximum likelihood parameter estimates and Laplace approximation of the marginal likelihoods. We illustrate the application of BMA using a mechanistic model for predicting the plant uptake of radiocaesium from contaminated soils (the 'Absalom Model'). Ten models were selected for averaging, these comprised the full Absalom model and nine reduced models each derived from the full model. To assess performance model predictions and ensemble predictions were compared using an independent data set. The PMPs estimated using the MCMC approach and the Laplace approximation were similar and strongly weighted the models with fewer parameters. The AIC- and BIC-based estimates of the PMPs were correlated but differed considerably from the Laplace and MCMC-based PMP methods. For our example the simplified BMA approach was performed as well as the full approach. Individual predictions differed among models and the prediction ensembles resulting from all the approaches captured this uncertainty. We conclude that BMA is a valuable approach, relevant to mechanistic model development, and suggest a framework for incorporating BMA into model development.