Constructing model credibility in the context of policy appraisal

• Authors: Craig A. Aumann
• Venue: Environmental Modelling & Software
• Year: 2011

This paper focuses on an approach for constructing credibility in models used to evaluate environmental policies. Credibility is defined as adequacy relative to an intended use. The proposed approach is located within a broader policy assessment framework and is general enough to function in both scientific and policy assessment contexts. A model assessment framework is introduced and linked to current understanding of mental models in cognitive psychology to provide a possible explanation of how stakeholders come to view modeling results as credible. Since policy evaluation is about an unknown and unknowable future, data to assess model performance will be limited. A number of options are suggested for how credibility can be constructed in such situations. The argument presented depends primarily on i) the hierarchical levels of simulation models, ii) the ability to apply model assessment criteria specified across these hierarchical levels, and iii) the public nature of these criteria relative to the mental models of stakeholders enables debate about the adequacy of the criteria relative to the intended purpose of the policy appraisal. Taken together these aspects enables stakeholders to reach agreement on necessary and desirable performance criteria for the modeling results to be credible.