Soft systems methodology in action
Soft systems methodology in action
Exploratory modeling for policy analysis
Operations Research
Verifying and validating simulation models
WSC '96 Proceedings of the 28th conference on Winter simulation
Note on free lunches and cross-validation
Neural Computation
Verification & validation in military simulations
Proceedings of the 29th conference on Winter simulation
Computational organization theory
Multiagent systems
Business Dynamics
Simulation Validation - a Confidence Assessment Methodology
Simulation Validation - a Confidence Assessment Methodology
A tutorial on verification and validation of simulation models
WSC '84 Proceedings of the 16th conference on Winter simulation
Modeling factions for "effects based operations": part I--leaders and followers
Computational & Mathematical Organization Theory
Modeling factions for `effects based operations', part II: behavioral game theory
Computational & Mathematical Organization Theory
A Discussion on Experimental Model Validation
UKSIM '09 Proceedings of the UKSim 2009: 11th International Conference on Computer Modelling and Simulation
An Approach for Validation of Semantic Composability in Simulation Models
PADS '09 Proceedings of the 2009 ACM/IEEE/SCS 23rd Workshop on Principles of Advanced and Distributed Simulation
Computational Methods for Counterterrorism
Computational Methods for Counterterrorism
Intelligent Decision Technologies - Engineering and management of IDTs for knowledge management systems
Engineering Principles of Combat Modeling and Distributed Simulation
Engineering Principles of Combat Modeling and Distributed Simulation
Sociocultural Games for Training and Analysis
IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans
A domain-independent framework for modeling emotion
Cognitive Systems Research
Using workflows in M&S software
Proceedings of the Winter Simulation Conference
| Hi-index | 0.00 |
The philosophical perspectives on model evaluation can be broadly classified into reductionist/logical positivist and relativist/holistic. In this paper, we outline some of our past efforts in, and challenges faced during, evaluating models of social systems with cognitively detailed agents. Owing to richness in the model, we argue that the holistic approach and consequent continuous improvement are essential to evaluating complex social system models such as these. A social system built primarily of cognitively detailed agents can provide multiple levels of correspondence, both at observable and abstract aggregated levels. Such a system can also pose several challenges, including large feature spaces, issues in information elicitation with database, experts and news feeds, counterfactuals, fragmented theoretical base, and limited funding for validation. We subscribe to the view that no model can faithfully represent reality, but detailed, descriptive models are useful in learning about the system and bringing about a qualitative jump in understanding of the system it attempts to model - provided they are properly validated. Our own approach to model evaluation is to consider the entire life cycle and assess the validity under two broad dimensions of (1) internally focused validity/quality achieved through structural, methodological, and ontological evaluations; and (2) external validity consisting of micro validity, macro validity, and qualitative, causal and narrative validity. In this paper, we also elaborate on selected validation techniques that we have employed in the past. We recommend a triangulation of multiple validation techniques, including methodological soundness, qualitative validation techniques, such as face validation by experts and narrative validation, and formal validation tests, including correspondence testing.