Towards a Multi-criteria Approach to Validate Agent-based Model for Socio-ecological Systems Transitions

Due to inherent complexity, path-dependency, deep uncertainty of socio-ecological systems (SES) in transition that agent-based models (ABM) are often meant to simulate, validation of ABM for SES is expanded beyond the straightforward evaluation of numerical fits between simulated and observed patterns in time and space. With a metaphorical thought about meaningful uses of the complex system models, the validity of this model type lies within the continuous review/evaluation with multiple criteria to inform users about the model’s usefulness, its limitations and development needs. From literature of model validation, we outline multiple criteria that should be considered in ABM evaluation: (1) the fitting of the model to the questions it is meant to answer, (2) the plausibility of the assumptions and theories forming the model (construct validity), (3) the validity of elementary causal relations used for constructing the model (e.g. behavioural rules and sub-models) (internal validity), (4) the validity of input data, and (5) the validity of model outputs. Furthermore, considering distinguished characteristics of ABMs compared to other modelling methods, we suggested specific foci in ABM validation. Because ABM is a generative modelling approach, construct validity should be the umbrella criterion to assess the validity of the approach. With lessons learned from efforts on validation of an ABM in a number of example cases, we demonstrate used criteria, successfulness, remained challenges and limitations, as well as discuss optional solutions along the validation criteria.