Keywords
Calibration, Process-based models, Uncertainty analysis, Sensitivity analysis, Adequacy
Start Date
15-9-2020 12:40 PM
End Date
15-9-2020 1:00 PM
Abstract
The perceived lack of formal calibration and validation methods for agent-based models is regarded a major obstacle to their widespread acceptance as a rigorous scientific method. Even when only considering the field of socio-ecological and bio-economic modelling, ABM are used in a very heterogeneous spectrum of application contexts that considerably differ in their theoretical grounding, data availability, required precision and accuracy, and modelling purpose. Defining a single standard validation procedure for all contexts is certainly not adequate. To ensure transparency and consistent communication about model quality, however, a set of criteria is required to guide modellers, reviewers and stakeholders which validation methods to select and more generally which questions to ask about their models. We discuss the basic requirements for such criteria starting from the observation that it is actually not the model that needs to be valid, but the conclusions that are derived from the simulation analysis with respect to well-defined research questions. Starting from the basic arguments underlying common validation approaches, we summarize the criteria for the use of a variety of established methods, and demonstrate how an explicit formal outline of the argumentation structure in simulation analysis helps focusing the analysis of validity also in less straightforward contexts. Our contribution is intended to encourage an extended discussion in the session leading to a joint position paper among the agent-based modellers in the field.
Formalizing validation? Towards criteria for valid conclusions from agent-based simulation
The perceived lack of formal calibration and validation methods for agent-based models is regarded a major obstacle to their widespread acceptance as a rigorous scientific method. Even when only considering the field of socio-ecological and bio-economic modelling, ABM are used in a very heterogeneous spectrum of application contexts that considerably differ in their theoretical grounding, data availability, required precision and accuracy, and modelling purpose. Defining a single standard validation procedure for all contexts is certainly not adequate. To ensure transparency and consistent communication about model quality, however, a set of criteria is required to guide modellers, reviewers and stakeholders which validation methods to select and more generally which questions to ask about their models. We discuss the basic requirements for such criteria starting from the observation that it is actually not the model that needs to be valid, but the conclusions that are derived from the simulation analysis with respect to well-defined research questions. Starting from the basic arguments underlying common validation approaches, we summarize the criteria for the use of a variety of established methods, and demonstrate how an explicit formal outline of the argumentation structure in simulation analysis helps focusing the analysis of validity also in less straightforward contexts. Our contribution is intended to encourage an extended discussion in the session leading to a joint position paper among the agent-based modellers in the field.
Stream and Session
false