Keywords
error surface; model complexity; exploratory landscape analysis; artificial neural networks;conceptual rainfall runoff model; model calibration
Start Date
7-7-2022 1:40 PM
End Date
7-7-2022 2:00 PM
Abstract
It is well-known that model complexity has an impact on the features of error surfaces of environmental models, which has impact on model calibration performance. For example, increasing model complexity results in an increase in the number of local optima of error surfaces, making it more difficult to identify globally optimal solutions. However, previous attempts at identifying these relationships have been ad-hoc, generally relying on graphical, lower-dimensional representations of higher-dimensional surfaces to identify relevant features in the error surface. This makes it difficult to determine the relationship between model complexity and model calibration performance in an objective fashion. In this study, Exploratory Landscape Analysis (ELA) metrics are applied to two types of environmental models, including multi-layer perceptron artificial neural networks (ANNs) and conceptual rainfall runoff (CRR) models. The metrics are used to determine the relationship between model complexity and calibration performance by assessing the features of corresponding error surfaces for a range of case studies. Results indicate that increase in structure complexity of ANNs leads to flatter error surfaces with more dispersed and deeper local optima. Consequently, simpler ANNs can be calibrated successfully using gradient-based methods, whereas more complex ANNs are best calibrated using a hybrid approach combining metaheuristics with gradient-based methods. Results also show that for CRR models, increasing model complexity results in an increase in relative error surface roughness and optima dispersion, while increasing catchment wetness increases the relative roughness of error surfaces but decreases optima dispersion. This suggests that optimisation efficiency decreases with model complexity and catchment wetness, while optimisation difficulty increases and parameter uniqueness deceases with model complexity and catchment dryness. These results demonstrate the potential for using ELA metrics to better understand the relationship between model complexity and calibration performance, assisting with the determination of optimal model complexity and/or calibration approaches that are most appropriate.
Improved Understanding of the Relationship between ModelComplexity and Calibration Performance using ExploratoryLandscape Analysis Metrics
It is well-known that model complexity has an impact on the features of error surfaces of environmental models, which has impact on model calibration performance. For example, increasing model complexity results in an increase in the number of local optima of error surfaces, making it more difficult to identify globally optimal solutions. However, previous attempts at identifying these relationships have been ad-hoc, generally relying on graphical, lower-dimensional representations of higher-dimensional surfaces to identify relevant features in the error surface. This makes it difficult to determine the relationship between model complexity and model calibration performance in an objective fashion. In this study, Exploratory Landscape Analysis (ELA) metrics are applied to two types of environmental models, including multi-layer perceptron artificial neural networks (ANNs) and conceptual rainfall runoff (CRR) models. The metrics are used to determine the relationship between model complexity and calibration performance by assessing the features of corresponding error surfaces for a range of case studies. Results indicate that increase in structure complexity of ANNs leads to flatter error surfaces with more dispersed and deeper local optima. Consequently, simpler ANNs can be calibrated successfully using gradient-based methods, whereas more complex ANNs are best calibrated using a hybrid approach combining metaheuristics with gradient-based methods. Results also show that for CRR models, increasing model complexity results in an increase in relative error surface roughness and optima dispersion, while increasing catchment wetness increases the relative roughness of error surfaces but decreases optima dispersion. This suggests that optimisation efficiency decreases with model complexity and catchment wetness, while optimisation difficulty increases and parameter uniqueness deceases with model complexity and catchment dryness. These results demonstrate the potential for using ELA metrics to better understand the relationship between model complexity and calibration performance, assisting with the determination of optimal model complexity and/or calibration approaches that are most appropriate.
Stream and Session
false