Keywords
Fisher Information KIC Model Selection
Start Date
27-6-2018 10:40 AM
End Date
27-6-2018 12:00 PM
Abstract
Groundwater models are commonly constructed using a small amount of data relative to the size of the area modeled. Considering only a single groundwater model ignores the uncertainty associated with model structure. Evaluating multiple groundwater models of a system considers model structural uncertainty and provides for a more realistic assessment of prediction uncertainty. Kashyap information criteria (KIC) is an equation used in multi-model analysis (MMA) to evaluate a set of models and the probability that a model represents the true, but unknown, system. Fisher Information (FI) measures information content of a model and is an influential component in the KIC equation. Low FI and low model error contribute to a low KIC score resulting in a high probability that the model represents the unknown system. However, models with low FI can be over-fitted models in which the model structure is not supported by the data. These models can produce extreme variance on both parameter estimates and model predictions. A synthetic groundwater model is used to represent a “true” but unknown system with hydraulic conductivity properties. A set of experimental models represent models created to simulate the unknown system. Unsupportable models in the model set have low FI and high KIC model probability. A prediction in the area lacking information has an extremely large variance and uncertainty of the estimate. Removing unsupportable models from the model set and re-evaluating using MMA, results in a more reasonable KIC estimate of the true but unknown prediction.
The influence of Fisher Information in KIC model selection
Groundwater models are commonly constructed using a small amount of data relative to the size of the area modeled. Considering only a single groundwater model ignores the uncertainty associated with model structure. Evaluating multiple groundwater models of a system considers model structural uncertainty and provides for a more realistic assessment of prediction uncertainty. Kashyap information criteria (KIC) is an equation used in multi-model analysis (MMA) to evaluate a set of models and the probability that a model represents the true, but unknown, system. Fisher Information (FI) measures information content of a model and is an influential component in the KIC equation. Low FI and low model error contribute to a low KIC score resulting in a high probability that the model represents the unknown system. However, models with low FI can be over-fitted models in which the model structure is not supported by the data. These models can produce extreme variance on both parameter estimates and model predictions. A synthetic groundwater model is used to represent a “true” but unknown system with hydraulic conductivity properties. A set of experimental models represent models created to simulate the unknown system. Unsupportable models in the model set have low FI and high KIC model probability. A prediction in the area lacking information has an extremely large variance and uncertainty of the estimate. Removing unsupportable models from the model set and re-evaluating using MMA, results in a more reasonable KIC estimate of the true but unknown prediction.
Stream and Session
Stream F: System Identification Approaches for Complex Environmental Systems
F3: Modelling and Decision Making Under Uncertainty