Keywords

Fisher Information KIC Model Selection

Start Date

27-6-2018 10:40 AM

End Date

27-6-2018 12:00 PM

Abstract

Groundwater models are commonly constructed using a small amount of data relative to the size of the area modeled. Considering only a single groundwater model ignores the uncertainty associated with model structure. Evaluating multiple groundwater models of a system considers model structural uncertainty and provides for a more realistic assessment of prediction uncertainty. Kashyap information criteria (KIC) is an equation used in multi-model analysis (MMA) to evaluate a set of models and the probability that a model represents the true, but unknown, system. Fisher Information (FI) measures information content of a model and is an influential component in the KIC equation. Low FI and low model error contribute to a low KIC score resulting in a high probability that the model represents the unknown system. However, models with low FI can be over-fitted models in which the model structure is not supported by the data. These models can produce extreme variance on both parameter estimates and model predictions. A synthetic groundwater model is used to represent a “true” but unknown system with hydraulic conductivity properties. A set of experimental models represent models created to simulate the unknown system. Unsupportable models in the model set have low FI and high KIC model probability. A prediction in the area lacking information has an extremely large variance and uncertainty of the estimate. Removing unsupportable models from the model set and re-evaluating using MMA, results in a more reasonable KIC estimate of the true but unknown prediction.

Stream and Session

Stream F: System Identification Approaches for Complex Environmental Systems

F3: Modelling and Decision Making Under Uncertainty

Download
COinS
 
Jun 27th, 10:40 AM Jun 27th, 12:00 PM

The influence of Fisher Information in KIC model selection

Groundwater models are commonly constructed using a small amount of data relative to the size of the area modeled. Considering only a single groundwater model ignores the uncertainty associated with model structure. Evaluating multiple groundwater models of a system considers model structural uncertainty and provides for a more realistic assessment of prediction uncertainty. Kashyap information criteria (KIC) is an equation used in multi-model analysis (MMA) to evaluate a set of models and the probability that a model represents the true, but unknown, system. Fisher Information (FI) measures information content of a model and is an influential component in the KIC equation. Low FI and low model error contribute to a low KIC score resulting in a high probability that the model represents the unknown system. However, models with low FI can be over-fitted models in which the model structure is not supported by the data. These models can produce extreme variance on both parameter estimates and model predictions. A synthetic groundwater model is used to represent a “true” but unknown system with hydraulic conductivity properties. A set of experimental models represent models created to simulate the unknown system. Unsupportable models in the model set have low FI and high KIC model probability. A prediction in the area lacking information has an extremely large variance and uncertainty of the estimate. Removing unsupportable models from the model set and re-evaluating using MMA, results in a more reasonable KIC estimate of the true but unknown prediction.