Keywords
uncertainty analysis, optimization, calibration, watershed modeling
Start Date
1-7-2006 12:00 AM
Abstract
Environmental simulation models areapproximations of reality, and are therefore allsubject to varying degrees of uncertainty. Ingeneral, uncertainty sources in environmentalmodeling include parameter, data and modelstructure. When the uncertainty of these modelinputs are quantified in terms of probabilitydistributions, a traditional Monte Carlopropagation of input uncertainty can be performed.However, this traditional approach becomes muchmore complicated when model calibration data isconsidered because the random input sets sampledfrom the joint parameter and input distributionsmust also be deemed to produce reasonablepredictions of the available measured calibrationdata. Two types of methods that were developedto cope with this complication are the GeneralizedLikelihood Uncertainty Estimation or GLUEmethodology (Beven and Binley, 1992) andMarkov Chain Monte Carlo or MCMC methods asdemonstrated for a watershed modeling case studyby Kuczera and Parent (1998).A review of the uncertainty literaturedemonstrating MCMC or GLUE methodologiesfor model calibration uncertainty analysis showsthat the number of model evaluations required(typically more than 10,000 or even 100,000depending on the number of uncertain modelinputs) is extremely prohibitive and perhapsimpossible computationally demanding models.The purpose of this research project is to developan alternative approximate, high-dimensionaluncertainty analysis methodology calledDynamically Dimensioned Search – UncertaintyApproximation (DDS-UA). The simple andefficient DDS algorithm developed by Tolson(2005) forms the basis method. We compareDDS-UA to uncertainty analysis results achievedwith the GLUE methodology. GLUE was selectedfor comparison based on its simplicity, prevalencein the literature and because it does not require astatistically based likelihood function. The DDSUAmethod is focused primarily on efficiently andeffectively identifying multiple high likelihood(i.e. high quality) model parameter sets. Weperformed comparisons against the GeneralizedLikelihood Uncertainty Estimation (GLUE)methodology for 14 to 30 uncertain calibrationparameters for a watershed modeling calibrationcase study and showed the DDS-UA approach tobe one to three orders of magnitude morecomputationally efficient. For example, in 20,000to 150,000 model evaluations, GLUE sampledlikelihoods were substantially lower than theknown maximum likelihood values generated byDDS-UA. DDS-UA sampling effectiveness andefficiency under both default and reducedparameter ranges was shown to be similar,indicating that DDS-UA can generate highlikelihood solutions even when little prior casestudy knowledge is available.DDS-UA enables more computationally expensivesimulation models and/or a wider range ofmodeling case studies to realistically undertake aglobal analysis of uncertainty during modelcalibration. As with GLUE, the simplicity of themethodology make it an attractive approach thatcan implemented without excessive programming.Unlike GLUE, which typically relies on uniformrandom sampling to find high likelihood solutions,DDS-UA utilizes the simple and efficient DDSoptimization algorithm to search more efficientlyand effectively for these solutions. DDS isspecifically used to find multiple high likelihoodsolutions from independent optimization trials.This approach distinguishes DDS-UA fromevolutionary optimization based approaches forapproximating uncertainty that identify highlikelihood solutions from the search history of asingle optimization trial.Beven, K. and Binley, A., 1992. The Future OfDistributed Models - Model Calibration AndUncertainty Prediction. Hydrological Processes,6(3): 279-298.Kuczera, G. and Parent, E., 1998. Monte Carloassessment of parameter uncertainty in conceptualcatchment models: the Metropolis algorithm.Journal Of Hydrology, 211(1-4): 69-85.Tolson, B. A. (2005), Automatic calibration,management and uncertainty analysis: Phosphorustransport in the Cannonsville Watershed, Ph.D.thesis, Cornell University, Ithaca, N.Y.
Using Optimization for Environmental Simulation Model Calibration Uncertainty Analysis
Environmental simulation models areapproximations of reality, and are therefore allsubject to varying degrees of uncertainty. Ingeneral, uncertainty sources in environmentalmodeling include parameter, data and modelstructure. When the uncertainty of these modelinputs are quantified in terms of probabilitydistributions, a traditional Monte Carlopropagation of input uncertainty can be performed.However, this traditional approach becomes muchmore complicated when model calibration data isconsidered because the random input sets sampledfrom the joint parameter and input distributionsmust also be deemed to produce reasonablepredictions of the available measured calibrationdata. Two types of methods that were developedto cope with this complication are the GeneralizedLikelihood Uncertainty Estimation or GLUEmethodology (Beven and Binley, 1992) andMarkov Chain Monte Carlo or MCMC methods asdemonstrated for a watershed modeling case studyby Kuczera and Parent (1998).A review of the uncertainty literaturedemonstrating MCMC or GLUE methodologiesfor model calibration uncertainty analysis showsthat the number of model evaluations required(typically more than 10,000 or even 100,000depending on the number of uncertain modelinputs) is extremely prohibitive and perhapsimpossible computationally demanding models.The purpose of this research project is to developan alternative approximate, high-dimensionaluncertainty analysis methodology calledDynamically Dimensioned Search – UncertaintyApproximation (DDS-UA). The simple andefficient DDS algorithm developed by Tolson(2005) forms the basis method. We compareDDS-UA to uncertainty analysis results achievedwith the GLUE methodology. GLUE was selectedfor comparison based on its simplicity, prevalencein the literature and because it does not require astatistically based likelihood function. The DDSUAmethod is focused primarily on efficiently andeffectively identifying multiple high likelihood(i.e. high quality) model parameter sets. Weperformed comparisons against the GeneralizedLikelihood Uncertainty Estimation (GLUE)methodology for 14 to 30 uncertain calibrationparameters for a watershed modeling calibrationcase study and showed the DDS-UA approach tobe one to three orders of magnitude morecomputationally efficient. For example, in 20,000to 150,000 model evaluations, GLUE sampledlikelihoods were substantially lower than theknown maximum likelihood values generated byDDS-UA. DDS-UA sampling effectiveness andefficiency under both default and reducedparameter ranges was shown to be similar,indicating that DDS-UA can generate highlikelihood solutions even when little prior casestudy knowledge is available.DDS-UA enables more computationally expensivesimulation models and/or a wider range ofmodeling case studies to realistically undertake aglobal analysis of uncertainty during modelcalibration. As with GLUE, the simplicity of themethodology make it an attractive approach thatcan implemented without excessive programming.Unlike GLUE, which typically relies on uniformrandom sampling to find high likelihood solutions,DDS-UA utilizes the simple and efficient DDSoptimization algorithm to search more efficientlyand effectively for these solutions. DDS isspecifically used to find multiple high likelihoodsolutions from independent optimization trials.This approach distinguishes DDS-UA fromevolutionary optimization based approaches forapproximating uncertainty that identify highlikelihood solutions from the search history of asingle optimization trial.Beven, K. and Binley, A., 1992. The Future OfDistributed Models - Model Calibration AndUncertainty Prediction. Hydrological Processes,6(3): 279-298.Kuczera, G. and Parent, E., 1998. Monte Carloassessment of parameter uncertainty in conceptualcatchment models: the Metropolis algorithm.Journal Of Hydrology, 211(1-4): 69-85.Tolson, B. A. (2005), Automatic calibration,management and uncertainty analysis: Phosphorustransport in the Cannonsville Watershed, Ph.D.thesis, Cornell University, Ithaca, N.Y.
