Efficient Inference for Mechanistic Models with Hamiltonian Monte Carlo

Bayesian inference for parameter estimation in mechanistic models is difficult due to parameters which are correlated or high-dimensional and frustrate Markov Chain Monte Carlo methods based on the Metropolis-Hastings algorithm. Hamiltonian Monte Carlo (HMC), a class of gradient-based Bayesian inference methods, can function efficiently in this setting but appears to be unknown in the environmental modeling community. We implemented a conceptual hydrology model in Theano, an open source numerical computing framework in Python designed for rapid gradient computation using automatic differentiation. With this implementation, we empirically show that HMC allows for statistically coherent parameter estimation with a larger class of hydrology models than was previously tractable with Metropolis-Hastings. We apply this methodology to several case studies using simulated and real hydrology data to showcase its potential. Secondary benefits to utilizing Theano include faster computation for non-HMC based posterior sampling methods as well as the possibility of training complex hydrology models with gradient descent algorithms as in the case of artificial neural networks.