Investigating Spatial Pattern Comparison Methods for Distributed Hydrological Model Assessment

Distributed hydrological models combine observations and knowledge about a hydrological system to make spatial predictions of hydrological attributes. These models require methods to assess their performance at spatial prediction. The current practice for assessment is simplistic. For qualitative assessment, simulated spatial patterns are compared visually against an observed pattern to assess their spatial similarity. To obtain a quantitative measure of similarity, each individual location is numerically compared to produce either a mean squared error (MSE) or correlation statistic. Both of these comparisons have their limitations. The visual comparison is subjective and the numerical comparison generally ignores the spatial structure of the patterns. There is demonstrable need for repeatable methods that can capture and quantify the important aspects of visual comparison. This paper demonstrates such a method from the image processing literature. It is a modification of the MSE statistic, called the information mean squared error (IMSE). This method weights each location in the spatial pattern by the ‘informativeness’ of ‘an event’ at that location. The weighted spatial patterns are then compared using a standard MSE statistic. IMSE aims to emulate human vision by more heavily weighting informative pixels. This paper applies IMSE to spatial patterns of soil moisture content. It is found to work well when using local variance as the ‘event’, as this helps enhance the general spatial trends that humans readily recognise. However, when the two spatial patterns are vastly different, IMSE proves to be less reliable due to the inconsistent weightings calculated for each spatial pattern.