How useful are “validated” artificial neural network models for increasing environmental system understanding?

Artificial neural network models have been used extensively for predictive purposes but can also be used to gain insights into the relationships that drive complex environmental systems that are poorly understood. However, while the validation of most types of environmental models includes residual analysis (replicative validation) and an assessment of how plausible the input-output relationship represented by the calibrated model is (structural validation), in addition to assessing predictive performance on an independent validation set (predictive validity), this is generally not done for ANN models. In order to enable these additional aspects of validation to be incorporated in the development of ANN models, a validation framework for ANNs and an R-package that enables this framework to be implemented in a user-friendly and consistent fashion are introduced and tested on two different environmental modelling case studies. Results highlight the importance of performing replicative and structural validation in addition to predictive validation, as in each case, the results revealed that ANN models producing the best fit to the data do not necessarily result in either plausible models or models which best capture the underlying relationship in the training data. By applying the proposed validation framework and R package, the chances of developing ANN models that are more suitable to increasing environmental system understanding are increased significantly.