Providing scientific certainty in predictive decision support: the role of closed questions

Uncertainty in scientific predictions of outcome of decisions is currently of substantial interest, with a common perception, at least in groundwater modelling, that some decision makers have unrealistic expectations of accuracy. A feeling shared by some practitioners is that with a proper stakeholder engagement process, uncertainty is no longer such a difficult issue. This paper argues that the key is precisely specifying purpose and expectations, and the definition of closed questions provides a useful approach to do so. A closed question, also known as a fixed-choice or pre-coded question, has a finite set of answers, e.g. yes/no or A, B, C. If an answer is possible, it will always be certain, but science may need to say “we do not know”. The approach is conceptually simple, though not necessarily trivial in practice, often requiring iteration between interest groups and modellers. Closed questions provide an interface between scientists and stakeholders which has a number of advantages, illustrated using examples from the published literature. Defining closed questions demands that proper attention be paid to problem formulation, which has long been recognised as important in the treatment of messy problems so pervasive in environmental modelling. The stakeholder engagement process manages the uncertainty in the definition of the problem, freeing the scientific process to concentrate on uncertainty in the model. Closed questions specify the purpose and required precision of the prediction, allowing the use of simpler models and techniques. This helps address the challenges of defining context-dependent criteria for acceptable model quality and determining appropriate model complexity. By addressing values in a proper process involving both stakeholders and scientists, the role of the scientist in providing objective predictions is separated from their potentially more subjective contributions to the remainder of the process. The common use of scientific hypothesis testing shows that probabilistic uncertainty can also be represented, allowing stakeholders to explicitly accept risk. As a result, a well-run, engaging process using closed questions allows science to promptly answer stakeholder questions with certainty, or recognize that an answer is not possible with the allocated resources, and therefore that the problem must be treated as trans-scientific.