A Methodology for Building Credible Models for Policy Evaluation

The need for tools capable of evaluating the potential impacts of alternative policies has been expressed by many. This paper focuses on a methodology that describes how credibility can be constructed for models used to evaluate alternative policies. Relative to modeling conducted in scientific contexts, however, modeling for policy evaluation has notable differences that would lead some to say that achieving credibility in such results is not possible. We first introduce a model assessment framework that enables us to describe how models for policy evaluation can still be more or less credible despite the differences from scientific modeling contexts. The argument presented depends primarily on i) the scalar hierarchical structure used to represent the complex policy system, ii) the ability of experimental frames to include a variety of constraints and/or weak data patterns across the scalar levels used to represent the system, and iii) the way in which the framework facilitates critique by stakeholders.