Model Validation and Testing:

Abstract

There is extensive literature on model validation and testing in ecosystem science. In spite of this solid foundation, a survey of the current literature reveals that the practice of model validation and testing has not matured over the past twenty-five years. Few exem-plars can be found that rigorously explain the rationale behind model formulation, the methods of parameter estimation, and the procedures for model testing. It is more common to find that the problems and pitfalls of validation and testing are poorly understood , inadequately executed, or entirely ignored. Among the reasons for this are the accelerating pace of technologies for model development has not been matched by the parallel development of methods for model analysis and testing; a trend in increased model complexity often requires the inclusion of critical variables that are unmeasured or unmeasurable (e.g., dispersal in population and metapopulation models); and the short duration of most research projects provides insufficient time and resources for adequate testing of model performance. These challenges are particularly daunting for models that are implemented spatially over large regions (e.g., landscape simulators). These difficulties are further exacerbated by the fact that no single method is suitable for testing all possible models and modeling objectives. Here we review a heuristic approach to model evaluation and present a general method for constructing informative model–data comparisons. Our premise is that models never work in their entirety (they always simplify reality to some extent), but even inadequate models can be used effectively so long as their behavior is well understood. This approach builds a richer appreciation for how a model matches data and the circumstances under which the model performs best. This fuller understanding of model–data comparisons allows client users to choose cases where the model can be used Validation and Testing 187 confidently or to take advantage of model bias to obtain conservative results. We believe that a generalized approach to a broad variety of models, including complex ecosystem simulators, must be developed. There are pressing needs for formal research on model construction , evaluation, and testing that should be a high priority for all ecosystems scientists.