K. B. Matthews

Validation of biophysical models: issues and methodologies. A review

The potential of mathematical models is widely acknowledged for examining components and interactions of natural systems, estimating the changes and uncertainties on outcomes, and fostering communication between scientists with different backgrounds and between scientists, managers and the community. For favourable reception of models, a systematic accrual of a good knowledge base is crucial for both science and decision-making. As the roles of models grow in importance, there is an increase in the need for appropriate methods with which to test their quality and performance. For biophysical models, the heterogeneity of data and the range of factors influencing usefulness of their outputs often make it difficult for full analysis and assessment. As a result, modelling studies in the domain of natural sciences often lack elements of good modelling practice related to model validation, that is correspondence of models to its intended purpose. Here we review validation issues and methods currently available for assessing the quality of biophysical models. The review covers issues of validation purpose, the robustness of model results, data quality, model prediction and model complexity. The importance of assessing input data quality and interpretation of phenomena is also addressed. Details are then provided on the range of measures commonly used for validation. Requirements for a methodology for assessment during the entire model-cycle are synthesised. Examples are used from a variety of modelling studies which mainly include agronomic modelling, e.g. crop growth and development, climatic modelling, e.g. climate scenarios, and hydrological modelling, e.g. soil hydrology, but the principles are essentially applicable to any area. It is shown that conducting detailed validation requires multi-faceted knowledge, and poses substantial scientific and technical challenges. Special emphasis is placed on using combined multiple statistics to expand our horizons in validation whilst also tailoring the validation requirements to the specific objectives of the application.

Implementation of a spatial decision support system for rural land use planning: integrating geographic information system and environmental models with search and optimisation algorithms

Abstract The implementation of a spatial decision support system (DSS) developed as a tool for rural land use planning at the management unit level is described. The DSS fulfils the need for a tool that allows rural land managers to explore their land use options and the potential impacts of land use change. The DSS is based on five components: a geographic information system (GIS); land use modules; impact assessment modules; a graphical user interface; and land use planning tools. These components are implemented across two software platforms Gensym’s G2 knowledge based system (KBS) development environment and Smallworld GIS. Following a review of the DSS components, the paper focuses on two aspects. First, the use of the object-orientation paradigm to facilitate the integration of geospatial information. Second is the proposed use of genetic algorithms, a class of search and optimisation algorithm, to find optimum land use plans using the integrated functionality of both KBS and GIS.

An integrated assessment approach to conduct analyses of climate change impacts on whole-farm systems

Abstract This paper argues that an integrated assessment (IA) approach, combining simulation modelling with deliberative processes involving decision makers and other stakeholders, has the potential to generate credible and relevant assessments of climate change impacts on farming systems. The justification for the approach proposed is that while simulation modelling provides an effective way of exploring the range of possible impacts of climate change and a means of testing the consequences of possible management or policy interventions, the interpretation of the outputs is highly dependent on the point of view of the stakeholder. Inevitably, whatever the responses to climate change, there will be trade-offs between the benefits and costs to a range of stakeholders. The use of a deliberative process that includes stakeholders, both in defining the topics addressed and in debating the interpretations of the outcomes, addresses many of the limitations that have been previously identified in the use of computer-based tools for agricultural decision support. The paper further argues that the concepts of resilience and adaptive capacity are useful for the assessment of climate change impacts as they provide an underpinning theory for processes of change in land use systems. The integrated modelling framework (IMF) developed for the simulation of whole-farm systems is detailed, including components for crop and soil processes, livestock systems and a tool for scheduling of resource use within management plans. The use of the IMF for assessing climate change impacts is then outlined to demonstrate the range of analyses possible. The paper concludes with a critique of the IA approach and notes that issues of quantification and communication of uncertainty are central to the success of the methodology.

Validation of Biophysical Models: Issues and Methodologies

The potential of mathematical models is widely acknowledged for examining components and interactions of natural systems, estimating the changes and uncertainties on outcomes, and fostering communication between scientists with different backgrounds and between scientists, managers and the community. For favourable reception of models, a systematic accrual of a good knowledge base is crucial for both science and decision-making. As the roles of models grow in importance, there is an increase in the need for appropriate methods with which to test their quality and performance. For biophysical models, the heterogeneity of data and the range of factors influencing usefulness of their outputs often make it difficult for full analysis and assessment. As a result, modelling studies in the domain of natural sciences often lack elements of good modelling practice related to model validation, that is correspondence of models to its intended purpose. Here we review validation issues and methods currently available for assessing the quality of biophysical models. The review covers issues of validation purpose, the robustness of model results, data quality, model prediction and model complexity. The importance of assessing input data quality and interpretation of phenomena is also addressed. Details are then provided on the range of measures commonly used for validation. Requirements for a methodology for assessment during the entire model-cycle are synthesised. Examples are used from a variety of modelling studies which mainly include agronomic modelling, e.g. crop growth and development, climatic modelling, e.g. climate scenarios, and hydrological modelling, e.g. soil hydrology, but the principles are essentially applicable to any area. It is shown that conducting detailed validation requires multi-faceted knowledge, and poses substantial scientific and technical challenges. Special emphasis is placed on using combined multiple statistics to expand our horizons in validation whilst also tailoring the validation requirements to the specific objectives of the application.

Raising the bar? - The challenges of evaluating the outcomes of environmental modelling and software

The intention of this paper it to open up debate within the environmental modelling and software (EMS) community on how best to respond to the increasing desire to evaluate the success of EMS projects in terms of outcomes rather than outputs. Outcomes in these regards are changes beyond the walls of the research organisation (typically to values, attitudes and behaviour). The authors recognise that outcome evaluation is essential in ensuring the relevance and effectiveness of activities. To date, however, there is a limited appreciation within the EMS community of the nature of the challenge inherent in outcome evaluations. The paper presents an exploratory analysis of the challenges that outcome assessment raises for EMS. It does so using mutually reinforcing conceptual and practical perspectives. The paper presents a conceptual framework of three loosely coupled phases - research, development and operations. The nature of activities and their interactions within these phases is outlined and the forms of evaluation associated with each stage set out. The paper notes how existing forms of evaluation (e.g. peer review, validation and relevance) underpin the delivery of outcomes but do not of themselves evaluate outcomes. The paper proposes that outcomes need conceptually to be seen as an element of complex social processes mediated by government, regulation, markets and the media rather than as simply another form of output from research and development projects. As such outcomes of EMS are: less easily tangible than are outputs; more likely to occur at a significant time lag after any intervention; more difficult to assign causality for and to be subject to significant contestation. Thus EMS activity, however well conducted technically, may only have a minor influence on outcomes and EMS practitioners will have limited control over those outcomes that do occur. The paper uses a series of linked EMS projects to populate the conceptual framework showing the role of evaluations in research, development and operations phases. The paper then presents two forms (quantitative and qualitative) of outcome evaluation used as part of an operational phase evaluation of a project communicating the consequences of climate change to remote-rural land managers in Scotland. The authors conclude that while the challenges of EMS evaluation can be met, there needs to be care from the EMS community not to raise expectations of outcomes that cannot be met.

Bridging the gaps between design and use: developing tools to support environmental management and policy

Abstract Integrated assessment models, decision support systems (DSS) and Geographic Information Systems (GIS) are examples of a growing number of computer-based tools designed to provide decision and information support to people engaged in formulating and implementing environmental policy and management. It is recognised that environmental policy and management users are often not as receptive to using such tools as desired but that little research has been done to uncover and understand the reasons. There is a diverse range of environmental decision and information support tools (DISTs) with uses including organisational and participatory decision support, and scientific research. The different uses and users of DISTs each present particular needs and challenges to the tool developers. The lack of appreciation of the needs of end-users by developers has contributed to the lack of success of many DISTs. Therefore it is important to engage users and other stakeholders in the tool development process to help bridge the gap between design and use. Good practice recommendations for developers to involve users include being clear about the purpose of the tool, working collaboratively with other developers and stakeholders, and building social and scientific credibility.

Climate change impact on distribution and abundance of wildlife species: an analytical approach using GIS.

An analytical approach to modelling the likely impact of climate change on the distribution and abundance of wildlife species is described using examples from Scotland. Data for present day distribution of wildlife and habitat are analysed using map data describing geographic variation in climatic factors. Climate data for the present day and under specified scenarios of change are themselves modelled within a GIS; climate modelling uses meteorological station data, climate change scenarios developed from GCMs and a variety of spatial interpolation techniques. The analytical procedure generates hypotheses defining ecological relationships between species distribution and climatic factors (monthly, seasonal and annual data). These relationships are then used to model the distribution of the species directly from climate and predict impacts of climate change. The analysis takes account of both direct impacts of climate on wildlife and indirect effects manifested through habitat response to climate change. The analytical procedure is implemented as a generic tool for inductive spatial analysis in GIS.

Climatic soil moisture deficit - climate and soil data integration in a GIS

This paper discusses a GIS based implementation of a model for soil droughtiness assessment evaluating the impact of possible climate change. It focuses, in particular, on the development of a methodology for mapping Available Water Capacity. An assessment of the Soil Drought Susceptibility for Scotland in the year 2030 is made and illustrated with maps and derived statistics.

The impact of climate change on the soil/moisture regime of Scottish mineral soils

The likely impact of climate change on the moisture regime of Scottish soils and consequently on agriculture and land use has been addressed using a novel Geographic Information Systems (GIS) approach. Current estimates of changes in summer precipitation by the year 2030 are 0% with an associated uncertainty of +/- 11%. This study considers the worst case scenario of a decrease in rainfall by 11% which will lead to some low rainfall areas experiencing an increased drought risk, particularly on lighter soils. Wet areas with heavy soils could benefit from an increase in the accessibility period for machinery. As the major agricultural land in Scotland is located on the relatively dry east coast where localised problems due to drought are not uncommon even under the present climate, the detrimental effects of a decrease in rainfall for the whole of Scotland are therefore likely to outweigh the benefits. Approximately 8% of Scotland has been identified in this study as soil/climate combinations which will be susceptible to drought should summer rainfall decrease by 11% and summer temperature increase by 1.4 degrees C.

Preface: Thematic issue on the assessment and evaluation of environmental models and software

1364-8152/