Gianni Bellocchi

Validation of biophysical models: issues and methodologies. A review

The potential of mathematical models is widely acknowledged for examining components and interactions of natural systems, estimating the changes and uncertainties on outcomes, and fostering communication between scientists with different backgrounds and between scientists, managers and the community. For favourable reception of models, a systematic accrual of a good knowledge base is crucial for both science and decision-making. As the roles of models grow in importance, there is an increase in the need for appropriate methods with which to test their quality and performance. For biophysical models, the heterogeneity of data and the range of factors influencing usefulness of their outputs often make it difficult for full analysis and assessment. As a result, modelling studies in the domain of natural sciences often lack elements of good modelling practice related to model validation, that is correspondence of models to its intended purpose. Here we review validation issues and methods currently available for assessing the quality of biophysical models. The review covers issues of validation purpose, the robustness of model results, data quality, model prediction and model complexity. The importance of assessing input data quality and interpretation of phenomena is also addressed. Details are then provided on the range of measures commonly used for validation. Requirements for a methodology for assessment during the entire model-cycle are synthesised. Examples are used from a variety of modelling studies which mainly include agronomic modelling, e.g. crop growth and development, climatic modelling, e.g. climate scenarios, and hydrological modelling, e.g. soil hydrology, but the principles are essentially applicable to any area. It is shown that conducting detailed validation requires multi-faceted knowledge, and poses substantial scientific and technical challenges. Special emphasis is placed on using combined multiple statistics to expand our horizons in validation whilst also tailoring the validation requirements to the specific objectives of the application.

RadEst3.00: software to estimate daily radiation data from commonly available meteorological variables

Abstract RadEst3.00 estimates and evaluates daily global solar radiation values at given latitudes. Radiation is calculated as the product of the atmospheric transmissivity of radiation times the radiation outside the earth atmosphere. Four models estimate the atmospheric transmissivity, based on the daily temperature range. Model parameters can be fitted over one or more years of data by iterative procedures. Graphical and statistical evaluations of the estimates are presented. Reports of the analysis can be exported in a variety of formats. Penman–Monteith or Priestley–Taylor reference evapotranspiration is estimated, using both measured and estimated radiation. Utilities are provided to process numerous files, or correct possible constant biases in the data. Samples of data for tropical and temperate sites are supplied with the software.

irene: a software to evaluate model performance

Abstract The software irene (Integrated Resources for Evaluating Numerical Estimates) is a data analysis tool designed to provide easy access to statistical techniques for use in model evaluation. Mostly, non-replicated model estimates ( E i ) are compared against non-replicated measurements ( M i ). The software also allows comparing individual estimates against replicated measurements (or vice versa) and replicated estimates against replicated measurements. The evaluation of model performance is essentially based on the difference E i − M i , or on the correlation–regression of E i vs. M i (or vice versa). In addition, model evaluation by probability distributions, pattern analysis, or fuzzy-based aggregation statistics is allowed. Graphics are included in most analytical tasks. The results are displayed in separate spreadsheets and can be exported into MS Excel workbooks.

An integrated assessment approach to conduct analyses of climate change impacts on whole-farm systems

Abstract This paper argues that an integrated assessment (IA) approach, combining simulation modelling with deliberative processes involving decision makers and other stakeholders, has the potential to generate credible and relevant assessments of climate change impacts on farming systems. The justification for the approach proposed is that while simulation modelling provides an effective way of exploring the range of possible impacts of climate change and a means of testing the consequences of possible management or policy interventions, the interpretation of the outputs is highly dependent on the point of view of the stakeholder. Inevitably, whatever the responses to climate change, there will be trade-offs between the benefits and costs to a range of stakeholders. The use of a deliberative process that includes stakeholders, both in defining the topics addressed and in debating the interpretations of the outcomes, addresses many of the limitations that have been previously identified in the use of computer-based tools for agricultural decision support. The paper further argues that the concepts of resilience and adaptive capacity are useful for the assessment of climate change impacts as they provide an underpinning theory for processes of change in land use systems. The integrated modelling framework (IMF) developed for the simulation of whole-farm systems is detailed, including components for crop and soil processes, livestock systems and a tool for scheduling of resource use within management plans. The use of the IMF for assessing climate change impacts is then outlined to demonstrate the range of analyses possible. The paper concludes with a critique of the IA approach and notes that issues of quantification and communication of uncertainty are central to the success of the methodology.

Validation of Biophysical Models: Issues and Methodologies

The potential of mathematical models is widely acknowledged for examining components and interactions of natural systems, estimating the changes and uncertainties on outcomes, and fostering communication between scientists with different backgrounds and between scientists, managers and the community. For favourable reception of models, a systematic accrual of a good knowledge base is crucial for both science and decision-making. As the roles of models grow in importance, there is an increase in the need for appropriate methods with which to test their quality and performance. For biophysical models, the heterogeneity of data and the range of factors influencing usefulness of their outputs often make it difficult for full analysis and assessment. As a result, modelling studies in the domain of natural sciences often lack elements of good modelling practice related to model validation, that is correspondence of models to its intended purpose. Here we review validation issues and methods currently available for assessing the quality of biophysical models. The review covers issues of validation purpose, the robustness of model results, data quality, model prediction and model complexity. The importance of assessing input data quality and interpretation of phenomena is also addressed. Details are then provided on the range of measures commonly used for validation. Requirements for a methodology for assessment during the entire model-cycle are synthesised. Examples are used from a variety of modelling studies which mainly include agronomic modelling, e.g. crop growth and development, climatic modelling, e.g. climate scenarios, and hydrological modelling, e.g. soil hydrology, but the principles are essentially applicable to any area. It is shown that conducting detailed validation requires multi-faceted knowledge, and poses substantial scientific and technical challenges. Special emphasis is placed on using combined multiple statistics to expand our horizons in validation whilst also tailoring the validation requirements to the specific objectives of the application.

Short communication: A software component for estimating solar radiation

GSRad (global solar radiation) is a software component containing models to estimate extra-terrestrial and ground-level solar radiation (global and photosynthetically active; direct, diffuse, and reflected components) from alternative methods. Radiation data are estimated as either 1-h or 24-h values. Moreover, GSRad provides methods to compute clear sky transmissivity, slope and aspect angles of tilted terrains from a grid of elevation points, and geometric factors to convert radiation estimates from horizontal to non-horizontal surfaces. The component is released as .NET assemblies, allowing the development of clients under Windows operating systems using one of the .NET languages. The component design allows extending the computing capabilities of GSRad without requiring its re-compilation. Examples of clients developed in C# are provided as source code. The component is available for free download, along with an extensive hypertext help. help.

Technical assessment and evaluation of environmental models and software

This letter details the collective views of a number of independent researchers on the technical assessment and evaluation of environmental models and software. The purpose is to stimulate debate and initiate action that leads to an improved quality of model development and evaluation, so increasing the capacity for models to have positive outcomes from their use. As such, we emphasize the relationship between the model evaluation process and credibility with stakeholders (including funding agencies) with a view to ensure continued support for modelling efforts.Many journals, including EM&S, publish the results of environmental modelling studies and must judge the work and the submitted papers based solely on the material that the authors have chosen to present and on how they present it. There is considerable variation in how this is done with the consequent risk of considerable variation in the quality and usefulness of the resulting publication. Part of the problem is that the review process is reactive, responding to the submitted manuscript. In this letter, we attempt to be proactive and give guidelines for researchers, authors and reviewers as to what constitutes best practice in presenting environmental modelling results. This is a unique contribution to the organisation and practice of model-based research and the communication of its results that will benefit the entire environmental modelling community. For a start, our view is that the community of environmental modellers should have a common vision of minimum standards that an environmental model must meet. A common vision of what a good model should be is expressed in various guidelines on Good Modelling Practice. The guidelines prompt modellers to codify their practice and to be more rigorous in their model testing. Our statement within this letter deals with another aspect of the issue - it prompts professional journals to codify the peer-review process. Introducing a more formalized approach to peer-review may discourage reviewers from accepting invitations to review given the additional time and labour requirements. The burden of proving model credibility is thus shifted to the authors. Here we discuss how to reduce this burden by selecting realistic evaluation criteria and conclude by advocating the use of standardized evaluation tools as this is a key issue that needs to be tackled.

Adaptation assessments for crop production in response to climate change in Cameroon

The Cameroonian agricultural sector, a critical part of the local ecosystem, is potentially vulnerable to climate change, thus raising concerns about food security in the country’s future. Adaptations policies may be able to mitigate some of this vulnerability. This article addresses the issue of selected adaptation options within the context of Cameroonian food production. A methodology is applied where transient diagnostics of two atmosphere-ocean general circulation models, the NASA/Goddard Institute GISS and the British HadCM3, are coupled to a cropping system simulation model (CropSyst). This methodology simulates current and future (2020, 2080) crop yields for selected key crops such as bambara nut, groundnut, maize, sorghum, and soybean, in eight agricultural regions of Cameroon. Our results show that for the future, substantial yield increases are estimated for bambara groundnut, soybean and groundnut, while little or no change or even decreases for maize and sorghum yields, varying according to the climate scenario and the agricultural region investigated. Taking the “no regrets” principle into consideration, we also explore the advantages of specific adaptation strategies specifically for three crops, maize, sorghum and bambara groundnut, under GISS A2 and B2 marker scenarios only. Here, changing sowing dates may be ineffective in counteracting adverse climatic effects because of the narrow rainfall band that strictly determines the timing of farm operations in Cameroon. In contrast, the possibility of developing later maturing new cultivars proved to be very effective in offsetting adverse impacts, giving the highest increases in productivity under different scenario projections without management changes. For example, under climate change scenario GISS A2 2080, a 14.6% reduction in maize yield was converted to a 32.1% increase; a 39.9% decrease in sorghum yield was converted to a 17.6% increase, and for bambara groundnut, yields were almost trebled due to increase length of growing period and the positive effects of higher CO2 concentrations. These results better inform wider studies and development strategies on sustainable agriculture in the area by providing an indication as to the potential direction in shifts in production capabilities. Our approach highlights the benefit of using models as tools to investigate potential climate change impacts, where results can supplement existing knowledge. The findings also provide useful guidance and motivation to public authorities and development agencies interested in food security issues in Cameroon and elsewhere.

Empirical Models of Macronutrient Uptake in Melon Plants Grown in Recirculating Nutrient Solution Culture

Abstract The article presents a number of empirical models for predicting the macronutrient uptake of melon plants grown in nutrient film technique under environmental conditions of plastic greenhouse in the Mediterranean region. Models were developed according to two statistical procedures: stepwise multiple regression (MR) and canonical correlation (CC). Independent variables considered by the modeling were global radiation and air temperature in the greenhouse, crop age (expressed as number of weeks from planting, growing degree days and photo-thermal units), and the uptake of water as well as of a guide-ion that could be routinely measured manually by means of easy-to-use test-kits or automatically with chemo-sensors. The best models, as selected on the basis of determination coefficient and the correlation coefficient for the relationship between residuals and observations, explained only 36–72% of the variance in the mineral uptake, depending on the considered nutrient. Moreover, the models were conservative, as predicted values tended to be less extreme with respect to the observed values and the residuals were positively correlated to the observations. The results of MR and CC were similar, although the validation of the models derived from CC produced better results compared to MR. The models provided evidence for the close relationship between ion and water uptake and indicated the possibility to predict the crop mineral requirements on the basis of the consumption of a guide-ion (i.e., nitrogen).

Use of pJANUS-02-001 as a calibrator plasmid for Roundup Ready soybean event GTS-40-3-2 detection: an interlaboratory trial assessment.

Owing to the labelling requirements of food and feed products containing materials derived from genetically modified organisms, quantitative detection methods have to be developed for this purpose, including the necessary certified reference materials and calibrator standards. To date, for most genetically modified organisms authorized in the European Union, certified reference materials derived from seed powders are being developed. Here, an assessment has been made on the feasibility of using plasmid DNA as an alternative calibrator for the quantitative detection of genetically modified organisms. For this, a dual-target plasmid, designated as pJANUS™-02-001, comprising part of a junction region of genetically modified soybean event GTS-40-3-2 and the endogenous soybean-specific lectin gene was constructed. The dynamic range, efficiency and limit of detection for the soybean event GTS-40-3-2 real-time quantitative polymerase chain reaction (Q-PCR) system described by Terry et al. (J AOAC Int 85(4):938–944, 2002) were shown to be similar for in house produced homozygous genomic DNA from leaf tissue of soybean event GTS-40-3-2 and for plasmid pJANUS™-02-001 DNA backgrounds. The performance of this real-time Q-PCR system using both types of DNA templates as calibrator standards in quantitative DNA analysis was further assessed in an interlaboratory trial. Statistical analysis and fuzzy-logic-based interpretation were performed on critical method parameters (as defined by the European Network of GMO Laboratories and the Community Reference Laboratory for GM Food and Feed guidelines) and demonstrated that the plasmid pJANUS™-02-001 DNA represents a valuable alternative to genomic DNA as a calibrator for the quantification of soybean event GTS-40-3-2 in food and feed products.