Cort J. Willmott

ON THE VALIDATION OF MODELS

Traditional methods of evaluating geographic models by statistical comparisons between observed and simulated variates are criticized. In particular, it is suggested that the correlation coefficient (r), its square and tests of their statistical significance are inadequate for such purposes. The root mean squared error (RMSE) and related measures as well as a new index of agreement (d) are alternatively presented as superior indices for making such comparisons. Arguments are made for increasing the number of digital algorithms and data plots being published.

Some Comments on the Evaluation of Model Performance

Quantitative approaches to the evaluation of model performance were recently examined by Fox (1981). His recommendations are briefly reviewed and a revised set of performance statistics is proposed. It is suggested that the correlation between model-predicted and observed data, commonly described by Pearsons product-moment correlation coefficient, is an insufficient and often misleading measure of accuracy. A complement of difference and summary univariate indices is presented as the nucleus of a more informative, albeit fundamentally descriptive, approach to model evaluation. Two models that estimate monthly evapotranspiration are comparatively evaluated in order to illustrate how the recommended method(s) can be applied.

Uncertainties in Precipitation and Their Impacts on Runoff Estimates

Water balance calculations are becoming increasingly important for earth-system studies. Precipitation is one of the most critical input variables for such calculations because it is the immediate source of water for the land surface hydrological budget. Numerous precipitation datasets have been developed in the last two decades, but these datasets often show marked differences in their spatial and temporal distribution of this key hydrological variable. This paper compares six monthly precipitation datasets—Climate Research Unit of University of East Anglia (CRU), Willmott‐Matsuura (WM), Global Precipitation Climate Center (GPCC), Global Precipitation Climatology Project (GPCP), Tropical Rainfall Measuring Mission (TRMM), and NCEP‐Department of Energy (DOE) Atmospheric Model Intercomparison Project (AMIP-II) Reanalysis (NCEP-2)—to assess the uncertainties in these datasets and their impact on the terrestrial water balance. The six datasets tested in the present paper were climatologically averaged and compared by calculating various statistics of the differences. The climatologically averaged monthly precipitation estimates were applied as inputs to a water balance model to estimate runoff and the uncertainties in runoff arising directly from the precipitation estimates. The results of this study highlight the need for accurate precipitation inputs for water balance calculations. These results also demonstrate the need to improve precipitation estimates in arid and semiarid regions, where slight changes in precipitation can result in dramatic changes in the runoff response due to the nonlinearity of the runoff-generation processes.

On the Evaluation of Model Performance in Physical Geography

Following the lead of Haggett and Chorley (1967), Rayner (1974), Terjung (1976), Strahler (1980) and many others, physical geography has adopted a “model-based paradigm” and, as a result, the development and application of a wide variety of models is now commonplace within virtually every sub-field from geomorphology to bioclimatology. Within climatology, many models have a predominately deductive genesis while other models are collages of statistical and empirical reasoning and, in a few cases, “best-fit” functions are extracted from data with seemingly little regard for the safeguards of a deductive stance. Still other models combine the mathematics of probability theory with empirically derived probabilities to create stochastic simulation models, e.g., Markov or Monte Carlo models. These categories of models are, by no means, mutually exclusive (or exhaustive for that matter) and a number of recent models may be considered combinatorial in that they incorporate two or more of the above-mentioned strategies into a single model.

Spatial statistics and models

The quantitative revolution in geography has passed. The spirited debates of the past decades have, in one sense, been resolved by the inclusion of quantitative techniques into the typical geographers set of methodological tools. A new decade is upon us. Throughout the quantitative revolution, geographers ransacked related disciplines and mathematics in order to find tools which might be applicable to problems of a spatial nature. The early success of Berry and Marbles Spatial Analysis and Garrison and Marbles volumes on Quantitative Geog- raphy is testimony to their accomplished search. New developments often depend heavily on borrowed ideas. It is only after these developments have been established that the necessary groundwork for true innovation ob- tains. In the last decade, geographers significantly -augmented their methodologi- cal base by developing quantitative techniques which are specifically directed towards analysis of explicitly spatial problems. It should be pointed out, however, that the explicit incorporation of space into quantitative techniques has not been the sole domain of geographers. Mathematicians, geologists, meteorologists, economists, and regional scientists have shared the geo- graphers interest in the spatial component of their analytical tools.

Analyzing the discharge regime of a large tropical river through remote sensing, ground‐based climatic data, and modeling

This study demonstrates the potential for applying passive microwave satellite sensor data to infer the discharge dynamics of large river systems using the main stem Amazon as a test case. The methodology combines (1) interpolated ground-based meteorological station data, (2) horizontally and vertically polarized temperature differences (HVPTD) from the 37-GHz scanning multichannel microwave radiometer (SMMR) aboard the Nimbus 7 satellite, and (3) a calibrated water balance/water transport model (WBM/WTM). Monthly HVPTD values at 0.25° (latitude by longitude) resolution were resampled spatially and temporally to produce an enhanced HVPTD time series at 0.5° resolution for the period May 1979 through February 1985. Enhanced HVPTD values were regressed against monthly discharge derived from the WBM/WTM for each of 40 grid cells along the main stem over a calibration period from May 1979 to February 1983 to provide a spatially contiguous estimate of time-varying discharge. HVPTD-estimated flows generated for a validation period from March 1983 to February 1985 were found to be in good agreement with both observed arid modeled discharges over a 1400-km section of the main stem Amazon. This span of river is bounded downstream by a region of tidal influence and upstream by low sensor response associated with dense forest canopy. Both the WBM/WTM and HVPTD-derived flow rates reflect the significant impact of the 1982–1983 El Nino-;Southern Oscillation (ENSO) event on water balances within the drainage basin.

An Empirical Method for the Spatial Interpolation of Monthly Precipitation within California

An empirical method for interpolating monthly precipitation totals within California is described and evaluated. Using 120 monthly precipitation totals observed from 1961-1970 at each of 90 randomly selected stations in California and a P-mode principal components analysis of a co-variance matrix, four independent sources of precipitation variability were identified and quantitatively paraphrased. The four principal components were then linked to three representative stations by polynomial regression. From these relationships, monthly precipitation totals can be interpolated anywhere in the state by reversing the principal components computations. The required input includes: a monthly precipitation total, for the month of interest, from each of the three representative stations as well as isarithmically interpolated estimates of the component loadings and station means which were derived from the initial (1961-1970) data set. A major asset of the procedure is that it only requires three pieces of new inf...

GLOBAL DISTRIBUTION OF PLANT-EXTRACTABLE WATER CAPACITY OF SOIL

Plant-extractable water capacity of soil is the amount of water that can be extracted from the soil to fulfill evapotranspiration demands. It is often assumed to be spatially invariant in large-scale computations of the soil-water balance. Empirical evidence, however, suggests that this assumption is incorrect. In this paper, we estimate the global distribution of the plant-extractable water capacity of soil. A representative soil profile, characterized by horizon (layer) particle size data and thickness, was created for each soil unit mapped by FAO (Food and Agriculture Organization of the United Nations)/Unesco. Soil organic matter was estimated empirically from climate data. Plant rooting depths and ground coverages were obtained from a vegetation characteristic data set. At each 0.5°×0.5° grid cell where vegetation is present, unit available water capacity (cm water per cm soil) was estimated from the sand, clay, and organic content of each profile horizon, and integrated over horizon thickness. Summation of the integrated values over the lesser of profile depth and root depth produced an estimate of the plant-extractable water capacity of soil. The global average of the estimated plant-extractable water capacities of soil is 8ċ6cm (Greenland, Antarctica and bare soil areas excluded). Estimates are less than 5, 10 and 15 cm—over approximately 30, 60, and 89 per cent of the area, respectively. Estimates reflect the combined effects of soil texture, soil organic content, and plant root depth or profile depth. The most influential and uncertain parameter is the depth over which the plant- extractable water capacity of soil is computed, which is usually limited by root depth. Soil texture exerts a lesser, but still substantial, influence. Organic content, except where concentrations are very high, has relatively little effect.

On the use of dimensioned measures of error to evaluate the performance of spatial interpolators

Spatial cross‐validation and average‐error statistics are examined with respect to their abilities to evaluate alternate spatial interpolation methods. A simple cross‐validation methodology is described, and the relative abilities of three, dimensioned error statistics—the root‐mean‐square error (RMSE), the mean absolute error (MAE), and the mean bias error (MBE)—to describe average interpolator performance are examined. To illustrate our points, climatologically averaged weather‐station temperatures were obtained from the Global Historical Climatology Network (GHCN), Version 2, and then alternately interpolated spatially (gridded) using two spatial‐interpolation procedures. Substantial differences in the performance of our two spatial interpolators are evident in maps of the cross‐validation error fields, in the average‐error statistics, as well as in estimated land‐surface‐average air temperatures that differ by more than 2°C. The RMSE and its square, the mean‐square error (MSE), are of particular interest, because they are the most widely reported average‐error measures, and they tend to be misleading. It (RMSE) is an inappropriate measure of average error because it is a function of three characteristics of a set of errors, rather than of one (the average error). Our findings indicate that MAE and MBE are natural measures of average error and that (unlike RMSE) they are unambiguous.

On the climatic optimization of the tilt and azimuth of flat-plate solar collectors

Abstract A numerical climatic model for computing total solar irradiance on the surface of a flat-plate collector, positioned at any tilt and azimuth, is described. Owing to a small time-step (one hour), and a quasi-realistic characterization of a collectors environment, the algorithm is able to produce credible estimates of both the climatically “optimal” position and the amount of energy lost to a collector when it is non-optimally positioned. Exemplary computations for Sterling, Virginia and Sunnyvale, California are presented and they suggest that the non-optimal positioning of a collector, e.g. as a simple function of latitude and a few highly summarized climatic-environmental variables, will not, in many cases, produce significant losses of available solar irradiance. In other situations, however, where a collectors horizon is significantly obstructed and/or the climatic environment of the area creates large diurnal or seasonal asymmetries in available irradiance, non-optimal positioning may cause sizeable energy losses. It is also apparent that even moderately sized horizonal obstructions, which are “seen” by a collector, can substantially reduce the amount of available irradiance, relative to an unobstructed horizon.