Gab Abramowitz

Evaluating the Performance of Land Surface Models

Abstract This paper presents a set of analytical tools to evaluate the performance of three land surface models (LSMs) that are used in global climate models (GCMs). Predictions of the fluxes of sensible heat, latent heat, and net CO2 exchange obtained using process-based LSMs are benchmarked against two statistical models that only use incoming solar radiation, air temperature, and specific humidity as inputs to predict the fluxes. Both are then compared to measured fluxes at several flux stations located on three continents. Parameter sets used for the LSMs include default values used in GCMs for the plant functional type and soil type surrounding each flux station, locally calibrated values, and ensemble sets encompassing combinations of parameters within their respective uncertainty ranges. Performance of the LSMs is found to be generally inferior to that of the statistical models across a wide variety of performance metrics, suggesting that the LSMs underutilize the meteorological information used in...

Are we unnecessarily constraining the agility of complex process-based models?

In this commentary we suggest that hydrologists and land-surface modelers may be unnecessarily constraining the behavioral agility of very complex physics-based models. We argue that the relatively poor performance of such models can occur due to restrictions on their ability to refine their portrayal of physical processes, in part because of strong a priori constraints in: (i) the representation of spatial variability and hydrologic connectivity, (ii) the choice of model parameterizations, and (iii) the choice of model parameter values. We provide a specific example of problems associated with strong a priori constraints on parameters in a land surface model. Moving forward, we assert that improving hydrological models requires integrating the strengths of the “physics-based” modeling philosophy (which relies on prior knowledge of hydrologic processes) with the strengths of the “conceptual” modeling philosophy (which relies on data driven inference). Such integration will accelerate progress on methods to define and discriminate among competing modeling options, which should be ideally incorporated in agile modeling frameworks and tested through a diagnostic evaluation approach.

Climate model dependence and the replicate Earth paradigm

Multi-model ensembles are commonly used in climate prediction to create a set of independent estimates, and so better gauge the likelihood of particular outcomes and better quantify prediction uncertainty. Yet researchers share literature, datasets and model code—to what extent do different simulations constitute independent estimates? What is the relationship between model performance and independence? We show that error correlation provides a natural empirical basis for defining model dependence and derive a weighting strategy that accounts for dependence in experiments where the multi-model mean would otherwise be used. We introduce the “replicate Earth” ensemble interpretation framework, based on theoretically derived statistical relationships between ensembles of perfect models (replicate Earths) and observations. We transform an ensemble of (imperfect) climate projections into an ensemble whose mean and variance have the same statistical relationship to observations as an ensemble of replicate Earths. The approach can be used with multi-model ensembles that have varying numbers of simulations from different models, accounting for model dependence. We use HadCRUT3 data and the CMIP3 models to show that in out of sample tests, the transformed ensemble has an ensemble mean with significantly lower error and much flatter rank frequency histograms than the original ensemble.

Systematic Bias in Land Surface Models

A neural network–based flux correction technique is applied to three land surface models. It is then used to show that the nature of systematic model error in simulations of latent heat, sensible heat, and the net ecosystem exchange of CO2 is shared between different vegetation types and indeed different models .B y manipulating the relationship between the dataset used to train the correction technique and that used to test it, it is shown that as much as 45% of per-time-step model root-mean-square error in these flux outputs is due to systematic problems in those model processes insensitive to changes in vegetation parameters. This is shown in the three land surface models using flux tower measurements from 13 sites spanning 2 vegetation types. These results suggest that efforts to improve the representation of fundamental processes in land surface models, rather than parameter optimization, are the key to the development of land surface model ability.

Optimally choosing small ensemble members to produce robust climate simulations

This study examines the subset climate model ensemble size required to reproduce certain statistical characteristics from a full ensemble. The ensemble characteristics examined are the root mean square error, the ensemble mean and standard deviation. Subset ensembles are created using measures that consider the simulation performance alone or include a measure of simulation independence relative to other ensemble members. It is found that the independence measure is able to identify smaller subset ensembles that retain the desired full ensemble characteristics than either of the performance based measures. It is suggested that model independence be considered when choosing ensemble subsets or creating new ensembles.

Neural Error Regression Diagnosis (NERD): A Tool for Model Bias Identification and Prognostic Data Assimilation

Abstract Data assimilation in the field of predictive land surface modeling is generally limited to using observational data to estimate optimal model states or restrict model parameter ranges. To date, very little work has attempted to systematically define and quantify error resulting from a models inherent inability to simulate the natural system. This paper introduces a data assimilation technique that moves toward this goal by accounting for those deficiencies in the model itself that lead to systematic errors in model output. This is done using a supervised artificial neural network to “learn” and simulate systematic trends in the model output error. These simulations in turn are used to correct the models output each time step. The technique is applied in two case studies, using fluxes of latent heat flux at one site and net ecosystem exchange (NEE) of carbon dioxide at another. Root-mean-square error (rmse) in latent heat flux per time step was reduced from 27.5 to 18.6 W m−2 (32%) and monthly f...

Climate Model Dependence and the Ensemble Dependence Transformation of CMIP Projections

AbstractObtaining multiple estimates of future climate for a given emissions scenario is key to understanding the likelihood and uncertainty associated with climate-related impacts. This is typically done by collating model estimates from different research institutions internationally with the assumption that they constitute independent samples. Heuristically, however, several factors undermine this assumption: shared treatment of processes between models, shared observed data for evaluation, and even shared model code. Here, a “perfect model” approach is used to test whether a previously proposed ensemble dependence transformation (EDT) can improve twenty-first-century Coupled Model Intercomparison Project (CMIP) projections. In these tests, where twenty-first-century model simulations are used as out-of-sample “observations,” the mean-square difference between the transformed ensemble mean and “observations” is on average 30% less than for the untransformed ensemble mean. In addition, the variance of t...

Which are the most important parameters for modelling carbon assimilation in boreal Norway spruce under elevated [CO2] and temperature conditions?

Photosynthesis is highly responsive to environmental and physiological variables, including phenology, foliage nitrogen (N) content, atmospheric CO2 concentration ([CO2]), irradiation (Q), air temperature (T) and vapour pressure deficit (D). Each of these responses is likely to be modified by long-term changes in climatic conditions such as rising air temperature and [CO2]. When modelling photosynthesis under climatic changes, which parameters are then most important to calibrate for future conditions? To assess this, we used measurements of shoot carbon assimilation rates and microclimate conditions collected at Flakaliden, northern Sweden. Twelve 40-year-old Norway spruce trees were enclosed in whole-tree chambers and exposed to elevated [CO2] and elevated air temperature, separately and in combination. The treatments imposed were elevated temperature, +2.8 °C in July/August and +5.6 °C in December above ambient, and [CO2] (ambient CO2 ∼370 μ mol mol(-1), elevated CO2 ∼700 μ mol mol(-1)). The relative importance of parameterization of Q, T and D responses for effects on the photosynthetic rate, expressed on a projected needle area, and the annual shoot carbon uptake was quantified using an empirical shoot photosynthesis model, which was developed and fitted to the measurements. The functional form of the response curves was established using an artificial neural network. The [CO2] treatment increased annual shoot carbon (C) uptake by 50%. Most important was effects on the light response curve, with a 67% increase in light-saturated photosynthetic rate, and a 52% increase in the initial slope of the light response curve. An interactive effect of light saturated photosynthetic rate was found with foliage N status, but no interactive effect for high temperature and high CO2. The air temperature treatment increased the annual shoot C uptake by 44%. The most important parameter was the seasonality, with an elongation of the growing season by almost 4 weeks. The temperature response curve was almost flat over much of the temperature range. A shift in temperature optimum had thus an insignificant effect on modelled annual shoot C uptake. The combined temperature and [CO2] treatment resulted in a 74% increase in annual shoot C uptake compared with ambient conditions, with no clear interactive effects on parameter values.

Land surface models systematically overestimate the intensity, duration and magnitude of seasonal-scale evaporative droughts

Land surfacemodels (LSMs)must accurately simulate observed energy andwater fluxes during droughts in order to provide reliable estimates of futurewater resources.We evaluated 8 different LSMs (14model versions) for simulating evapotranspiration (ET) during periods of evaporative drought (Edrought) across sixflux tower sites. Using an empirically defined Edrought threshold (a decline in ET below the observed 15th percentile), we show that LSMs simulated 58 Edrought days per year, on average, across the six sites,∼3 times asmany as the observed 20 d. The simulated Edrought magnitudewas∼8 times greater than observed and twice as intense. Ourfindings point to systematic biases across LSMswhen simulating water and energy fluxes underwater-stressed conditions. The overestimation of key Edrought characteristics undermines our confidence in themodels’ capability in simulating realistic drought responses to climate change and haswider implications for phenomena sensitive to soilmoisture, including heat waves.

Influence of Leaf Area Index Prescriptions on Simulations of Heat, Moisture, and Carbon Fluxes

AbstractLeaf area index (LAI), the total one-sided surface area of leaf per ground surface area, is a key component of land surface models. The authors investigate the influence of differing, plausible LAI prescriptions on heat, moisture, and carbon fluxes simulated by the Community Atmosphere Biosphere Land Exchange version 1.4b (CABLEv1.4b) model over the Australian continent. A 15-member ensemble monthly LAI dataset is generated using the Moderate Resolution Imaging Spectroradiometer (MODIS) LAI product and gridded observations of temperature and precipitation. Offline simulations lasting 29 years (1980–2008) are carried out at 25-km resolution with the composite monthly means from the MODIS LAI product (control simulation) and compared with simulations using each of the 15-member ensemble monthly varying LAI datasets generated. The imposed changes in LAI did not strongly influence the sensible and latent fluxes, but the carbon fluxes were more strongly affected. Croplands showed the largest sensitivit...