Tapio Schneider

Analysis of Incomplete Climate Data: Estimation of Mean Values and Covariance Matrices and Imputation of Missing Values

Estimating the mean and the covariance matrix of an incomplete dataset and filling in missing values with imputed values is generally a nonlinear problem, which must be solved iteratively. The expectation maximization (EM) algorithm for Gaussian data, an iterative method both for the estimation of mean values and covariance matrices from incomplete datasets and for the imputation of missing values, is taken as the point of departure for the development of a regularized EM algorithm. In contrast to the conventional EM algorithm, the regularized EM algorithm is applicable to sets of climate data, in which the number of variables typically exceeds the sample size. The regularized EM algorithm is based on iterated analyses of linear regressions of variables with missing values on variables with available values, with regression coefficients estimated by ridge regression, a regularized regression method in which a continuous regularization parameter controls the filtering of the noise in the data. The regularization parameter is determined by generalized cross-validation, such as to minimize, approximately, the expected mean-squared error of the imputed values. The regularized EM algorithm can estimate, and exploit for the imputation of missing values, both synchronic and diachronic covariance matrices, which may contain information on spatial covariability, stationary temporal covariability, or cyclostationary temporal covariability. A test of the regularized EM algorithm with simulated surface temperature data demonstrates that the algorithm is applicable to typical sets of climate data and that it leads to more accurate estimates of the missing values than a conventional noniterative imputation technique.

The physical basis for increases in precipitation extremes in simulations of 21st-century climate change.

Global warming is expected to lead to a large increase in atmospheric water vapor content and to changes in the hydrological cycle, which include an intensification of precipitation extremes. The intensity of precipitation extremes is widely held to increase proportionately to the increase in atmospheric water vapor content. Here, we show that this is not the case in 21st-century climate change scenarios simulated with climate models. In the tropics, precipitation extremes are not simulated reliably and do not change consistently among climate models; in the extratropics, they consistently increase more slowly than atmospheric water vapor content. We give a physical basis for how precipitation extremes change with climate and show that their changes depend on changes in the moist-adiabatic temperature lapse rate, in the upward velocity, and in the temperature when precipitation extremes occur. For the tropics, the theory suggests that improving the simulation of upward velocities in climate models is essential for improving predictions of precipitation extremes; for the extratropics, agreement with theory and the consistency among climate models increase confidence in the robustness of predictions of precipitation extremes under climate change.

Estimation of parameters and eigenmodes of multivariate autoregressive models

Dynamical characteristics of a complex system can often be inferred from analysis of a stochastic time series model fitted to observations of the system. Oscillations in geophysical systems, for example, are sometimes characterized by principal oscillation patterns, eigenmodes of estimated autoregressive (AR) models of first order. This paper describes the estimation of eigenmodes of AR models of arbitrary order. AR processes of any order can be decomposed into eigenmodes with characteristic oscillation periods, damping times, and excitations. Estimated eigenmodes and confidence intervals for the eigenmodes and their oscillation periods and damping times can be computed from estimated models parameters. As a computationally efficient method of estimating the parameters of AR models from high-dimensional data, a stepwise least squares algorithm is proposed. This algorithm computes models of successively decreasing order. Numerical simulations indicate that, with the least squares algorithm, the AR model coefficients and the eigenmodes derived from the coefficients and eigenmodes are rough approximations of the confidence intervals inferred from the simulaitons.

Migrations and dynamics of the intertropical convergence zone

Rainfall on Earth is most intense in the intertropical convergence zone (ITCZ), a narrow belt of clouds centred on average around six degrees north of the Equator. The mean position of the ITCZ north of the Equator arises primarily because the Atlantic Ocean transports energy northward across the Equator, rendering the Northern Hemisphere warmer than the Southern Hemisphere. On seasonal and longer timescales, the ITCZ migrates, typically towards a warming hemisphere but with exceptions, such as during El Niño events. An emerging framework links the ITCZ to the atmospheric energy balance and may account for ITCZ variations on timescales from years to geological epochs.

Algorithm 808: ARfit—a matlab package for the estimation of parameters and eigenmodes of multivariate autoregressive models

ARfit is a collection of Matlab modules for modeling and analyzing multivariate time series with autoregressive (AR) models. ARfit contains modules to given time series data, for analyzing eigen modes of a fitted model, and for simulating AR processes. ARfit estimates the parameters of AR models from given time series data with a stepwise least squares algorithm that is computationally efficient, in particular when the data are high-dimensional. ARfit modules construct approximate confidence intervals for the estimated parameters and compute statistics with which the adequacy of a fitted model can be assessed. Dynamical characteristics of the modeled time series can be examined by means of a decomposition of a fitted AR model into eigenmodes and associated oscillation periods, damping times, and excitations. The ARfit module that performs the eigendecomposition of a fitted model also constructs approximate confidence intervals for the eigenmodes and their oscillation periods and damping times.

Eddy Influences on Hadley Circulations: Simulations with an Idealized GCM

An idealized GCM is used to investigate how the strength and meridional extent of the Hadley circulation depend on the planet radius, rotation rate, and thermal driving. Over wide parameter ranges, the strength and meridional extent of the Hadley circulation display clear scaling relations with regime transitions, which are not predicted by existing theories of axisymmetric Hadley circulations. For example, the scaling of the strength as a function of the radiative-equilibrium equator-to-pole temperature contrast exhibits a regime transition corresponding to a regime transition in scaling laws of baroclinic eddy fluxes. The scaling of the strength of the cross-equatorial Hadley cell as a function of the latitude of maximum radiative-equilibrium temperature exhibits a regime transition from a regime in which eddy momentum fluxes strongly influence the strength to a regime in which the influence of eddy momentum fluxes is weak. Over a wide range of flow parameters, albeit not always, the Hadley circulation strength is directly related to the eddy momentum flux divergence at the latitude of the streamfunction extremum. Simulations with hemispherically symmetric thermal driving span circulations with local Rossby numbers in the horizontal upper branch of the Hadley circulation between 0.1 and 0.8, indicating that neither nonlinear nearly inviscid theories, valid for Ro → 1, nor linear theories, valid for Ro → 0, of axisymmetric Hadley circulations can be expected to be generally adequate. Nonlinear theories of axisymmetric Hadley circulations may account for aspects of the circulation when the maximum radiative-equilibrium temperature is displaced sufficiently far away from the equator, which results in cross-equatorial Hadley cells with nearly angular momentumconserving upper branches. The dependence of the Hadley circulation on eddy fluxes, which are themselves dependent on extratropical circulation characteristics such as meridional temperature gradients, suggests that tropical circulations depend on the extratropical climate.

The Hydrological Cycle over a Wide Range of Climates Simulated with an Idealized GCM

A wide range of hydrological cycles and general circulations was simulated with an idealized general circulation model (GCM) by varying the optical thickness of the longwave absorber. While the idealized GCM does not capture the full complexity of the hydrological cycle, the wide range of climates simulated allows the systematic development and testing of theories of how precipitation and moisture transport change as the climate changes. The simulations show that the character of the response of the hydrological cycle to variations in longwave optical thickness differs in different climate regimes. The global-mean precipitation increases linearly with surface temperature for colder climates, but it asymptotically approaches a maximum at higher surface temperatures. The basic features of the precipitation–temperature relation, including the rate of increase in the linear regime, are reproduced in radiative– convective equilibrium simulations. Energy constraints partially account for the precipitation–temperature relation but are not quantitatively accurate. Large-scale condensation is most important in the midlatitude storm tracks, and its behavior is accounted for using a stochastic model of moisture advection and condensation. The precipitation associated with large-scale condensation does not scale with mean specific humidity, partly because the condensation region moves upward and meridionally as the climate warms, and partly because the mean condensation rate depends on isentropic specific humidity gradients, which do not scale with the specific humidity itself. The local water vapor budget relates local precipitation to evaporation and meridional moisture fluxes, whose scaling in the subtropics and extratropics is examined. A delicate balance between opposing changes in evaporation and moisture flux divergence holds in the subtropical dry zones. The extratropical precipitation maximum follows the storm track in warm climates but lies equatorward of the storm track in cold climates.

A Conceptual Framework for Predictability Studies

A conceptual framework is presented for a unified treatment of issues arising in a variety of predictability studies. The predictive power (PP), a predictability measure based on information‐theoretical principles, lies at the center of this framework. The PP is invariant under linear coordinate transformations and applies to multivariate predictions irrespective of assumptions about the probability distribution of prediction errors. For univariate Gaussian predictions, the PP reduces to conventional predictability measures that are based upon the ratio of the rms error of a model prediction over the rms error of the climatological mean prediction. Since climatic variability on intraseasonal to interdecadal timescales follows an approximately Gaussian distribution, the emphasis of this paper is on multivariate Gaussian random variables. Predictable and unpredictable components of multivariate Gaussian systems can be distinguished by predictable component analysis, a procedure derived from discriminant analysis: seeking components with large PP leads to an eigenvalue problem, whose solution yields uncorrelated components that are ordered by PP from largest to smallest. In a discussion of the application of the PP and the predictable component analysis in different types of predictability studies, studies are considered that use either ensemble integrations of numerical models or autoregressive models fitted to observed or simulated data. An investigation of simulated multidecadal variability of the North Atlantic illustrates the proposed methodology. Reanalyzing an ensemble of integrations of the Geophysical Fluid Dynamics Laboratory coupled general circulation model confirms and refines earlier findings. With an autoregressive model fitted to a single integration of the same model, it is demonstrated that similar conclusions can be reached without resorting to computationally costly ensemble integrations.

The Surface Branch of the Zonally Averaged Mass Transport Circulation in the Troposphere

The near-surface branch of the overturning mass transport circulation in the troposphere, containing the equatorward flow, is examined in isentropic and geometric coordinates. A discussion of the zonal momentum balance within isentropic layers shows that the equatorward flow at a given latitude is confined to isentropic layers that typically intersect the surface at that latitude. As a consequence of mass transport within the surface mixed layer, much of the equatorward flow occurs in layers with potential temperatures below the mean surface potential temperature. In the conventional transformed Eulerian mean formulation for geometric coordinates, the surface branch of the overturning circulation is represented in an unrealistic manner: streamlines of the residual circulation do not close above the surface. A modified residual circulation is introduced that is free from this defect and has the additional advantage that its computation, unlike that of the conventional residual circulation, does not require division by the static stability, which may approach zero in the planetary boundary layer. It is then argued that cold air advection by the residual circulation is responsible for the formation of surface inversions at all latitudes in idealized GCMs with weak thermal damping. Also included is a discussion of how a general circulation theory for the troposphere must be built upon a theory for the near-surface meridional mass fluxes.

The tropopause and the thermal stratification in the extratropics of a dry atmosphere

A dynamical constraint on the extratropical tropopause height and thermal stratification is derived by considerations of entropy fluxes, or isentropic mass fluxes, and their different magnitudes in the troposphere and stratosphere. The dynamical constraint is based on a relation between isentropic mass fluxes and eddy fluxes of potential vorticity and surface potential temperature and on diffusive eddy flux closures. It takes baroclinic eddy fluxes as central for determining the extratropical tropopause height and thermal stratification and relates the tropopause potential temperature approximately linearly to the surface potential temperature and its gradient. Simulations with an idealized GCM point to the possibility of an extratropical climate in which baroclinic eddy fluxes maintain a statically stable thermal stratification and, in interaction with large-scale diabatic processes, lead to the formation of a sharp tropopause. The simulations show that the extratropical tropopause height and thermal stratification are set locally by extratropical processes and do not depend on tropical processes and that, across a wide range of atmospheric circulations, the dynamical constraint describes the relation between tropopause and surface potential temperatures well. An analysis of observational data shows that the dynamical constraint, derived for an idealized dry atmosphere, can account for interannual variations of the tropopause height and thermal stratification in the extratropics of the earths atmosphere. The dynamical constraint implies that if baroclinic eddies determine the tropopause height and thermal stratification, an atmosphere organizes itself into a state in which nonlinear interactions among eddies are inhibited. The inhibition of nonlinear eddy–eddy interactions offers an explanation for the historic successes of linear and weakly nonlinear models of large-scale extratropical dynamics.