Xiaolan L. Wang

A Review and Comparison of Changepoint Detection Techniques for Climate Data

Abstract This review article enumerates, categorizes, and compares many of the methods that have been proposed to detect undocumented changepoints in climate data series. The methods examined include the standard normal homogeneity (SNH) test, Wilcoxon’s nonparametric test, two-phase regression (TPR) procedures, inhomogeneity tests, information criteria procedures, and various variants thereof. All of these methods have been proposed in the climate literature to detect undocumented changepoints, but heretofore there has been little formal comparison of the techniques on either real or simulated climate series. This study seeks to unify the topic, showing clearly the fundamental differences among the assumptions made by each procedure and providing guidelines for which procedures work best in different situations. It is shown that the common trend TPR and Sawa’s Bayes criteria procedures seem optimal for most climate time series, whereas the SNH procedure and its nonparametric variant are probably best whe...

Accounting for Autocorrelation in Detecting Mean Shifts in Climate Data Series Using the Penalized Maximal t or F Test

Abstract This study proposes an empirical approach to account for lag-1 autocorrelation in detecting mean shifts in time series of white or red (first-order autoregressive) Gaussian noise using the penalized maximal t test or the penalized maximal F test. This empirical approach is embedded in a stepwise testing algorithm, so that the new algorithms can be used to detect single or multiple changepoints in a time series. The detection power of the new algorithms is analyzed through Monte Carlo simulations. It has been shown that the new algorithms work very well and fast in detecting single or multiple changepoints. Examples of their application to real climate data series (surface pressure and wind speed) are presented. An open-source software package (in R and FORTRAN) for implementing the algorithms, along with a user manual, has been developed and made available online free of charge.

Climatology and Changes of Extratropical Cyclone Activity: Comparison of ERA-40 with NCEP–NCAR Reanalysis for 1958–2001

Abstract In this study, a cyclone detection/tracking algorithm was used to identify cyclones from two gridded 6-hourly mean sea level pressure datasets: the 40-yr ECMWF Re-Analysis (ERA-40) and the NCEP–NCAR reanalysis (NNR) for 1958–2001. The cyclone activity climatology and changes inferred from the two reanalyses are intercompared. The cyclone climatologies and trends are found to be in reasonably good agreement with each other over northern Europe and eastern North America, while ERA-40 shows systematically stronger cyclone activity over the boreal extratropical oceans than does NNR. However, significant differences between ERA-40 and NNR are seen over the austral extratropics. In particular, ERA-40 shows significantly greater strong-cyclone activity and less weak-cyclone activity over all oceanic areas south of 40°S in all seasons, while it shows significantly stronger cyclone activity over most areas of the austral subtropics in the warm seasons. The most notable historical trends in cyclone activit...

Penalized maximal t test for detecting undocumented mean change in climate data series

Abstract In this paper, a penalized maximal t test (PMT) is proposed for detecting undocumented mean shifts in climate data series. PMT takes the relative position of each candidate changepoint into account, to diminish the effect of unequal sample sizes on the power of detection. Monte Carlo simulation studies are conducted to evaluate the performance of PMT, in comparison with the most popularly used method, the standard normal homogeneity test (SNHT). An application of the two methods to atmospheric pressure series recorded at a Canadian site is also presented. It is shown that the false-alarm rate of PMT is very close to the specified level of significance and is evenly distributed across all candidate changepoints, whereas that of SNHT can be up to 10 times the specified level for points near the ends of series and much lower for the middle points. In comparison with SNHT, therefore, PMT has higher power for detecting all changepoints that are not too close to the ends of series and lower power for d...

Comments on “Detection of Undocumented Changepoints: A Revision of the Two-Phase Regression Model”

is too large to be attributed to chance variation. The most prominent changepoint is estimated as the argument(s) c that maximizes Fc. Lund and Reeves (2002) pointed out errors that had propagated in the related literature and presented the true percentiles of the Fmax distribution under the null hypothesis of no changepoint, which is very useful for detecting stepand trend-type changepoints. However, as mentioned in Lund and Reeves (2002), changes in trend slopes could be rooted in true climate change. Also, a trend-type changepoint could well be just a point between two phases of long quasi-periodic variation (e.g., multidecadal fluctuations). Thus, extra caution should be exercised when using the above changepoint detection technique, especially when a trend-type changepoint is involved. In particular, Eq. (4.2) in Lund and Reeves (2002), which shows how to adjust the time series for the detected changepoint, should be used with caution. If the ultimate goal of the changepoint detection is to form homogeneous climate data series by correcting biases, a trend-type change in climate data series should only be adjusted if there is sufficient evidence showing that it is related to a change at the observing station, such as a change in the exposure or location of the station, or in its instrumentation or observing procedures. In addition, a simpler situation is most often encountered when correcting biases in climate observations but was not considered explicitly in Lund and Reeves (2002). That is a case when a1 5 a2 5 a; that is, a two-phase regression model with a common trend a:

Penalized Maximal F Test for Detecting Undocumented Mean Shift without Trend Change

Abstract In this study, a penalized maximal F test (PMFT) is proposed for detecting undocumented mean shifts that are not accompanied by any sudden change in the linear trend of time series. PMFT aims to even out the uneven distribution of false alarm rate and detection power of the corresponding unpenalized maximal F test that is based on a common-trend two-phase regression model (TPR3). The performance of PMFT is compared with that of TPR3 using Monte Carlo simulations and real climate data series. It is shown that, due to the effect of unequal sample sizes, the false alarm rate of TPR3 has a W-shaped distribution, with much higher than specified values for points near the ends of the series and lower values for points between either of the ends and the middle of the series. Consequently, for a mean shift of certain magnitude, TPR3 would detect it with a lower-than-specified level of confidence and hence more easily when it occurs near the ends of the series than somewhere between either of the ends and...

Daily mean sea level pressure reconstructions for the European-North Atlantic region for the period 1850-2003

Abstract The development of a daily historical European–North Atlantic mean sea level pressure dataset (EMSLP) for 1850–2003 on a 5° latitude by longitude grid is described. This product was produced using 86 continental and island stations distributed over the region 25°–70°N, 70°W–50°E blended with marine data from the International Comprehensive Ocean–Atmosphere Data Set (ICOADS). The EMSLP fields for 1850–80 are based purely on the land station data and ship observations. From 1881, the blended land and marine fields are combined with already available daily Northern Hemisphere fields. Complete coverage is obtained by employing reduced space optimal interpolation. Squared correlations (r 2) indicate that EMSLP generally captures 80%–90% of daily variability represented in an existing historical mean sea level pressure product and over 90% in modern 40-yr European Centre for Medium-Range Weather Forecasts Re-Analyses (ERA-40) over most of the region. A lack of sufficient observations over Greenland and...

New Techniques for the Detection and Adjustment of Shifts in Daily Precipitation Data Series

Abstract This study integrates a Box–Cox power transformation procedure into a common trend two-phase regression-model-based test (the extended version of the penalized maximal F test, or “PMFred,” algorithm) for detecting changepoints to make the test applicable to non-Gaussian data series, such as nonzero daily precipitation amounts or wind speeds. The detection-power aspects of the transformed method (transPMFred) are assessed by a simulation study that shows that this new algorithm is much better than the corresponding untransformed method for non-Gaussian data; the transformation procedure can increase the hit rate by up to ∼70%. Examples of application of this new transPMFred algorithm to detect shifts in real daily precipitation series are provided using nonzero daily precipitation series recorded at a few stations across Canada that represent very different precipitation regimes. The detected changepoints are in good agreement with documented times of changes for all of the example series. This st...

Homogenization and Trend Analysis of Canadian Near-Surface Wind Speeds

Near-surface wind speeds recorded at 117 stations in Canada for the period from 1953 to 2006 were analyzed in this study. First, metadata and a logarithmic wind profile were used to adjust hourly wind speeds measured at nonstandard anemometer heights to the standard 10-m level. Monthly mean near-surface wind speed series were then derived and subjected to a statistical homogeneity test, with homogeneous monthly mean geostrophic wind (geowind) speed series being used as reference series. Homogenized monthly mean near-surface wind speed series were obtained by adjusting all significant mean shifts, using the results of the statistical test and modeling along with all available metadata, and were used to assess the long-term trends. This study shows that station relocation and anemometer height change are the main causes for discontinuities in the near-surface wind speed series, followed by instrumentation problems or changes, and observing environment changes. It also shows that the effects of artificial mean shifts on the results of trend analysis are remarkable, and that the homogenized near-surface wind speed series show good spatial consistency of trends, which are in agreement with long-term trends estimated from independent datasets, such as surface winds in the United States and cyclone activity indices and ocean wave heights in the region. These indicate success in the homogenization of the wind data. During the period analyzed, the homogenized near-surface wind speed series show significant decreases throughout western Canada and most parts of southern Canada (except the Maritimes) in all seasons, with significant increases in the central Canadian Arctic in all seasons and in the Maritimes in spring and autumn.

Trends of Atlantic Wave Extremes as Simulated in a 40-Yr Wave Hindcast Using Kinematically Reanalyzed Wind Fields

Abstract In this study, seasonal extremes of wave height in the North Atlantic are analyzed. The analysis is based on a 40-yr (1958–97) numerical wave hindcast using an intensive kinematic reanalysis of wind fields. Changes in the ocean wave extremes are identified by performing the Mann–Kendall test, and are further related to changes in the atmospheric circulation (sea level pressure) by means of redundancy analysis. The relationship between sea level pressure and ocean wave extremes is also used to reconstruct the seasonal wave statistics for the last century (back to 1899). Consistent with previous studies, this high-resolution Atlantic wave hindcast also shows that the northeast North Atlantic Ocean has experienced significant multidecadal variations in the last century, and it has indeed roughened in winters of the last four decades. The winter wave height increases are closely related to changes in the North Atlantic oscillation during the last four decades. While showing trend patterns similar to ...