William W. Hsieh

Applying Neural Network Models to Prediction and Data Analysis in Meteorology and Oceanography.

Empirical or statistical methods have been introduced into meteorology and oceanography in four distinct stages: 1) linear regression (and correlation), 2) principal component analysis (PCA), 3) canonical correlation analysis, and recently 4) neural network (NN) models. Despite the great popularity of the NN models in many fields, there are three obstacles to adapting the NN method to meteorology–oceanography, especially in large-scale, low-frequency studies: (a) nonlinear instability with short data records, (b) large spatial data fields, and (c) difficulties in interpreting the nonlinear NN results. Recent research shows that these three obstacles can be overcome. For obstacle (a), ensemble averaging was found to be effective in controlling nonlinear instability. For (b), the PCA method was used as a prefilter for compressing the large spatial data fields. For (c), the mysterious hidden layer could be given a phase space interpretation, and spectral analysis aided in understanding the nonlinear NN relat...

Nonlinear principal component analysis by neural networks

Nonlinear principal component analysis (NLPCA) can be performed by a neural network model which nonlinearly generalizes the classical principal component analysis (PCA) method. The presence of local minima in the cost function renders the NLPCA somewhat unstable, as optimizations started from different initial parameters often converge to different minima. Regularization by adding weight penalty terms to the cost function is shown to improve the stability of the NLPCA. With the linear approach, there is a dichotomy between PCA and rotated PCA methods, as it is generally impossible to have a solution simultaneously(a) explaining maximum global variance of the data, and (b) approaching local data clusters. With the NLPCA, both objectives (a) and (b) can be attained together, thus the nonlinearity in NLPCA unifies the PCA and rotated PCA approaches. With a circular node at the network bottleneck, the NLPCA is able to extract periodic or wave modes. The Lorenz (1963) 3-component chaotic system and the monthly tropical Pacific sea surface temperatures (1950-1999) are used to illustrated the NLPCA approach.

Nonlinear canonical correlation analysis by neural networks

Canonical correlation analysis (CCA) is widely used to extract the correlated patterns between two sets of variables. A nonlinear canonical correlation analysis (NLCCA) method is formulated here using three feedforward neural networks. The first network has a double-barreled architecture, and an unconventional cost function, which maximizes the correlation between the two output neurons (the canonical variates). The remaining two networks map from the canonical variates back to the original two sets of variables. Tested on data sets with correlated nonlinear structures, NLCCA showed that the underlying nonlinear structures could be retrieved accurately under moderately noisy conditions. After a mode had been retrieved, NLCCA was applied to the residual to successfully retrieve the next mode. When tested for prediction skills, the NLCCA outperformed the CCA when the two sets of variables contained correlated nonlinear structures.

The Free Kelvin Wave in Finite-Difference Numerical Models

Abstract The effects of viscosity and finite- differencing on free Kelvin waves in numerical models (which employ the Arakawa B- or C-grid difference schemes) are investigated using the f-plane shallow-water equations with offshore finite-difference grids, (assuming alongshore geostrophy). Three nondimensional parameters arise: Δ [=(offshore grid spacing)/(Rossby radius)], ϵ characterizes the offshore lateral viscous effect and α the combined vertical and alongshore viscous effect. This study is more relevant to baroclinic Kelvin waves which tend to suffer poor offshore resolution because of their small Rossby radii. For inviscid models (ϵ = α = 0), as Δ increases (resolution worsens), the alongshore speed increases dramatically in the B-grid, but stays constant at the gravity wave speed in the C-grid. Models with damping only (α > 0, ϵ = 0) behave similarly. With lateral viscosity (ϵ > 0, α > 0), increasing ϵ decreases the speed in both the B- and C-grids—the drop in speed being less severe when the free...

Nonlinear Canonical Correlation Analysis of the Tropical Pacific Climate Variability Using a Neural Network Approach

Abstract Recent advances in neural network modeling have led to the nonlinear generalization of classical multivariate analysis techniques such as principal component analysis and canonical correlation analysis (CCA). The nonlinear canonical correlation analysis (NLCCA) method is used to study the relationship between the tropical Pacific sea level pressure (SLP) and sea surface temperature (SST) fields. The first mode extracted is a nonlinear El Nino–Southern Oscillation (ENSO) mode, showing the asymmetry between the warm El Nino states and the cool La Nina states. The nonlinearity of the first NLCCA mode is found to increase gradually with time. During 1950–75, the SLP showed no nonlinearity, while the SST revealed weak nonlinearity. During 1976–99, the SLP displayed weak nonlinearity, while the weak nonlinearity in the SST was further enhanced. The second NLCCA mode displays longer timescale fluctuations, again with weak, but noticeable, nonlinearity in the SST but not in the SLP.

Skill Comparisons between Neural Networks and Canonical Correlation Analysis in Predicting the Equatorial Pacific Sea Surface Temperatures.

Abstract Among the statistical methods used for seasonal climate prediction, canonical correlation analysis (CCA), a more sophisticated version of the linear regression (LR) method, is well established. Recently, neural networks (NN) have been applied to seasonal climate prediction. Unlike CCA and LR, NN is a nonlinear method, which leads to the question whether the nonlinearity of NN brings any extra prediction skill. In this study, an objective comparison between the three methods (CCA, LR, and NN) in predicting the equatorial Pacific sea surface temperatures (in regions Nino1+2, Nino3, Nino3.4, and Nino4) was made. The skill of NN was found to be comparable to that of LR and CCA. A cross-validated t test showed that the difference between NN and LR and the difference between NN and CCA were not significant at the 5% level. The lack of significant skill difference between the nonlinear NN method and the linear methods suggests that at the seasonal timescale the equatorial Pacific dynamics is basically l...

2006 Special issue: Neural network forecasts of the tropical Pacific sea surface temperatures

A nonlinear forecast system for the sea surface temperature (SST) anomalies over the whole tropical Pacific has been developed using a multi-layer perceptron neural network approach, where sea level pressure and SST anomalies were used as predictors to predict the five leading SST principal components at lead times from 3 to 15 months. Relative to the linear regression (LR) models, the nonlinear (NL) models showed higher correlation skills and lower root mean square errors over most areas of the domain, especially over the far western Pacific (west of 155 degrees E) and the eastern equatorial Pacific off Peru at lead times longer than 3 months, with correlation skills enhanced by 0.10-0.14. Seasonal and decadal changes in the prediction skills in the NL and LR models were also studied.

Interactive Feedback between the Tropical Pacific Decadal Oscillation and ENSO in a Coupled General Circulation Model

The output from a coupled general circulation model (CGCM) is used to develop evidence showing that the tropical Pacific decadal oscillation can be driven by an interaction between the El Nino-Southern Oscillation (ENSO) and the slowly varying mean background climate state. The analysis verifies that the decadal changes in the mean states are attributed largely to decadal changes in ENSO statistics through nonlinear rectification. This is seen because the time evolutions of the first principal component analysis (PCA) mode of the decadal- varying tropical Pacific SST and the thermocline depth anomalies are significantly correlated to the decadal variations of the ENSO amplitude (also skewness). Its spatial pattern resembles the residuals of the SST and thermocline depth anomalies after there is uneven compensation from El Nino and La Nina events. In ad- dition, the stability analysis of a linearized intermediate ocean-atmosphere coupled system, for which the background mean states are specified, provides qualitatively consistent results compared to the CGCM in terms of the relationship between changes in the background mean states and the characteristics of ENSO. It is also shown from the stability analysis as well as the time integration of a nonlinear version of the in- termediate coupled model that the mean SST for the high-variability ENSO decades acts to intensify the ENSO variability, while the mean thermocline depth for the same decades acts to suppress the ENSO activity. Thus, there may be an interactive feedback consisting of a positive feedback between the ENSO activity and the mean state of the SST and a negative feedback between the ENSO activity and the mean state of the thermocline depth. This feedback may lead to the tropical decadal oscillation, without the need to invoke any external mechanisms.

The Nonlinear Patterns of North American Winter Temperature and Precipitation Associated with ENSO

Abstract Nonlinear projections of the tropical Pacific sea surface temperature anomalies (SSTAs) onto North American winter (November–March) surface air temperature (SAT) and precipitation anomalies have been performed using neural networks. During El Nino, the linear SAT response has positive anomalies centered over Alaska and western Canada opposing weaker negative anomalies centered over the southeastern United States. In contrast, the nonlinear SAT response, which is excited during both strong El Nino and strong La Nina, has negative anomalies centered over Alaska and northwestern Canada and positive anomalies over much of the United States and southern Canada. For precipitation, the linear response during El Nino has a positive anomaly area stretching from the east coast to the southwest coast of the United States and another positive area in northern Canada, in opposition to the negative anomaly area over much of southern Canada and northern United States, and another negative area over Alaska. In c...

A nonlinear analysis of the ENSO cycle and its interdecadal changes

Abstract The nonlinear principal component analysis (NLPCA), via a neural network approach, was applied to thermocline anomalies in the tropical Pacific. While the tropical sea surface temperature (SST) anomalies had been nonlinearly mapped by the NLPCA mode 1 onto an open curve in the data space, the thermocline anomalies were mapped to a closed curve, suggesting that ENSO is a cyclic phenomenon. The NLPCA mode 1 of the thermocline anomalies reveals the nonlinear evolution of the ENSO cycle with much asymmetry for the different phases: The weak heat accumulation in the whole equatorial Pacific is followed by the strong El Nino, and the subsequent strong drain of equatorial heat content toward the off-equatorial region precedes a weak La Nina. This asymmetric ENSO evolution implies that the nonlinear instability enhances the growth of El Nino, but dwarfs the growth of La Nina. The nonlinear ENSO cycle was found to have changed since the late 1970s. For the pre-1980s the ENSO cycle associated with the ther...