Graph Neural Networks for Improved El Niño Forecasting
Salva Rühling Cachay, Emma Erickson, Arthur Fender C. Bucker, Ernest Pokropek, Willa Potosnak, Salomey Osei, Björn Lütjens
GGraph Neural Networks for Improved El NiñoForecasting
Salva Rühling Cachay
Technical University of Darmstadt [email protected]
Emma Erickson ∗ University of Illinois at Urbana-Champaign
Arthur Fender C. Bucker ∗ University of São Paulo & TU Munich
Ernest Pokropek ∗ Warsaw University of Technology
Willa Potosnak ∗ Duquesne University
Salomey Osei
African Institute for Mathematical Sciences
Björn Lütjens
Massachusetts Institute of Technology
Abstract
Deep learning-based models have recently outperformed state-of-the-art seasonalforecasting models, such as for predicting El Niño-Southern Oscillation (ENSO).However, current deep learning models are based on convolutional neural networkswhich are difficult to interpret and can fail to model large-scale atmospheric patternscalled teleconnections. Hence, we propose the application of spatiotemporal GraphNeural Networks (GNN) to forecast ENSO at long lead times, finer granularityand improved predictive skill than current state-of-the-art methods. The explicitmodeling of information flow via edges may also allow for more interpretableforecasts. Preliminary results are promising and outperform state-of-the art systemsfor projections 1 and 3 months ahead.
Figure 1: We propose spatiotemporal Graph Neural Networks (GNNs) to forecast ENSO. GNNs can better exploitlarge-scale, spatiotemporal patterns indicative of ENSO than CNNs, which are based on local convolutions. ∗ Contributed equally as second authors.Tackling Climate Change with Machine Learning workshop at NeurIPS 2020. a r X i v : . [ c s . L G ] F e b l Niño–Southern Oscillation (ENSO) is an irregularly recurring phenomenon involving fluctuatingtemperatures—the alternation of warm El Niño and cold La Niña conditions—in the tropical PacificOcean. It is a major driver of climate variability, causes disasters such as floods, droughts andheavy rains in various regions of the world [1, 2, 3, 4, 5, 6, 7] and has implications for agriculture[8, 9, 10] and public health [11, 12, 13, 14]. Worldwide teleconnections, i.e. interlinked, large-scalephenomena, as well as the high variability regarding its manifestations have kept long-term ENSOforecasts at traditionally low skill .While previous studies indicate that more frequent, long-term or variable El Niño conditions mayresult due to global warming from greenhouse gases [15, 16, 17, 18], the extent of influence climatechange will have on ENSO is yet unknown and still debated given its complexity [19, 20, 21, 22, 23].This work proposes the first application of graph neural networks to seasonal forecasting and showsinitial results that outperform existing dynamical and deep learning ENSO models for 1 and 3 leadmonths. The forecasting methods in use can be broadly classified into dynamical and statistical systems[22, 24, 25]. The former are based on physical processes/climate models (e.g. atmosphere–oceancoupled models), while the latter are data-driven (including ML based approaches).
Machine Learning for ENSO forecasting
Recently, deep learning was successfully used toforecast ENSO 1 yr ahead [26] as well as with a lead time of up to . [27], thus out-performingstate-of-the-art dynamical methods. Both project the Oceanic Niño Index (ONI) for various leadtimes. The former only use the ONI index time series as input of a temporal Convolutional NeuralNetwork (CNN), while the latter feed sea surface temperature (SST) and heat content anomaly mapsdata to a CNN. Most statistical methods can only predict the single-valued index, an averaged metricover SST anomalies that does not convey more zonal information. A notable exception, makes use ofan encoder-decoder approach [28]. An overview over other machine learning methods used to projectENSO, is given in [29]. Climate networks
In climate networks [30], which stem from the field of complex networks,each grid cell of a climate dataset is considered as a network node and edges between pairs ofnodes are set up using some similarity measure . They have been used to detect and characterizeSST teleconnections [31] and to successfully project ENSO prior [32]. The latter exploits theobservation that, a year before an ENSO event, a large-scale cooperative mode seems to link theequatorial Pacific corridor (“El Niño basin”) and the rest of the Pacific ocean.
Graph neural networks
In the past years, GNNs have surged as a popular sub-area of researchwithin machine learning [33]. Interestingly, they have scarcely been used in earth and atmosphericsciences—a few applications using them for earthquake source detection [34], power outage prediction[35] and wind-farm power estimation [36]. GNNs have just recently been extended to spatiotemporalsettings, with a focus on traffic forecasting [37, 38, 39, 40].Our GNN approach to ENSO forecasting builds on the climate network’s precedent of describingclimate as a network of nodes related by non-local connections. Based on this precedent and therecent success of GNNs for spatiotemporal tasks, it is expected that spatiotemporal GNNs will beable to learn the large-scale dependencies in between climate nodes and accurately model the inherentcomplexity of the ENSO phenomenon. We are currently extending a state-of-the-art spatiotemporalGNN architecture [37], that does not require pre-defined edges and supports multi-step forecasting,to the domain of ENSO forecasting.
ENSO depends on and affects different environmental factors. Amongst these are sea-level pressure,zonal and meridional components of the surface wind, sea surface temperature, surface air temperature[21]. The climate variable, time series datasets of interest for this research are:• NOAA ERSSTv5 [41], with SST data recorded since 1854, that we have used for ourpreliminary experiments (we train on 1871-1973 and test on the 1984-2020 period)2 Coupled Model Intercomparison Project phase5 (CMIP5) [42] historical simulationsrecorded since 1861, that are particularly interesting for pre-training the model since onlyfew observational data are available• Simple Ocean Data Assimilation (SODA) [43] , reanalysis data recorded from 1870-2010• Global Ocean Data Assimilation System (GODAS) [44] reanalysis data (since 1980).The last three datasets are open-source in the processed form they were used by [27] for pre-training,training and testing, respectively. The suitability of these datasets to deep learning methods has beendemonstrated by [27]. Preliminary analysis will focus on these datasets, but more datasets may beincorporated to include other relevant variables, such as sea-level pressure and surface wind.
Graph Neural Networks (GNN) generalize the notion of locality that is exploited by ConvolutionalNeural Networks (CNN), allowing us to model arbitrarily complex connections that are paramountfor long-term forecasts of phenomena like ENSO, where relations are non-Euclidean. Importantly,CNNs assume translation equivariance of the input [45]. For seasonal forecasts, however, spatiallyshared representations for the globe do not seem adequate, since it does matter where exactly acertain phenomenon or pattern occurs (e.g. at a teleconnection versus at a distant, unrelated part ofthe world). Additionally, GNNs are more efficient than recurrent neural networks and LSTMs [40],which are often used in ENSO forecasting models [28, 46].Climate datasets are often gridded, therefore, the grid cells (i.e. geographical locations) can benaturally mapped to the nodes of a GNN. The graph’s edges, which model the flow of informationbetween nodes, are the main argument in favor of a GNN approach. Edges can be chosen based onmid- and long-range climate dependencies (e.g. based on domain expertise or on edges analyzed inclimate networks research), or they can be inferred by the GNN using recent graph structure learningapproaches [37]. The explicit modeling of interdependencies based on domain expertise, or theGNN’s choice of meaningful edges (e.g. well known patterns or teleconnections), greatly enhancesthe model’s interpretability.Moreover, most statistical methods only forecast the single-valued index and not the zonal sea surfacetemperature (SST) anomalies (which can be used for, e.g., ENSO type classification [22] and a moreinformed forecast). A GNN can naturally overcome these limitations by forecasting the target variableat the nodes—which correspond to geographical regions—of interest (in our case the SST anomaliesin the ONI region). The multiple spatiotemporal GNN architectures that have been recently proposedseem particularly well suited as a starting point [37, 38, 39, 40]. A high-level visualization of ourapproach is illustrated in Fig. 1.
The presence of an ENSO event is commonly measured via the running mean over k months of seasurface temperature anomalies (SSTA) over the Oceanic Niño Index (ONI, k = 3 ) region (5N-5S,120-170W), also known as the Niño3.4 index region ( k = 5 ). Table 1: Correlation skill for n lead months Model n = 1 n = 3 n = 6 CNN [27] ≈ mon aheadforecasts (Table 1).We use the SST anomalies within the ONI regionover 3 mon , and a simple architecture with only twolayers and no pre-defined edges. Longer lead times were not yet satisfying, which we expect to becaused by 1) the small dataset (1233 data points in the training set), which we hope to overcome byusing transfer learning like [27]; 2) while SST anomalies are good short-term predictors of ENSO,long-term ENSO projections usually rely on other variables such as heat content anomalies, whichwe aim to incorporate in our model. 3 Discussion and Future Works
An improved model could have a significant impact on global seasonal climate prediction, due toENSOs teleconnections. Leveraged as a tool by climate researchers, longer lead-time predictionswould provide more time to determine the potential impact of the phenomenon. These lead-timeswould allow those in the various impacted areas to prepare for and adapt to the predicted climate andits effects on industry, agriculture, safety, and human quality of life.In addition to helping populations impacted by ENSO, a successful deployment of a GNN architecturefor ENSO forecasting would show its suitability to non-linear and complex earth and atmosphericmodeling in general, such as projection of other oscillations or weather forecasting.Finally, future work might explore using ENSO indicators as predictors in the GNN, forecastingENSO’s impacts (such as precipitation) across the globe due to teleconnections.4 cknowledgments
We would like to thank the ProjectX organizing committee for motivating this work. We gratefullyacknowledge the computational support by the Microsoft AI for Earth Grant. We would also like tothank Captain John Radovan for sharing his expertise regarding the current ENSO models and theirglobal applications, Suyash Bire for his guidance on ENSO model limitations and result interpretation,as well as Chen Wang for his guidance on GNN architecture.Björn Lütjens’ research has been sponsored by the United States Air Force Research Laboratoryand the United States Air Force Artificial Intelligence Accelerator and was accomplished underCooperative Agreement Number FA8750-19-2-1000. The views and conclusions contained in thisdocument are those of the authors and should not be interpreted as representing the official policies,either expressed or implied, of the United States Air Force or the U.S. Government. The U.S. Gov-ernment is authorized to reproduce and distribute reprints for Government purposes notwithstandingany copyright notation herein.
References [1] J. Lin and T. Qian, “A new picture of the global impacts of el nino-southern oscillation,”
Scientific Reports , vol. 9, 12 2019.[2] P. J. Ward, B. Jongman, M. Kummu, M. D. Dettinger, F. C. S. Weiland, and H. C. Winsemius,“Strong influence of el niño southern oscillation on flood risk around the world,”
Proceedings ofthe National Academy of Sciences , vol. 111, no. 44, pp. 15 659–15 664, 2014.[3] G. B. Williamson, W. F. Laurance, A. A. Oliveira, P. Delamônica, C. Gascon, T. E. Lovejoy,and L. Pohl, “Amazonian tree mortality during the 1997 el nino drought,”
Conservation Biology ,vol. 14, no. 5, pp. 1538–1542, 2000.[4] F. Siegert, G. Ruecker, A. Hinrichs, and A. Hoffmann, “Increased damage from fires in loggedforests during droughts caused by el nino,”
Nature , vol. 414, no. 6862, pp. 437–440, 2001.[5] J. P. Donnelly and J. D. Woodruff, “Intense hurricane activity over the past 5,000 years controlledby el niño and the west african monsoon,”
Nature , vol. 447, no. 7143, pp. 465–468, 2007.[6] C. F. Ropelewski and M. S. Halpert, “Global and regional scale precipitation patterns associatedwith the el niño/southern oscillation,”
Monthly weather review , vol. 115, no. 8, pp. 1606–1626,1987.[7] ——, “North american precipitation and temperature patterns associated with the elniño/southern oscillation (enso),”
Monthly Weather Review , vol. 114, no. 12, pp. 2352–2362,1986.[8] A. R. Solow, R. F. Adams, K. J. Bryant, D. M. Legler, J. J. O’brien, B. A. McCarl, W. Nayda,and R. Weiher, “The value of improved enso prediction to us agriculture,”
Climatic change ,vol. 39, no. 1, pp. 47–60, 1998.[9] R. M. Adams, C.-C. Chen, B. A. McCarl, and R. F. Weiher, “The economic consequences ofenso events for agriculture,”
Climate Research
Agricultural and forest meteorology , vol. 262, pp. 298–309,2018.[11] R. S. Kovats, M. J. Bouma, S. Hajat, E. Worrall, and A. Haines, “El niño and health,”
TheLancet , vol. 362, no. 9394, pp. 1481–1489, 2003.[12] M. E. Marlier, R. S. DeFries, A. Voulgarakis, P. L. Kinney, J. T. Randerson, D. T. Shindell,Y. Chen, and G. Faluvegi, “El niño and health risks from landscape fire emissions in southeastasia,”
Nature climate change , vol. 3, no. 2, pp. 131–136, 2013.[13] J. A. Patz, D. Campbell-Lendrum, T. Holloway, and J. A. Foley, “Impact of regional climatechange on human health,”
Nature , vol. 438, no. 7066, pp. 310–317, 2005.[14] G. R. McGregor and K. Ebi, “El niño southern oscillation (enso) and health: an overview forclimate and health researchers,”
Atmosphere , vol. 9, no. 7, p. 282, 2018.515] S.-W. Yeh, J.-S. Kug, B. Dewitte, M.-H. Kwon, B. P. Kirtman, and F.-F. Jin, “El niño in achanging climate,”
Nature , vol. 461, no. 7263, pp. 511–514, 2009.[16] W. Cai, S. Borlace, M. Lengaigne, P. Van Rensch, M. Collins, G. Vecchi, A. Timmermann,A. Santoso, M. J. McPhaden, L. Wu et al. , “Increasing frequency of extreme el niño events dueto greenhouse warming,”
Nature climate change , vol. 4, no. 2, pp. 111–116, 2014.[17] C. Rosenzweig and D. Hillel,
Climate Variability and the Global Harvest: Impacts of El Niñoand Other Oscillations on Agro-Ecosystems . New York, N.Y.: Oxford University Press, 2008.[18] W. Cai, G. Wang, B. Dewitte, L. Wu, A. Santoso, K. Takahashi, Y. Yang, A. Carréric, and M. J.McPhaden, “Increased variability of eastern pacific el niño under greenhouse warming,”
Nature ,vol. 564, no. 7735, pp. 201–206, 2018.[19] W. Cai, A. Santoso, G. Wang, S.-W. Yeh, S.-I. An, K. M. Cobb, M. Collins, E. Guilyardi, F.-F.Jin, J.-S. Kug et al. , “Enso and greenhouse warming,”
Nature Climate Change , vol. 5, no. 9, pp.849–859, 2015.[20] H. Paeth, A. Scholten, P. Friederichs, and A. Hense, “Uncertainties in climate change prediction:El niño-southern oscillation and monsoons,”
Global and Planetary Change et al. , “El niño–southern oscillation complexity,”
Nature , vol.559, no. 7715, pp. 535–545, 2018.[22] C. Wang, C. Deser, J.-Y. Yu, P. DiNezio, and A. Clement,
El Niño and Southern Oscillation(ENSO): A Review . Dordrecht: Springer Netherlands, 2017, pp. 85–106. [Online]. Available:https://doi.org/10.1007/978-94-017-7499-4_4[23] W. Cai, B. Ng, T. Geng, L. Wu, A. Santoso, and M. McPhaden, “Butterfly effect and a self-modulating el niño response to global warming.”
Nature , vol. 585 7823, pp. 68–73, 2020.[24] D. Chen, M. A. Cane, A. Kaplan, S. E. Zebiak, and D. Huang, “Predictability of el niño overthe past 148 years,”
Nature , vol. 428, no. 6984, p. 733—736, April 2004. [Online]. Available:https://doi.org/10.1038/nature02439[25] A. V. Fedorov, S. L. Harper, S. G. Philander, B. Winter, and A. Wittenberg, “How Predictable isEl Niño?”
Bulletin of the American Meteorological Society , vol. 84, no. 7, pp. 911–920, 072003. [Online]. Available: https://doi.org/10.1175/BAMS-84-7-911[26] J. Yan, L. Mu, L. Wang, R. Ranjan, and A. Y. Zomaya, “temporal convolutional networks forthe advance prediction of enso,”
Scientific Reports , vol. 10, no. 1, pp. 1–15, 2020.[27] Y.-G. Ham, J.-H. Kim, and J.-J. Luo, “Deep learning for multi-year enso forecasts,”
Nature , vol.573, pp. 568–572, 9 2019.[28] D. He, P. Lin, H. Liu, L. Ding, and J. Jiang, “Dlenso: A deep learning enso forecasting model,”in
PRICAI , 2019.[29] H. Dijkstra, E. Hernandez-Garcia, C. Lopez et al. , “The application of machine learningtechniques to improve el nino prediction skill,”
Frontiers in Physics , vol. 7, p. 153, 2019.[30] A. A. Tsonis, K. L. Swanson, and P. J. Roebber, “What Do Networks Have to Do with Climate?”
Bulletin of the American Meteorological Society , vol. 87, no. 5, pp. 585–596, 05 2006. [Online].Available: https://doi.org/10.1175/BAMS-87-5-585[31] A. Agarwal, L. Caesar, N. Marwan, R. Maheswaran, B. Merz, and J. Kurths, “Network-basedidentification and characterization of teleconnections on different scales,”
Scientific Reports ,vol. 9, no. 1, pp. 1–12, 2019.[32] J. Ludescher, A. Gozolchiani, M. I. Bogachev, A. Bunde, S. Havlin, and H. J. Schellnhuber,“Very early warning of next el niño,”
Proceedings of the National Academy of Sciences
IEEE Transactions on Neural Networks and Learning Systems , 2020.[34] M. P. van den Ende and J.-P. Ampuero, “Automated seismic source characterisation using deepgraph neural networks,”
Geophysical Research Letters , p. e2020GL088690, 2020.635] D. Owerko, F. Gama, and A. Ribeiro, “Predicting power outages using graph neural networks,”in . IEEE,2018, pp. 743–747.[36] J. Park and J. Park, “Physics-induced graph neural network: An application to wind-farm powerestimation,”
Energy , vol. 187, p. 115883, 2019.[37] Z. Wu, S. Pan, G. Long, J. Jiang, X. Chang, and C. Zhang, “Connecting the dots: Multivariatetime series forecasting with graph neural networks,” in
Proceedings of the 26th ACM SIGKDDInternational Conference on Knowledge Discovery and Data Mining , ser. KDD ’20. NewYork, NY, USA: Association for Computing Machinery, 2020, p. 753–763. [Online]. Available:https://doi.org/10.1145/3394486.3403118[38] C. Song, Y. Lin, S. Guo, and H. Wan, “Spatial-temporal synchronous graph convolutionalnetworks: A new framework for spatial-temporal network data forecasting,” in
Proceedings ofthe AAAI Conference on Artificial Intelligence , vol. 34, no. 01, 2020, pp. 914–921.[39] Z. Wu, S. Pan, G. Long, J. Jiang, and C. Zhang, “Graph wavenet for deep spatial-temporalgraph modeling,” in
Proceedings of the Twenty-Eighth International Joint Conference onArtificial Intelligence , S. Kraus, Ed., 2019, pp. 1907–1913, international Joint Conference onArtificial Intelligence 2019, IJCAI-19. [Online]. Available: https://ijcai19.org/[40] B. Yu, H. Yin, and Z. Zhu, “Spatio-temporal graph convolutional networks: A deep learningframework for traffic forecasting,” in
Proceedings of the Twenty-Seventh International JointConference on Artificial Intelligence, IJCAI-18 , 7 2018, pp. 3634–3640. [Online]. Available:https://doi.org/10.24963/ijcai.2018/505[41] B. Huang, P. W. Thorne, V. F. Banzon, T. Boyer, G. Chepurin, J. H. Lawrimore, M. J.Menne, T. M. Smith, R. S. Vose, and H.-M. Zhang, “Extended Reconstructed SeaSurface Temperature, Version 5 (ERSSTv5): Upgrades, Validations, and Intercomparisons,”
Journal of Climate , vol. 30, no. 20, pp. 8179–8205, 09 2017. [Online]. Available:https://doi.org/10.1175/JCLI-D-16-0836.1[42] K. E. Taylor, R. J. Stouffer, and G. A. Meehl, “An overview of cmip5 and the experiment design,”
Bulletin of the American Meteorological Society , vol. 93, no. 4, pp. 485–498, 2012.[43] B. S. Giese and S. Ray, “El niño variability in simple ocean data assimilation (soda),1871–2008,”
Journal of Geophysical Research: Oceans , vol. 116, no. C2, 2011. [Online].Available: https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2010JC006695[44] D. Behringer and Y. Xue, “Evaluation of the global ocean data assimilation system at ncep: Thepacific ocean,” 2004.[45] I. Goodfellow, Y. Bengio, and A. Courville,
Deep learning , 2016.[46] C. Broni-Bedaiko, F. A. Katsriku, T. Unemi, M. Atsumi, J.-D. Abdulai, N. Shinomiya, andE. Owusu, “El niño-southern oscillation forecasting using complex networks analysis of lstmneural networks,”