Introduction

“The fight against climate change is inseparable from the fight against poverty and inequality”1. It is crucial to study the natural variability of climate to fully comprehend its equilibrium. Both Western Africa (WAF - Fig. 1) and Northeast Brazil (NEB - Fig. 1) have an agricultural-based economy, making them highly dependent on rainfall2,3,4,5,6,7. These regions are sensitive to climatic changes and presented large decadal precipitation (precipitation) variability in recent history5,7,8. The Failed State Index for 2018, which is a social, political, and economic index shows that West African countries are either in the alert or high alert category. In its most recent report (AR6), the Intergovernmental Panel on Climate Change (IPCC) concluded that the population living in drylands is expected to double over northern Africa, the western Sahel, and Southern Africa by 2050. Considering the recent migration trend, Africa is the most rapidly urbanizing region globally and is projected to transition to a majority urban population in the 2030s, a 700% increase in mid-latitude Africa alone, and climate change has the potential to greatly increase this projection. At 1.5 °C warming in West Africa, urban populations exposed to severe droughts are projected to increase 65.3 ± 34.1 million by 2050. Furthermore, considering an expected 1.7 °C global warming, 17–40 million people could migrate internally in sub-Saharan Africa (>60% in West Africa)9. This region exhibits severe droughts caused by irregular decadal rainfall anomalies, where the understanding of what is causing climate variability and its effects are still subject of investigation4,7,10.

Fig. 1: The Tropical and South Atlantic.
figure 1

NEB (50°–35° W and 15° S–0°) and WAF (20° W–10° E and 0°–15° N) box regions highlighted according to the IPCC criteria for Northeast Brazil and Western Africa in Assessment Report 67.

Full size image

Furthermore, according to the IPCC’s fifth and sixth Assessment Reports, NEB appears as one of the most vulnerable regions concerning droughts, fires, agricultural losses, climatic-related migrations, and displacements5,7,11,12,13,14. 70,000 km2 have reached a point where agriculture is no longer possible. Intense droughts have triggered migration to urban centers in and outside NEB. More than 10 million people in this region have been impacted by the drought of 2012/14, which was responsible for water shortage and contamination15. On the other hand, the 1.5 °C increase in global projection16 may increase by 100–200 % of the population affected by floods in Brazil7. Rainfall in the Tropical Atlantic and adjacent continents (Fig. 1) is mostly related to the band of intense convection and surface wind convergence known as the Intertropical Convergence Zone (ITCZ)17. The movement of the ITCZ seems to be the primary factor that drives the rainy and dry seasons in NEB and WAF. The ITCZ meridional displacements bring the rainfall season to the Sahel (a semi-arid region within WAF) from June to September and to NEB in late November18. Wind, pressure, and sea surface temperature gradients move this convergence zone across the equator into the warmer hemisphere19,20. All regions that are dependent on the ITCZ are affected by these anomalies and consequently, the Atlantic decadal variability cycle2,20.

The precipitation link between eastern South America and western Africa has been subject of interest in various paleoclimate studies21,22,23, in particular in the mid-Holocene (around 6000 years ago), a period characterized by higher than present summer insolation in the Northern Hemisphere24. While the direct effects of the mid-Holocene insolation pattern produced an enhancement in the Northern Hemisphere monsoon, the Southern Hemisphere precipitation experienced a reduction when compared to present day25,26,27. The mid-Holocene is known for a more vegetated Sahara and Sahel, which could have influenced precipitation in South America28,29,30. Although precipitation proxies retrieved from different paleo archives indicated more ITCZ incursions in the NEB coast due to the weakening of the South Atlantic Subtropical High, the semi-arid region of NEB presented drier than present conditions during that time23. 2000 years later (around 4000 years ago), northern Africa became drier, whereas South America experienced wetter conditions21.

Since the main natural forcing of climate variability comes from changes in insolation31 and large volcanic eruptions32, the decadal precipitation patterns in a specific region can often be understood based on the climate epoch determined by orbital factors during that time series. However, studying the correlation between two regions separated by an Ocean basin is not trivial. Precipitation is dependent on the ocean and atmosphere interactions arising from non-linear dynamical processes with large-scale climate phenomena and their influences19,33. These processes are also referred to as modes of variability, which can have large impacts on regional rainfall34,35,36,37.

The SST structure of the Tropical Atlantic impacts the meridional position of the ITCZ, which in turn influences the decadal rainfall distribution in the adjacent regions38,39. Therefore, at decadal time scales, we may observe an anti-correlation between NEB and WAF precipitation anomalies. This decadal anti-correlation should be similar to the interannual anti-correlation from the wet and dry seasons of these regions. However, at different timescales, different processes and teleconnections may come into play and therefore slight deviations in the anti-correlation between the two regions may occur39,40.

Climate modes of variability can be identified using linear methods (e.g., principal component analysis - PCA), which do not account for the nonlinearity of intrinsic internal variability20,41,42. Self organizing maps (SOM), on the other hand, can extract the linear modes of input data but are not limited to depicting the linear behavior of climate data41. Instead of forcing orthonormality between the patterns, the SOM algorithm forces the feature space patterns to closely correlate with the input data features. Although climate indices and principal components are useful in investigating highly correlated events and local climate features, it can be challenging to link multiple climate indices to larger-scale phenomena19. At the global scale, climate data are highly nonlinear and affected by several different ocean and atmosphere interactions and feedback. Therefore, a nonlinear approach such as the SOM methodology can be a more efficient tool for understanding the evolution and entanglement arising from climate data variables43,44.

This study aims to investigate the physical systems that govern decadal precipitation in Northeast Brazil and Western Africa during the 1979–2015 period. We then extend our analysis to a longer time period using only SST anomalies going from 1870 to 2019, to better evaluate the possibility of decadal periodicity of this system. In doing so, we use Neural Networks to elucidate how the Atlantic modes of climate variability are related to changes in rainfall on both sides of the Tropical Atlantic at decadal time scales.

Results

We first calculate the decadal Standardized Precipitation Index (SPI) for NEB and WAF regions using precipitation data from the satellite era (1979–2015). The SPI uses only precipitation data to create a drought index by estimating the probability of drought in a predetermined time scale45. Then, a multivariable SOM neural network with 800 neurons using decadal SST, precipitation, pressure, and wind anomalies for the Tropical and South Atlantic Ocean (65° W–25° E, 29° N–59° S) is implemented to extract the data’s features in a self-organized map. An analysis of the correlation between each variable of the SOM feature space and the climate data time series is used to discuss the interdependence in the Atlantic decadal variability and establish the temporal causality between the SOM clusters. Lastly, to investigate the potential recurrence of specific patterns within decadal variability cycles of the tropical Atlantic Ocean, we utilized the twentieth-century decadal SST anomaly to create another 800-neuron SOM and analyze its cluster patterns correlation to the time series.

1979–2015 SPI series and SOM patterns

The decadal SPI series for NEB and WAF (Fig. 2) starts in 1989 (the first SPI value for January 1989 is computed with the precipitation from 1979 to 1989). Over a decadal time scale, the anti-correlation between NEB and WAF precipitation emerges, showing a correlation value of −0.82 (p-Value = 0.048) from 1979 to 2005. From 1989 to 1995, the SPI index of NEB is above 1 (indicating very wet conditions) while WAF shows values lower than -1. From 1995 to 2005, NEB gets gradually drier, while WAF precipitation increases. NEB SPI reaches its most negative value in 2000, peaking at over -2 (severe drought), while WAF reaches its most wet period a few years after (2004). From 2005 to 2015, the anti-correlation weakens as the precipitation anomalies in both regions are not particularly significant, never exceeding 1 in absolute value in the WAF region and only briefly, around 2009, in the NEB.

Fig. 2: The Standardize Precipitaion Index.
figure 2

The Standardized Precipitation Index series with a decadal filter from 1989 to 2015, calculated with land-only precipitation from the GPCP dataset74. Orange for negative and green for positive precipitation anomalies. NEB region on the top and WAF region on the bottom. The dashed line indicates the +1 and −1 SPI limits for wet and dry conditions. The correlation between the two curves is −0.82 (p < 0.05) from 1980 to 2005.

Full size image

To associate the climate patterns with the out-of-phase decadal rainfall distribution for the Tropical and South Atlantic adjacent regions, a neuron network using the SOM algorithm with hexagonal geometry, gaussian neighboring, PCA initialization and eight hundred (20 × 40) neuron-grid points is created (Fig. S1). The SOM is then trained using the 1979–2015 observed anomalies of precipitation, sea surface temperature, winds, and sea level pressure for the Tropical and South Atlantic region. After the network converges, the k-means and silhouette algorithms yield a total of seven clusters in the SOM’s feature space, each one representing a possible climate state for the Atlantic Ocean (Fig. 3).

Fig. 3: The SOM Patterns at the Satellite Era.
figure 3

SOM patterns evolution from 1979 to 2015. The top row: precipitation anomaly cluster patterns, from orange (negative anomaly) to green (positive anomaly). The bottom row: SST (shading), wind (vectors), and pressure anomalies (contours, anomaly values in Pascals). The patterns are organized by their evolution in time from 0 to 6.

Full size image

To identify which cluster (i.e. possible climate state) best represents the Atlantic Ocean system at each time step, a correlation between the SOM feature space with each variable of the dataset was calculated (shown for the SST anomaly patterns in Fig. 4 and for all the variables in Fig. S2). For example, the SST anomalies from SOM pattern 0 (Fig. 3 - bottom left, blue to red shadings) show a positive correlation with the SST anomalies from the observational data from 1979 to 1985, reaching its peak in 1983, about 90% (Fig. 4, top curve). This allowed us to establish a temporal causality between the cluster patterns. The time evolution of the Atlantic Ocean SST anomalies can also be seen in Supplementary Video 1 with the multivariable SOM evolution and mapping of the SST anomalies from 1979 to 2015.

Fig. 4: The time causality of SOM patterns and the Atlantic Ocean Modes.
figure 4

SOM patterns correlation to the satellite era (1979–2015) dataset and Atlantic Ocean Modes. In the first seven lines, from top to bottom: the Atlantic Ocean SST anomalies data correlated to the SST SOM patterns (SST anomalies in Fig. 3) during the satellite era for patterns 0 through 6. In the three final lines, from top to bottom: the satellite era indexes from the South Atlantic Subtropical Dipole (SASD), the Atlantic Meridional Mode (AMM), and the Atlantic Equatorial Mode (AEM). The mode’s spatial structures are presented in the Supplementary Fig. S5. Their indexes were calculated as follows: SASD is the difference between the SST anomalies mean in 30°–40° S, 10°–30° W and 15°–25° S, 0°–20° W; the AMM is the difference between the SST anomalies mean in 15°–5° N, 50°–20° W and 15° S–5° S, 20° W–10° E; and the AEM is the SST anomalies mean in 3° N–3° S, 20° W–0°.

Full size image

In the satellite era, the Tropical and South Atlantic initial states are correlated to the SOM feature space with patterns 0 and 1 (Fig. 3 - first and second columns; Fig. 4 - first and second rows), when a positive SST anomaly followed by a low-pressure anomaly in the Subtropical South Atlantic (SSA - from 20° to 40° S) initiated a movement towards the Tropical South Atlantic (TSA from 0° to 20°S). This change from pattern 0 to 1 shows a clear inversion of the South Atlantic Subtropical Dipole (SASD). The SASD is defined as a SST dipole in the Tropical South Atlantic oriented in the northeast-southwest direction. This mode has been correlated to changes in the South Atlantic Subtropical High and the South Atlantic Convergence Zone46,47,48. As an outcome of this positive SST anomaly migrating to the TSA, a north-south SST gradient emerges. The Atlantic Meridional Mode (AMM) is the first linear mode of the Atlantic Ocean PCA. It represents variations in the SST gradient along the equator in the Atlantic and is correlated with a southward displacement of the ITCZ19,49,50. These SST and pressure anomalies create the southward wind anomaly from pattern 1, bringing positive precipitation anomalies to the South American coast (Fig. 3 - second column). Near the Benguela region (10° S–30° S, 10° E), on the Central African coast, a SST anomaly dipole shapes the eastern Atlantic sector of these patterns. In the Gulf of Guinea, the top negative part of this dipole can be identified as the Atlantic Equatorial Mode (AEM) or Atlantic Niño mode of variability. These quasiperiodic interannual SST anomalies are identified as one of the main linear modes of the Atlantic Ocean19. It develops along the equatorial Atlantic and all the way down to the Southern Central Africa coastal region. When developed as a cold tongue, it is known to keep the summer rain band from advancing into the interior sites of the WAF region, while increasing precipitation in interdecadal timescales at the Guinea coast51,52. This set of conditions gives origin to the anti-correlation of highly positive/negative precipitation anomalies in NEB/WAF from the 1980s to the early 1990s (Fig. 2).

From 1995 to 2005, the Tropical and South Atlantic system evolved from pattern 1–2 to 3–4 in the SOM feature space (Fig. 3). The pressure anomalies from SSA shift from negative to positive, reversing the direction of wind anomalies, which interrupts the positive precipitation anomalies over NEB, while the Benguela coast SST dipole changes its sign in the eastern part of the Atlantic Ocean. This scenario depicts a physical mechanism known at the intraseasonal time scale as the Lindzen-Nigam process53,54,55. The Lindzen-Nigam process describes how a negative SST anomaly in the western SSA and the warm tongue of the AEM, referred to as the South Atlantic Ocean Dipole, gives rise to wind divergence over the SSA linked to convergence and vigorous upward motion over the TSA56. This highly convective system on the east side of the Atlantic is contrasted with a large negative precipitation anomaly in the South American continent (Fig. 3 - pattern 3). The precipitation anti-correlation between these two sides of the Atlantic Ocean persists, and it is illustrated by the phase inversion of the SPI with the NEB negative SPI reaching its lowest value in 2000, while WAF experienced its highest precipitation peak in 2004 (Fig. 2).

Correlated with the Tropical and South Atlantic system from 2005 to 2015, patterns 4, 5, and 6 show the Atlantic Ocean evolution into another SST pattern, defined positive anomalies in Tropical North Atlantic (TNA - from 20∘N to 0) and negative in the TSA, inverting the AMM phase seen in pattern 1. The AMM signal combined with positive pressure anomalies in the SSA is responsible for weakening the South Atlantic Convergence Zone, causing negative precipitation patterns in the South American coast (Fig. 3 - pattern 6). Furthermore, a positive southwest wind anomaly near the Equator triggers a northerly displacement of the decadal ITCZ position, increasing precipitation in western Africa (north of the WAF region box). However, the negative pressure and positive SST anomalies in pattern 4 weaken in the Benguela region. This AEM weakening changes the wind pattern in the Gulf of Guinea from a southerly to a clockwise wind anomaly regime, leading to drier conditions along the coast at the southern edge of the WAF domain (Fig. 3 - patterns 5 and 6). Other factors influence rainfall in this region, in particular, the larger scale vertical motion in the mid-troposphere which controls the relative humidity and, therefore, the cloud formation57,58. Given that the ITCZ is displaced northward by the AMM, subsidence is expected to the south of the domain (i.e., over central Africa’s coast), implying that the drier conditions in this region are a combination of weakened AEM and enhanced subsidence in the middle troposphere.

The last four patterns (Fig. 3) show a transition to a drier climate in South America, in agreement with droughts impacting most parts of Brazil in particular from 2011 to 20175,12,59,60. However, it should be noted that the SPI in NEB is positive between 2007 and 2013, peaking in 2009. In NEB, the coastline is meridionally oriented in the eastern part of the domain and zonally oriented in its northern part. Along the meridionally oriented coastlines, precipitation is mainly related to the zonal wind and extends inland61,62,63. In SOM pattern 4, correlated with the Tropical Atlantic system in the 2000s (Fig. 4 - fourth row), positive SST anomalies at the South American coast combined with wind anomalies pointing in the direction of the continent were responsible for transporting moisture into NEB interior sites. This mechanism caused the NEB SPI series to spike from 2007 to 2013 (Fig. 2), despite a northward ITCZ displacement at that time.

Table 1 describes each SOM pattern in terms of the NEB and WAF decadal precipitation anti-correlation and Atlantic modes.

Table 1 Description of the SOM patterns.
Full size table

1870–2019 Atlantic SST anomaly cycles

Decadal climate variability can be seen as an oscillatory evolution. Although the climate never precisely repeats itself, many similarities can be found when investigating the evolution of climate patterns in time. Tropical and South Atlantic SST anomalies from 1870 to 2019 concerning the 1979–2015 period are calculated and detrended. This time series is then used to create another 800-grid SOM neural network. Our analysis shows that the Tropical and South Atlantic system evolution is represented in the SST SOM feature space with a 50-year periodicity (Fig. 5; see also Supplementary video 2). At the beginning of each cycle from the 1870–2019 time series, the Tropical and South Atlantic system is correlated with the same pattern in the SOM feature space: positive SST anomalies in the TNA and mainly negative SST anomalies in the TSA and SSA (central panel in Fig. 5). Moreover, this pattern evolves similarly in every cycle, either developing a positive SST anomaly in the SSA or the TSA and, sometimes, accompanied by a shift to a negative SST anomaly in the TNA. The recurrence of this pattern depicts a periodicity of the decadal SST anomalies in the Tropical and South Atlantic of the order of 50-year timescale.

Fig. 5: The sea surface temperature SOM patterns at the twentieth century.
figure 5

SST anomaly cycles identified by the SOM algorithm. On the top, the twentieth century SST anomaly patterns evolution in the form of a directed graph. On the bottom, in the form of three timelines, one for each cycle. CP stands for the central piece of the directed graph. Cycle number 1 (yellow arcs, left of the directed graph) was identified as the evolution of SST anomaly patterns from 1870 to 1920; Cycle number 2 (green arcs, top of the directed graph) was identified from 1920 to 1970; Cycle number 3 (red arcs, right of the directed graph) represents the SST anomaly patterns in the 1970–2019 period. This Figure is a representation of video 2 from the Supplementary Materials.

Full size image

Cycle 1 (1870–1920)

The SST anomaly time series begins in 1870 with the main pattern (Fig. 5, central pattern) represented by a latitudinal dipole with a positive SST anomaly in the TNA and mainly negative SST anomalies in the TSA and the SSA. This pattern seems to appear cyclically, therefore, we considered it as the centerpiece (CP) of the directed graph. The evolution of this cycle begins with erratic transitions between the positive SST anomaly in the TNA to the SSA (shown by the α arc in Fig. 5, in yellow).

At the beginning of the twentieth century, the positive SST anomaly migrates to the TSA (from 0° to 30° S) and returns to the central pattern (from 1900 to 1905 shown by the β arc in Fig. 5, in yellow); later on, a small intensification in the TNA SST positive anomaly takes place together with a positive SST anomaly in SSA (from 1910 to 1920, shown by the γ arc in Fig. 5), however the anomalies quickly return to the main pattern.

Cycle 2 (1920–1970)

Cycle number 2 resembles a combination of different patterns seen in Cycle 1, this time the SSA develops a positive SST, while the TSA stays negative and the TNA SST anomaly is positive. The SSA positive anomaly is closer to Africa’s coast (especially near the Benguela region - from the tip of the Central Africa coast to 12° S). The positive SST anomaly moves northward to the TSA (peaking in 1960), returning to the main pattern in 1970.

Cycle 3 (1970–2019)

Cycle number 3 is the same pattern evolution seen in the SOM Patterns 0 through 6 seen in Fig. 3, captured via the SST SOM mapping of the full SST reanalysis dataset. The pattern begins with a positive SST anomaly in SSA and a negative one along the equator. From 1970 to 2018 the positive SST anomaly slowly migrates into the TNA, returning to the main pattern once again. As seen earlier, this cycle was the main driver of the decadal anti-correlation seen between NEB and WAF precipitation anomalies from 1979–2015 (Fig. 2).

Discussion and conclusions

In this study, we use the Self-Organizing Maps (SOM) to explore the decadal variability of rainfall over the Tropical and South Atlantic and in particular to better understand the anticorrelation (Fig. 2) we found between precipitation anomalies in Northeast Brazil (NEB) and western Africa (WAF). The anticorrelation is very strong from 1979 to 2004 (82%, p < 0.05), and it then weakens in the last decade (2005–2015). Such weakening is associated with rainfall over the NEB and WAF regions that do not show any significant anomaly (Fig. 2), except for a brief positive precipitation spike in NEB in 2009. Consequently, the rainfall anti-correlation between the two regions weakens as regional effects and non-linear influences become more dominant. In the satellite era (1979–2015), the SOM methodology successfully reduced the data dimensionality from 432 months to 7 patterns, which were ordered in time (Fig. 3) according to their correlation with the climate data time series (Fig. 4). This created a temporal causality between the SOM patterns, which allowed the interpretation of the Tropical and South Atlantic system dynamics. In the last part of our study, we trained another SOM, this time with the 1870–2019 HadISST reanalysis dataset. When correlating its feature space represented by nine patterns (Fig. 5) with the sea surface temperature (SST) data time series, a 50-year periodicity containing the full spectrum of the Atlantic SST decadal variability in the twentieth century was found. This SST periodicity at decadal scales is an indication that the anti-correlation between NEB and WAF might be a periodic feature from the Tropical and South Atlantic system.

The multivariable SOM patterns (Fig. 3) describe how SST, wind, and pressure anomalies control the precipitation on both sides of the Atlantic Ocean. By determining an order of causality between the different SOM patterns, our analysis shows how the Atlantic modes of variability are intertwined. Several studies3,19,52,60,64,65 have used linear methods such as the principal component analysis (PCA) that help explain regional features from each climate snapshot of the Tropical and South Atlantic system. However, the EOF/PCA analysis by itself is unable to examine the linear mode interactions with each other, as they are defined as independent orthogonal vectors retrieved by the dataset state space66. The SST anomaly patterns determined by the SOM (Fig. 3) depict the natural composition of the South Atlantic SST decadal variability, where each pattern is formed by a complex combination of the Atlantic Ocean modes (Table 1). The causality and evolution of these linear modes of variability naturally emerge from the SOM feature space (Fig. 4), enabling the previous discussions of their interactions and multiple variable entanglements. For example, pattern 0 (Fig. 3) presents a non-neutral South Atlantic Subtropical Dipole with a negative pressure anomaly at the Subtropical South Atlantic, which weakens the South Atlantic Subtropical High. This pattern is correlated with the Tropical and South Atlantic system at the beginning of the satellite era and it holds the initial condition that sets the Atlantic Ocean’s climate system in motion. As the system state advances in the SOM feature space represented by patterns 0 through 6, the positive SST anomalies consistently move to the north in the direction of North Tropical Atlantic, while shifting the phases of the Atlantic Equatorial Mode, South Atlantic Ocean Dipole and Atlantic Meridional Mode, one step at a time. These shifts drive the precipitation patterns responsible for the NEB and WAF decadal anti-correlation.

In the last part of our study, another SOM was trained with the entire 1870-2019 HadISST reanalysis dataset. Represented by nine patterns (Fig. 5), the SST SOM feature space found a 50-year periodicity containing the full spectrum of the Atlantic SST decadal variability in the twentieth century. These cycles are an indication of the possible periodicity of the decadal precipitation anti-correlation between NEB and WAF seen in the satellite era. However, the further we go back in our reanalysis dataset, we have larger uncertainties67,68. Therefore, no specific physical mechanism can be discussed in detail from these 50-year cycles besides the apparent periodicity of the main pattern (center panel from Fig. 5). Each cycle may contain different precipitation patterns, as they are evolving from slightly different initial conditions. Climate signals from SST anomalies can differ in position and magnitude of their amplitude maximum, generating each time a unique climate response40. In other words, other precipitation, wind, and pressure anomaly patterns may emerge from the twentieth-century Atlantic SST cycles that are not present in the 1979–2015 available satellite data.

Since higher-frequency events involving droughts/floods have a higher chance of occurring during decadal low/high precipitation anomaly periods69, mitigating the impacts of such events on vulnerable populations can also be achieved through accounting for these decadal patterns. This investigation of the decadal precipitation anti-correlation between the NEB and WAF regions and its underlying dynamics constitutes a significant advancement in our understanding of precipitation variability in the Atlantic region.

Methods

Data

Precipitation data

The Global Precipitation Climatology Center (GPCC) uses the global data collections of the Climate Research Unit (CRU)70, Food and Agriculture Organization (FAO), and GHCN71, as well as other precipitation data collections from international regional projects, while the CPC Merged Analysis of Precipitation (CMAP)72 and GPCP are the most widely recognized and used merged data sets. The GPCP is based on the sequential combination of microwave, infrared, and gauge data. From 1979 to 2020 and offer globally complete satellite-only precipitation estimates73.

To examine the precipitation patterns over the Atlantic and their relations to different climate regimes, we utilized the GPCP Version 2.3 Combined Precipitation dataset74. This dataset provides both continental and ocean precipitation data in a monthly, 2,5° × 2,5° grid, which enabled us to identify climate patterns during the 1979–2015 period and calculate the NEB/WAF decadal SPI (as described below).

Pressure and wind data

The pressure and wind stress data used in this study are both from NCEP-NCAR Reanalysis 1, a 1.875° × 1.875° monthly dataset75. The period used to create the Atlantic patterns was the same as for the precipitation limits, from 1979 to 2015.

Sea surface temperature data

The temperature data comes from HadISST1. A monthly 1° × 1° dataset67 which is a Reanalysis dataset that uses observational data from ship expeditions and platforms interpolated by a numerical model to recreate the Global SST. The time examined is from 1870 to 2019, but it is important to take into consideration that, previous to the 1920s, the observation inputs used for the dataset numerical reconstruction were sparse, especially in the Southern Hemisphere68. The sea surface temperature anomaly was calculated based on the 1979–2015 climatology. To separate the Atlantic SST anomaly pattern of variability from any global warming signal the mean global temperature anomaly was also calculated and subtracted from the SST anomaly time series76,77,78.

Low frequency filters

To study the decadal variability of the Atlantic Ocean, decadal filters were applied to all observational and Reanalysis data sets. These filters are calculated with the simple decadal mean from the original monthly time series.

Methods

The Standardized Precipitation Index (SPI)

The SPI was designed to quantify the precipitation deficit for multiple timescales45. The index is based on only one variable (precipitation). It measures positive and negative precipitation anomalies periods and it has been used in several studies to quantify the rainfall in different regions of the world79,80,81,82.

Different time scales can be used when calculating the SPI. To study the low-frequency fluctuations of precipitation we used the decadal SPI (Fig. 2)45.

The SPI is a normalized time series (Fig. 2) that can be calculated using a pre-developed algorithm83 and has a simple numerical interpretation: SPI values between −1 and 1 indicate a near-normal precipitation regime, from 1 to 1.49 (−1 to −1.49) moderately wet (dry), 1.50 to 1.99 (−1.50 to −1.99) indicates very wet (dry) and above 2 (below -2) extremely wet (dry).

Self organizing maps

A neural network is a machine learning technique where a collection of interconnected neurons incrementally learn from their environment (data) to capture essential linear and nonlinear patterns and trends in complex data. Used in many areas of study, neural networks can be used to represent information, characteristics, and dynamics from a main problem84,85.

The self organizing maps (SOM) are a non-supervised neural network. Its goal is to map a high dimensional space of data into a lower dimensional space, without disrupting the original data topology and creating a function able to project the input data in the SOM feature space and back to the input space (Fig. S1)86,87.

The SOM feature space can be thought of as a non-orthonormal vector base (Fig. S1), preserving the non-linearity of the datasets41. However, it is no coincidence that some SOM patterns can resemble modes of variability obtained with the definition of climate indices and the principal component analysis (PCA) of the Atlantic decadal SST. When using the PCA, the linear modes obtained maximize the variance projected into an orthonormal base of the data64,66,84. In Oja’s work88, it was proved that a single-layer perceptron with a simple Hebbian updating rule, similar to the SOM algorithm (using cooperation and competition from the neuron’s weights in a non-supervised learning algorithm), naturally converges to the input data’s first component84. In other words, if we created a single neuron SOM grid, the training process would result in a vector that maximizes the input data variance.

The analysis for this work was implemented in both single variable and multivariable forms, with two different time intervals: for the satellite era: from 1979 to 2015; and for the SST reanalysis dataset period (1870-2019). In the first analyses, precipitation, SST, wind, and pressure monthly anomalies of the Tropical Atlantic region (65° W–25° E, 29° N–59° S) from 1979 to 2015 were given as the input data to the multivariable SOM algorithm to generate an eight-hundred neuron grid. In Fig. S3 this input data appears as the x’s. Random values starting weights were used to initialize the eight-hundred-neuron grid representing the Atlantic region climate variables from 1979 to 2015. The arcs from x’s in Fig. S3 represent those weights, and their sum creates the neuron grid points (y’s from Fig. S3). The SOM algorithm will then use the input data in a loop to compare and improve its representations of the data, creating separated regions of its feature space that correlate with different features of the dataset (presented after the training process with its correlation to a specific time step of the dataset in Fig. S1).

To classify the number of hidden classes after the network’s training, the neuron grid was clusterized. In many practical situations, the number of clusters from the data set is unknown. There are various k-means algorithms to determine the number of hidden groups in a dataset89. After defining a metric from the feature space, most methods consist of measuring which clustering minimizes the distance from members inside the same cluster (compact clustering), while also maximizing the distance between clusters (separation of different classes). Here we used the Silhouette and k-means/elbow methods to find the optimal number90.

The Silhouette metric goes from −1 to 1, the closer the Silhouette value is to 1 the closer you are to the optimal number of clusters. While the k-means/elbow method uses the Sum of Squared Errors (SSE) curve to define what is the optimal cluster number (the limit from which the error reduction we get from raising the number of clusters gives us hardly any advantage over the previous number of clusters), this value sometimes can be visually interpreted as an elbow in the k-means graph due to the sharp decay from the 1st derivative of the SSE per number of clusters.

With a fixed number of clusters, the patterns from the neuron grid can be expressed as in Fig. 3. After the causality of each SOM pattern is established by their correlation with the observational data time series (Figs. 4 and S2), the SOM patterns are used to elucidate some of the ocean and atmospheric physics behind the rainfall regime in NEB and WAF regions for the last four decades (Fig. 2).

Following the same SOM methodology, the 1870–2019 SST anomalies from the Tropical and South Atlantic Ocean region were used to create the SOM pattern evolution and SST anomaly cycles (Fig. 5 and video in supplementary material).

The SOM neural network was trained to solve a classification problem. The dynamical analysis emerging from the time evolution of the multivariable SOM clusters is used to understand the physical mechanisms of the Atlantic Ocean driving the decadal SPI anti-correlation between NEB and WAF. Although possible, the SOM was not used to classify datasets different from its training datasets. The main idea of the SOM application for this work is to generate a feature space that represents a continuous evolution from the input dataset in time and promote a dimensionality reduction via clusterization of its feature space. The possibility of using the same Neural Network to classify unobserved climate data representing the South Atlantic Ocean was not explored.

SOM sensitivity

The SOM utilized here has a 20 × 40 neuron grid with hexagonal geometry, Gaussian Neighboring, and PCA initialization for fast convergence. The results are not sensitive to the rectangular/hexagonal topology or the PCA/random initialization. Different grid dimensions were tested to produce Figs. 3 and 5, their clusterization indexes are shown in Fig. S4. The ideal number of clusters was unanimously 7, both in the Silhouette method, where the curves from 30 × 50, 20 × 40, and 10 × 20 SOMs show the global maximum and the 5 × 10 SOM presents a local maximum; and in the elbow methodology using the k-means algorithm. The results emerging from the clustering of the SOMs feature spaces using 7 clusters are not sensitive to the discussed grid changes.

As mentioned before, the multivariable SOM consisted of a rectangular grid of 800 neurons (20 × 40, Fig. S1), while the full-time dimension of the data was 1728 (4 variables with 432 monthly data each). The single variable SOM using 1870–2019 SST anomalies has an 800-neuron grid (40 × 20), while the full-time dimension of the HadISST dataset is 1788 monthly SST. The SOMs grid dimensions were chosen after extensive research, trial, and error. They were a consequence of the chosen metric, used in the neural network’s learning and the subsequent analysis (differences in R-squared values from images). The feature space should be able to capture and project the nuances and evolution of the data, therefore, the grid’s dimension must be comparable to the input data’s dimension we wish to reduce, otherwise, the R-squared difference may not have positive values when calculating the correlation between each SOM neuron and the time series at each time step (Fig. S1). The SOM neural grid should not apply a drastic dimensionality reduction by itself. In this study, it is the clustering of the SOM grid (clusterization of the feature space) that provides the ultimate dimensionality reduction.

When applying a drastic dimensionality reduction directly through the SOM’s feature space, some caveats regarding the clustering and neighboring features should be taken into consideration. When using a small grid (the 5x10 grid in this case), neurons from different clusters will be closer together. At each training step, when looking at one of the input vectors, the SOM algorithm searches for the best matching unit (BMU) neuron in its grid. After finding the BMU, the weights connecting the BMU and its neighbors to that input vector are increased. If the neighboring range is not adjusted in the training algorithm, this cooperation, which was created to generate separable clusters inside the feature space, might have the exact opposite result. If the neighboring range is too high in comparison to the feature space dimensions, all the neurons of the grid might influence very different input vectors, creating more neutral patterns. Although this feature might be desirable in approaches that search for strong signals in large noisy datasets, they can erase important non-linear features from the data’s evolution. This problem will appear in the clustering of the feature space. For example, the Silhouette curve of the lower dimension grid (5 × 10) in Fig. S4, where high values are seen for a small number of clusters. Since the patterns are more neutral and similar to each other, the algorithm points to them being well-represented by fewer classes. Nevertheless, the k-means still points to 7 clusters and the Silhouette curve presents a local maximum with this number, identifying 7 as the optimum number of clusters.

For capturing the non-linearity of climate data, a large dimension of the SOM neural grid is needed (how large depends on the nature of the phenomena being studied). Our study explains the Atlantic Ocean control over the NEB and WAF decadal precipitation anomalies. To study smaller influences in the climate patterns coming from other Ocean basins it might be useful to use larger than 800 neurons SOM grids.