Forecasting asylum-related migration flows with machine learning and data at scale

The sudden and unexpected migration flows that reached Europe during the so-called ‘refugee crisis’ of 2015–2016 left governments unprepared, exposing significant shortcomings in the field of migration forecasting. Forecasting asylum-related migration is indeed problematic. Migration is a complex system, drivers are composite, measurement incorporates uncertainty, and most migration theories are either under-specified or hardly actionable. As a result, approaches to forecasting generally focus on specific migration flows, and the results are often inconsistent and difficult to generalise. Here we present an adaptive machine learning algorithm that integrates administrative statistics and non-traditional data sources at scale to effectively forecast asylum-related migration flows. We focus on asylum applications lodged in countries of the European Union (EU) by nationals of all countries of origin worldwide, but the same approach can be applied in any context provided adequate migration or asylum data are available. Uniquely, our approach (a) monitors drivers in countries of origin and destination to detect early onset change; (b) models individual country-to-country migration flows separately and on moving time windows; (c) estimates the effects of individual drivers, including lagged effects; (d) delivers forecasts of asylum applications up to four weeks ahead; (e) assesses how patterns of drivers shift over time to describe the functioning and change of migration systems. Our approach draws on migration theory and modelling, international protection, and data science to deliver what is, to our knowledge, the first comprehensive system for forecasting asylum applications based on adaptive models and data at scale. Importantly, this approach can be extended to forecast other social processes.


Introduction. Problem and background
The 2015-16 refugee crisis in Europe was sudden and unexpected, and the actions taken by Governments to secure borders and uphold access to asylum procedures were generally reactive, uncoordinated and ineffective. The ultimate cause of the crisis was poor capacity to anticipate the movements of asylum seekers.
Here we demonstrate that unsupervised adaptive machine learning algorithms can combine administrative and non-traditional data at scale to deliver short term forecasts of flows of asylum seekers from any country of origin to European Union (EU) countries -and in principle to any country that collects data on asylum applications with adequate frequency.
Forecasting asylum-related migration is extremely problematic. Migration is a complex system, 1 which means that causal factors interact nonlinearly, are highly context dependent, and show little or no persistence over time. Potential drivers are diverse, 2,3 plus effect sizes and interactions vary wildly between and within individual migration flows. In one context extreme conflict, violence and persecution may generate few asylum seekers; whereas elsewhere relatively subtle social unrest may spark large international displacements, particularly if they are a tipping point of deteriorating conditions. The effect of migration drivers is subject to threshold and feedback effects. Once activated, country to country flows tend to trigger self-reinforcing processes resulting in the establishment of migration systems [4][5][6] .
Asylum related migration is therefore a highly uncertain process, 7 which complicates migration modelling. 8 Among migration types, forced or asylum-related migration is associated with the highest uncertainty. 9 As a consequence, quantitative asylum migration models often focus on single drivers in countries of origin (e.g. conflicts [10][11][12] or destination (e.g. migration or asylum policies [13][14][15] ). More comprehensive asylum migration models have been developed but these aim to increase retrospective understanding rather than forecasting flows 10,[16][17][18][19][20] , with exceptions mostly confined to the prediction of single country to country flows 21 .
Data on migration in general and its drivers also contain uncertainty, which further complicates migration modelling 7 . Despite recent advances in the collection of official statistics, particularly in the subfield of asylum, and in spite of the ongoing efforts to improve data collections at the international 4 and global 5 levels, most data collections are limited in terms of frequency, definitions, coverage, accuracy, timeliness, and quality assurance [22][23][24] . This is also the case for data on migration drivers such as conflicts, the state of human rights and the economy -notably with regards to their frequency, accuracy and timeliness -all of which are prerequisites for effective forecasting.
Recent developments in data at scale and computational technologies underpin some progress in asylum migration modelling and forecasting. Large data sets containing vast reams of structured and unstructured data have been proposed as an opportunity to observe the unbroken totality of potential migration drivers as they occur in near to real time 25,26 . New data sources include mobile data 27 , social media 28,29 , and internet searches 30 . Big data are increasingly analysed with models such as agent-based modelling 31 and techniques such as machine learning 32 to detect patterns and identify potential migration drivers that would otherwise go unnoticed. Such developments enabled the development of novel migration forecasting models, including forced and asylum migration, with encouraging results in terms of reliability and timeliness -which makes them potentially useful in operational scenarios 32,33 . However, to our knowledge even the most advanced models have been applied to a limited number of flows rather than generalised to the regional or global levels.

Design Approach
Here we show that adaptive machine learning algorithms that integrate official statistics with nontraditional data sources can effectively capture early warning signals of asylum-related migration and forecast asylum applications in countries of destinations. Our early warning and forecasting system proceeds by monitoring countries of origin to detect drivers of migration early onset; estimating the effects of individual drivers, including lagged effects; on those bases, producing forecasts of asylum applications up to four weeks ahead; finally, assessing how patterns of drivers shift over time to describe the functioning and change of migration systems.
To observe migration covariates at different stages of migration processes, we exploit three tiers of data: geolocated events and internet searches in countries of origin; detections of irregular crossings at the borders of the EU; and asylum recognition rates in countries of destination. We leverage the potential of non-traditional data sources to position the analysis as close as possible to migration drivers as they happen.
Our approach addresses the complexity of migration systems and drivers by modelling each single countryto-country migration flow separately. Moreover, models are trained on moving time windows to account for change over time even within individual flows. We applied our method on a wide range of bilateral asylum flows covering all (circa 200) countries of origin worldwide, and 30 countries of destination in Europe. In the Extended Data section we report performance results for 70 country-to-country flows, generated by seven countries of origin (Afghanistan, Eritrea, Iraq, Nigeria, Syria, Turkey and Venezuela) and nine countries of destination (Austria, Belgium, Germany, Greece, Spain, France, Italy, The Netherlands and Sweden), plus the EU+ as a whole. The selected flows represent a suitably large diversity on the variables analysed; but our method can be applied in any context, provided adequate migration or asylum data are available.
The workflow of the early warning and forecasting system is sketched in Figure 1.

Target variable
Our dependent variable is the number of asylum applications lodged in each country member of the EU Common European Asylum System (which includes EU Member States plus Norway and Switzerland, and is referred to as EU+), and in the EU+ as a whole [ Figure 1- (4)]. These data, broken down by nationality of each applicant, are shared with the European Asylum Support Office (EASO -the EU asylum agency) on a weekly basis. Although provisional, in 2019 these data underestimated asylum applications by just 6% at the EU+ level, compared to official national statistics compiled by Eurostat (EU 862/2007). Estimates of asylum applications typically have a maximum value imposed upon them by the capacity of the receiving country to react to sudden influxes and quickly register each application. This ceiling effect is demonstrated by fewer applications being registered during Christmas and Easter holidays due to the brief closure of some asylum offices, although applications tend to be registered shortly thereafter 34 .

Covariates
To measure covariates we combine administrative statistics with non-traditional data.

TIER1: Data in countries of origin Events
We estimate the timing and location of 'push factor' events in nearly all countries of origin from the Global Database of Events, Language, and Tone (GDELT) project, [ Figure 1-(1)], a repository of 316 types of geolocated event reported in the world's broadcast, print and web media, in 100 languages. 6 We used GDELT 1.0 data, which are updated on a daily basis; GDELT 2.0 data are updated every 15 minutes and could potentially feed a real-time system. Single events can be covered multiple times by different media outlets around the world and therefore occur multiple times in the GDELT data, so we extract individual events from the overall media coverage and include each event only once in our data. Not all events reported by the media are expected to be migration drivers and so we selected a subset of 240 events as potential drivers of migration and displacement. 35 Individual events differ in the extent to which they are likely to affect displacement and migration, therefore we weighted (+-) each individual event according to its potential to generate displacement. Then we aggregated all weighted events into five macro-categories: political events, social unrests, conflicts, economic events, governance-related events [ Figure 1- (5)]. Finally, the total number of weighted events per macro-category per week were used in the following models, in order to align them with the frequency of the response variable. More details on event selection and weighing and on the general construction of event indexes are presented in the Methods section. The complete list of events and weights is included in the Supplementary Information section. 6 See Web Page at: https://www.gdeltproject.org. GDELT "monitors the world's broadcast, print, and web news from different countries in over 100 languages and identifies the people, locations, organizations, counts, themes, sources, emotions, counts, quotes, images and events driving society daily".

Internet searches
Internet searches for particular topics may anticipate migration, and indeed internet searches have recently been used to estimate migration intentions and predict migration flows 30 . To estimate patterns of relevant internet searches, we use Google Trends, a publicly available search engine providing multi-language, geolocated data on the relative frequency of search topics and terms 7 [ Figure 1- (2)].
As shown in Table 1, we selected 17 topics (clusters of keywords) related to international migration and travelling in general (visa or passport), asylum seeking (right of asylum or refugee), countries of transit (e.g. Jordan or Turkey for searches that take place in Syria) and countries of destination (e.g. Germany, France, or EU). Then we downloaded the relative search frequencies for these topics in non-EU countries. The General User Interface of our system permits to easily customise the topic searches to include in the analysis of single countries. For example, we included searches related to some particular 'transit countries' among search topics (see Table 1). We selected those particular countries of transit for inclusion among topic searches because in the period covered by this analysis the arrivals were high on the Greek-Turkish border, and countries in the Middle East and Central Asia were the origin of a significant share of asylum applicants in Europe. Our model would then discard those variables if not relevant as predictors (for example, while performing forecasts of asylum applications by nationals of Venezuela). In principle, we could have included different sets of topic searches for each individual country of origin included in the analysis.

TIER 3 -Asylum processes data in the EU+
To capture the potential effect on asylum applications of asylum processes and practice in countries of destination, we include data on 'recognition rates' [ Figure 1-(4)] calculated as the share of total asylum decisions that grant (rather than reject) international protection. 9

EU+ Member States and Schengen
Associated Countries share monthly asylum decisions with EASO broken down by the nationality of the applicant; validated data are later released by Eurostat. Recognition rates vary markedly between receiving countries, and they have been shown to be positively related to asylum applications 15 . Moreover, recognition 8 Data are defined by EBGCA as "data reported on a monthly basis by Member States and Schengen Associated Countries on detections of illegal border-crossing on entry between Border Crossing Points of the external borders of the Member States of the EU and Schengen Associated Countries, and aggregated by routes". As EBCGA reports, "The data refer to detections of illegal border-crossing rather than the number of persons, as the same person may cross the external border several times. However, there is currently no EU system in place capable of tracing each person's movements following an illegal border-crossing. Therefore, it is not possible to establish the precise number of persons who have illegally crossed the external border." (source: notes to the data spreadsheet downloaded at https://frontex.europa.eu/along-eu-borders/migratory-map/, last checked on 17 February 2020). 9 Refugee status or Subsidiary protection status as defined in Article 2 of the Qualification Directive 2011/95/EU rates can cause a deflection effects 13,16 whereby lower recognition rates in one country may induce asylum seekers to lodge their applications in countries with higher recognition rates.

Procedure
One of the most severe constraints to migration theory and modelling is that migration processes connect single country of origin to country of destination dyads in complex systems whose functioning vary largely over time and space. To address this constraint, rather than attempting to build a single asylum migration model we model each individual country-to-country dyad separately. In practice, the procedure starts with the selection of one or more countries of origin and proceeds with the analysis of time series on three tiers of potential covariates: o asylum applications in all EU+ countries and in the EU+ as a whole Given that uncertainty increases with the forecasting horizon, we forecast the number of asylum applications (one, two, three, and) four weeks ahead. While a relatively short term, four weeks is a highly valuable time window for planning and preparedness purposes in an operational context. We design a system that works in two steps: early warning and forecasting. 10 In the period covered in the analyses, the United Kingdom was still a Member State of the European Union and the Common European Asylum System.

Early Warning
In the first step -early warning analysis [ Figure 1 The optimal predictive lag of each time series on the outcome variable -asylum applications -is found using a lead-lag estimation method 36 . In addition to lead-lag analysis and change point estimation, the acceleration of each time series is measured based on the ratio between shorter (6 weeks) and longer (24 weeks) moving averages. We borrow this 'momentum approach' from quantitative finance 37 , and set at ± 110% the ratio's threshold for triggering alerts (cf. 38,39 ), but like the other parameters of the early warning system this one also can be customised. The early warning analysis is explained in full details in the Methods section.      The procedure for the forecasting step is explained in detail in the Methods section, and most parameters can easily be customised. By way of illustration, Figure 3 shows a sample forecast for asylum applications lodged by nationals of Afghanistan in the EU+ in early 2019. The real number of asylum applications is represented by the green line before the point when the forecast is simulated, and by the dotted blue line after that point. The four-week forecast is the red line.
Afghan nationals had been among the top-three nationalities for asylum applications in the EU+ for most of the time between 2014 and 2018. As was the case with most top-ranking nationalities, the volume of applications by Afghani nationals fell sharply after the EU and Turkey agreed to end irregular migration from Turkey -the main country of transit at that time -to the EU in March 2016. 11 In addition to this structural change, the flow was also subject to cyclical movements and some seasonality -notably a drop at the end of each year when the processing capacity of asylum authorities tends to be severely limited, followed by subsequent increase. Both AdaENet and Arima attempt to capture this behaviour, but AdaENet is far more effective. In the four weeks considered in the back-test, the total number of applications was 3424; AdaENet forecasted 3445 (0.6% relative error, 21 units absolute error) and Arima 2826 (-17.5% relative error, 598 units). 11 See https://ec.europa.eu/commission/presscorner/detail/en/MEMO_16_963. As a result, Irregular crossings at the EU border by Afghan nationals went down from a monthly average of about 26700 the year before, to one of about 1430 the year after. Asylum applications decreased markedly as a result, although the change in the asylum trend became visible some six months after the EU-Turkey statement. Between October 2015 and September 2016, the average number of monthly asylum applications lodged in the EU+ was 123308, which went down to a monthly average of 63170 between October 2016 and September 107.

Back-testing and forecasting errors
To test the forecasting performance, we simulated weekly forecasts from 30 April 2017 to 1 September 2019.
The forecasting system takes data from April 2016 to April 2017 and then iteratively moves onward by one week at every step. This means that the procedure replicates a hypothetical real forecast using only information that would have been available at each point in time, each time running early warning analyses to generate lagged variables that can be retained by the system in the forecasting step. properties of these administrative data such as the drop at the end of each year that has been observed above for Afghans applicants.
In almost all weeks, the forecast stays within the confidence bands of +-2 standard errors, which means that

b-c. Summary statistics for the relative error (b) and for the absolute error (c). Arima is a benchmark model which is only based on the auto-correlation of the application timeseries
To further illustrate the performance of the system over time and space, in the Supplementary Information section we present an analysis of forecasting performances for a selection of 70 dyads comprising of seven countries of origin (Afghanistan, Eritrea, Iraq, Nigeria, Syria, Turkey and Venezuela) 13 and ten destinations (Austria, Belgium, Germany, Greece, Spain, France, Italy, The Netherlands and Sweden, and the EU+ as a 12 Applications lodged by Syrian nationals in Greece, Sweden and the EU+; of Venezuelans in France, Spain and the EU+; of Nigerians in Germany and Italy; of Afghans in Germany. 13 Between 2016 and 2019, applications by nationals of the seven selected countries of origin lodged 1 654 040 applications in the EU+, 47.6% out of a total of 3 473 050 applications received. a b whole). As discussed further in the Supplementary Information section, these dyads were selected to provide a large variation across relevant variables and thus test the performance of the forecasting system over largely different conditions. More extensive analyses have been carried out. 14 However, the Early Warning and Forecasting System analyses all (circa 200) countries of origin and 30 EU+ countries of destination, and the same performance statistics can be generated for all country-of-origin-to-country-ofdestination dyads.

Adaptiveness of the AdaENet model and assessment of change in migration systems
The matrix in Figure 5 shows the variables selected by the AdaENet model in each week of the back-testing period. The colours represent the relative importance of the predictor in the AdaENet model. Because the model is adaptive, the variables selected as well as their relative importance may vary from week to week.
The relative rank is evaluated through a Random Forest algorithm on the restricted model selected by AdaENet and represented through colours in Figure 5 -from 0 ("variable not included") to 1 ("most important").
The heatmap illustrates some key features of the model and of the underlying processes. First, the effects of single variables tend to persist for several weeks (horizontal colour bands). This indicates that country-tocountry asylum flows have some (temporal) structure, which the model is able to capture. Variables are clustered in the heatmap in such a way to show this persistence. In Figure

Conclusion
With increased numbers of displaced persons around the world, irregular migration and asylum have risen up the political agenda. If governments are to effectively manage mixed migration flows, they need to understand cause and effect and plan for future influxes. However, context dependent, short lived complexity combined with sparse data means that forecasts are rare, tentative and unreliable.
This work takes a novel approach to align data on events and internet searches in countries of origin, detections of irregular crossing at the European border, and asylum decisions in European countries. These data are fed into an adaptive machine learning system that delivers short-term forecasts of individual mixed migration flows that consistently outperform benchmark models, as well as assessments of causal factors in the medium to long term. Forecasts of individual migration flows have the potential to inform short term preparedness and resource allocation in the field or in national asylum offices. Assessments of medium and long term drivers contribute to the understanding and modelling of mixed migration flows, and can also be used to inform policy decisions.
Our approach bridges the worlds of migration theory and modelling, international protection, and data science to deliver an effective system for early warning and forecasting of migration. But this approach has the potential to be adapted to understand and forecast other systems and processes characterised by complexity and sparse data.

Events indices
We extract from the GDELT database those events that occur in the lead paragraph of a document (coded as Because it is grounded on migration theory and should therefore reduce the level of noise in the indicators, and because it also seems more closely correlated to asylum applications and recognition rates, we prefer our event indexes. The complete list of GDELT events and those selected for our indicators are available in the Supplementary Information section.

Internet search queries
We rely on the Google Trends service to crawl the relative search of each topic (see Table 1) for all the countries in the world on a weekly basis.
Forecasting procedure. Early warning function

Statistical data cleaning and filtering
A preliminary analysis of each time series is performed in order to drop from the analysis any time series that has insufficient variability (i.e. no statistical information), which can be due either to too low values or to too many missing values as can occasionally be the case for applications and detections of irregular bordercrossing. This stage of the pre-analysis has several filtering parameters as shown in Table M1. Before performing this task, the data are aligned to have a common weekly frequency. GDELT data have daily frequency and can be aggregated at the weekly level. Google Trends and asylum applications data are weekly.
Data on irregular border crossings and recognition rates have a monthly frequency and are transformed into weekly series through linear interpolation.
The variability is taken into account in terms of the coefficient of variation (cv = standard deviation/|mean|).
Being a pure number, i.e. without measure unit, it allows for variable independent thresholding: a time series whose variability is below 5% (the default value) is assumed to be statistically unreliable.

Change point analysis
At this point the algorithm performs two change point tests: one for the mean of the time series and one for the volatility in the last 12 months. Statistical hypotheses testing on the change point are calculated with respect to the p-value parameter.

Spotting acceleration
Some characteristics of the time series could potentially complicate or bias the early warning analysis. For example, a time series may have a slope (positive or negative) without any statistically significant change point, or a change point may have occurred far in the past. The time series may also be highly volatile and change points detected only due to isolated spikes, which however would not imply any persistent change.
To take into account the volatility or the speed of change, we compare a short and long term time series at MA1=6 months and MA2=24 months period. In a period of stationarity, those time series converge, but in case of positive (negative) acceleration, they diverge. If the MA1/MA2 is larger than a given threshold of 1.1, then a signal is fired by the early warning system. There might be isolated signals (the case of huge but isolated spikes) or a series of consecutive signals. In the latter case, the acceleration in activity/trend can be considered as a real signal. This technique is based on high frequency quantitative finance in fast portfolio optimization called the momentum approach 37-39 .

Correlation and lead-lag analysis
Instead of analysing simultaneous correlations, Google Trends and GDELT data were analysed with a Lead-Lag approach, with one time series thereby anticipating another. The lead-lag effect is commonly applied in financial econometrics. In time series, this notion can be compared to Granger causality (see 41 ; for empirical evidence, cf. [42][43][44][45] ). However, Granger-like approaches face several constraints: 1) the time series must be of the same frequency; 2) the time series must be linear, and 3) testing for causality often leads to bidirectional effects. An additional problem is the Epps effect which states that, as the sampling frequency of time series increases, the empirical correlation is reduced 46 . The Lead-Lag approach overcomes the Epps effect by using Hoffman's Lead-Lag estimator 36 based on the Hayashi-Yoshida asynchronous covariance estimator [47][48][49] . This permits to apply a Lead-Lag approach to asynchronous, non-linear time series with different frequencies and missing data.
More precisely, let ∈ (− , ) be the time lag between the two nonlinear time series and . The approach consists in constructing a contrast function ( ) which evaluates the Hayashi-Yoshida estimator for the times series and + and then to maximize it as a function of . The lead-lag estimator ̂ of is defined When the value of ̂ is positive it means that and +̂ (or −̂ and ) are strongly correlated, therefore we say " leads by an amount of time ̂" , so is the leader and is the lagger. Vice versa for negative ̂. The assessment of the identified lag is done through a statistical test.
To retain the most significant lagged correlation effects, we also calculate the Lead-Lag-Ratio (LLR). LLR is used when there are two lead-lag effects, one for a positive lag and one for a negative lag, both statistically significant. In this case, the strongest among the two shortest (i.e. close to 0) lead-lag effects is returned by the LLR test statistics. Lead-lag and LLR as well as asynchronous correlation are available at present only through the yuima R package 50,51 .
Forecasting procedure. Forecasting function The forecasting system attempts to forecast the value of the variable applicants from a country of origin (CoO) to different countries of destination (CoD) including the EU+ in aggregate. To this aim, the information from the early warning step is also used. This means that this task includes the dropping of some of the variables as explained the above.
The analysis focuses on the variables named applicant.*, where * stands for one of CoD.
When data for "future" applications are available, as in back-testing, the system compares the forecast with the actual data. The complete list of arguments of the function forecast are illustrated in  The forecasting model is the result of a meta-analysis based on all variables with lead-lag effects that entered and survived the previously exposed early warning step.
The forecasting strategy follows these steps: • Set up an adaptive elastic net model them. Notice that AdaENet is also a variance shrinking method, which implies that the standard errors of the coefficients of the selected variables are relatively small compared to, e.g., linear regression.
The AdaENet model itself has two tuning parameters: and . The first one, , is set to 0.5 in our approach, which means Lasso (L1-penalty) and Ridge (L2-penalty) estimations are equally weighted in the loss function of the optimization problem. The parameter , the adaptive scaling factor for the penalties, is first estimated using cross-validation over a very large set of possible values in order to minimise the forecast MSE in the training data. This 'optimal' forecasting value is then used in the AdaENet penalty function. This procedure is performed every week, so the optimal changes from week to week [ Figure 1-(13)].
The tuning parameter is quite important. The larger this number, the stronger AdaENet will shrink the estimated coefficients to zero, in a potentially artificial way. To take into account this potential source of bias, the elastic net considers a grid of different values for and estimates the penalized regression model for a particular choice of (in our case 0.5). Then is then automatically selected ex-post by cross-validation.
In practice, what happens inside our forecast function is more complicated than the above classical AdaENet algorithm. In particular, historical data on a time varying window are used to estimate the best AdaENet model, i.e. the one with the cross-validated . Then the window is moved one week ahead, and the estimation procedure is calculated. This analysis is iterated until the current data available inside the forecast function. The final model, i.e. the final , is chosen among those models such that they attain the lowest prediction error (as measured by the mean squared error, MSE): for all models such that the estimated MSE is less or equal the variability of the time series, the optimal is extracted and then, the lowest among all lambda's, is used in the subsequent analysis.
In summary, instead of considering the best model for the whole period of data, the forecast function selects the best model for each sub-period (time varying window) and then defines the optimal as the average and the minimal lambda in all the estimated models. This guarantees that the final forecasting model produces on average and in a robust way the best forecast in all periods. The optimal lambda shrinks the coefficients more than the minimal lambda.

VAR modelling
To obtain the forecast, the model also needs to simulate the future values of the covariates.

Sources of uncertainty
In practice there are two sources of approximation: the first is the simulation of the future values through a VAR/Arima/AR model and the second one is the prediction error of the estimated model on the historical data. Nevertheless, the forecasting models seem to be able to do a statistically sound job, in the sense that forecasts are generally within the 2 standard errors bands of the moving average process based on historical data (as seen, e.g., in Figure 4).

Random forest for variable ranking
To have an additional insight on the relative importance of each variable selected by the AdaENet model, we run ex-post a random forest model and we rank the predictors according to the importance measure of this algorithm. As the number of predictors may vary from week to week, we consider the relative rank. This relative rank generates the colour scale in Figure 5.
Code R 4.0.0 language was used for the data cleaning, data mashup, statistical analysis and data visualization.
Several R packages have been used. in particular glmnet for the implementation of the Adaptive Elastic Net; yuima for the lead-lag analysis; sde, for the change point analysis as well as the package ts and VAR respectively for the ARIMA and VAR models estimation; randomForest was used to extract the relative importance of each predictor in the model selected by the Adaptive Elastic Net algorithm. The rest of the analysis was produced via own written R code.

Data availability
The data that support the findings of this study are available from the authors. Restrictions apply to the availability of some of the data, and notably to weekly data on asylum applications. Any request of weekly asylum data should be agreed with the national asylum authorities of EU+ Member State, owner of the data.

Extended data
This Extended Data section includes ten figures providing statistics and details about the forecasting performance of the system on selected country-to-country asylum flows. The sample flows were selected to provide a diverse set of contexts covering some major destination countries (France, Germany, Greece, Italy, Spain, and the EU+ in the aggregate) ; some major countries of origin, from different global regions (Middle East, Asia, Africa, South America); relatively more (e.g. Nigeria to Italy) and less (e.g. Syria to Greece) volatile flows. Moreover, the associated forecasts range between very good (Extended Data

Selection of countries and flows for back-testing
The early warning and forecasting system runs for all pairs of countries of origin (circa 200) to countries of destination (30 EU+ countries at the time of the analysis), which amounts to a total of circa 6000 dyads or country to country flows. Here we report results of back-testing on a selection of 70 dyads, generated by Countries of origin -Global region and distance from countries of destination: o Countries in the Middle East (SY and TR), Asia (AF and PK), Africa (NG and ER) and America (VE) -Internal situation during the period analysed:

Expectations
The high diversity represented in the sample of cases selected for back-testing provides a useful combination for assessing the adaptive capacity of the model. For example, events might be expected to be useful predictors of applications by nationals of such a country as Syria that has been in a civil war for years; or of Afghanistan, although to a lesser extent. Events should also be relevant to predicting asylum flows from countries that were unstable although not plagued by civil wars, like Pakistan; as well as from Nigeria and Venezuela afflicted by severe economic crises. Key transit countries can provide powerful barriers against migration to Europe, and in the back-testing we have included among covariates events in Turkey to account for that.
Internet searches, in turn, may be relevant to predicting migration from a range of countries (Bohme et al 2020), but less so from countries where internet penetration is limited. For some extreme cases like Eritrea, which is a stable dictatorship where the population has barely any access to the internet, events and internet searches are expected to be poor predictors. Our model, however, also includes variables related to EU border crossing as well as policy processes and outputs in countries of destination, such as applications and recognition rates in EU+ countries. These should be useful to support predictions when events or internet searches are not good predictors.
In general, the complexity of migration systems means that we cannot have strong expectations about the predictors of individual asylum flows. Even where we can conjecture about the relevance of individual factors to a given country to country flow, we still expect the system to change over time -and predictors to change accordingly. Therefore, we can only have some very general expectations about to the functioning and performance of the early warning and forecasting system, that we can group as follows: 1) Expectations related to the predictive capacity of the system: • E1. The system should be able to predict asylum applications under different contexts • E2. The forecasting error should normally stay within an acceptable confidence interval, i.e. +-2 standard errors from the real trend • E3. Forecasts of relatively more stable, more patterned, and less volatile trends should have smaller errors than forecasts of more unstable, less patterned and more volatile trends • E4. The Ada-ENet adaptive model should outperform a benchmark ARIMA model based on extrapolation from the trend 2) Expectations related to the capacity of the system to adapt to changing patterns: • E5. Even in cases of unstable or volatile flows, the system should manage to quickly adapt to the trend.
• E6. In order to adapt to the functioning of individual migration systems, the system will select certain particular predictors and discard others. The selected predictors will vary across flows, as well as over time within the same flows to adapt to their dynamics. In addition to country to country flows, table A1 also includes flows from single countries of origin to the EU.

Results
We cannot use the same ranges designed for country-level flows to assess the performance of the system in forecasting EU-level asylum applications, therefore for the EU level we focus on relative errors only. All EU- our Ada-ENet model performs very well in 30 cases, well in four cases, poorly in three cases, and very poorly in six cases. In other words, within our (highly diverse) sample, seven out of ten Ada-ENet forecasts are very good, and eight out of ten are very good or good; two out of ten are poor or very poor.
In comparison, the ARIMA model performs considerably worse (although perhaps still acceptably). Out of 43 dyads, the ARIMA produces 11 very good forecasts and 12 good forecasts. Poor and very poor forecasts are eight each. In all but four cases out of 43 (or in 90% of the observed cases) Ada-ENet outperforms the ARIMA model. Of the four dyads in which ARIMA performs better, one (Eritreans to Italy) is very poor with both models, and one (Syrians to The Netherlands) is very good with both models; therefore, we can consider the improvement of ARIMA on Ada-ENet as non-significant. Significant improvements can be seen with the forecasts of Syrians to Spain and Iraqis to Greece -respectively, from very poor to poor and from poor to good.  AT  AT  BE  BE  DE  DE  EL  EL  ES  ES  FR  FR  IT  IT  NL  NL  SE  SE  EU  EU  SY  14,1  38  3,6  -7  7  113  13,2  31  47,5  60  18,8  45  75,2  11  -2,7  -12  13,7 Table M1. Parameters for the Early Warning function. Event categories within the GDELT-CAMEO event codebook, and details on the construction of the event indices for the purpose of this study: -The events selected as potential drivers of migration (GDELT code and topic); -The sign associated to single events, denoting a potential to generate (+) or constrain (-) international displacements; -The grouping factor (category label) used to aggregate selected events into the five macro-categories defined for the purposes of this study: conflict, economic events, social unrest, governance-related events, political events; -The strength of the potential displacement-generation effect associated to single events.