Near Real-Time Wildfire Progression Monitoring with Sentinel-1 SAR Time Series and Deep Learning

In recent years, the world witnessed many devastating wildfires that resulted in destructive human and environmental impacts across the globe. Emergency response and rapid response for mitigation calls for effective approaches for near real-time wildfire monitoring. Capable of penetrating clouds and smoke, and imaging day and night, Synthetic Aperture Radar (SAR) can play a critical role in wildfire monitoring. In this communication, we investigated and demonstrated the potential of Sentinel-1 SAR time series with a deep learning framework for near real-time wildfire progression monitoring. The deep learning framework, based on a Convolutional Neural Network (CNN), is developed to detect burnt areas automatically using every new SAR image acquired during the wildfires and by exploiting all available pre-fire SAR time series to characterize the temporal backscatter variations. The results show that Sentinel-1 SAR backscatter can detect wildfires and capture their temporal progression as demonstrated for three large and impactful wildfires: the 2017 Elephant Hill Fire in British Columbia, Canada, the 2018 Camp Fire in California, USA, and the 2019 Chuckegg Creek Fire in northern Alberta, Canada. Compared to the traditional log-ratio operator, CNN-based deep learning framework can better distinguish burnt areas with higher accuracy. These findings demonstrate that spaceborne SAR time series with deep learning can play a significant role for near real-time wildfire monitoring when the data becomes available at daily and hourly intervals with the launches of RADARSAT Constellation Missions in 2019, and SAR CubeSat constellations.

Reference data. High resolution Worldview-3 post-fire imagery over the Elephant Hill Fire site was acquired on September 28, 2017 and used to visually verify the SAR-based burnt area maps. Fieldwork was also conducted in the Elephant Hill Fire area in July, 2018, one-year after the wildfire. Ground truth data representing various burn severities were collected by field inspection and with a drone. Figure 2 shows various burn severities under different terrain conditions.
To verify and validate the SAR-based mapping results, cloud-free Sentinel-2 Multispectral Instrument (MSI) imagery before, during, and after the wildfires were selected in the Elephant Hill Fire in 2017, the Camp Fire in 2018 and the Chuckegg Creek Fire in 2019. For validation, burnt areas were automatic extracted using pre-fire and post-fire Normalized Burn Ratio (NBR) and their difference (denoted as dNBR) using Eqs. (1) and (2) 48 . For each study area, the accuracy of the SAR-based final burnt area map was quantitatively assessed using 20,000 validation points (10,000 points each for burnt and unburnt areas respectively) randomly selected based on the burnt area map derived from the post-fire Sentinel-2 imagery. It should be noted that the randomly selected training and validation data sets are from the same geographical region. Therefore there is a slight chance that a few of the validation samples overlap or in close vicinity of the training samples. This may affect the overall accuracy of the mapping results. Due to lack of ground truth data during the wildfires and no Sentinel-2 or Landsat imagery to couple with each SAR acquisition during the wildfires, the SAR-based progression maps were visually compared with the burnt area maps that were derived from Sentinel-2 imagery acquired right after each acquisition date of the SAR data.
pre fire p ost fire To better understand the behavior of SAR temporal backscatter under different conditions, precipitation data over the study areas was collected. For the Elephant Hill Fire, the precipitation data used is PERSIANN-CDR, a daily precipitation estimation from remotely sensed information with artificial neural network-Climate Data Record, at a resolution of 0.25 arc degrees 49,50 . For the Camp Fire, the precipitation data is from the Climate Hazards Group InfraRed Precipitation with Station data (CHIRPS), which is a 30 year quasi-global daily rainfall dataset and it incorporates 0.05° resolution satellite imagery with in-situ station data 51 .

Results and Discussion
Sentinel-1 SAR temporal backscatter patterns of burnt and unburnt vegetation. To better understand the SAR backscatter behaviors of burnt and unburnt vegetation, several areas of interest (AOI) on SAR time series representing forest and grassland in similar vegetation (pre-fire) and topographic conditions (elevation, slope, aspect) were selected for analysis and comparison of their temporal backscatter patterns. The SAR backscatter statistics corresponding to several comparable pairs of AOIs for burnt and unburnt forest and grassland is presented in Table 1. For each AOI, the location, size and topographic information as well as their temporal means and standard deviations of SAR backscatter are listed. The reason why we use differently sized www.nature.com/scientificreports www.nature.com/scientificreports/ AOIs for these two fire events is that the terrain of the Camp Fire is much more complex and locally dynamic than that of the Elephant Hill Fire. Therefore, the smaller AOIs are more homogenous and representative.
The examples of the SAR temporal backscatter behaviors of burnt and unburnt forest and grassland, corresponding to the AOIs listed in Table 1 and shown in Fig. 3. For each pair of the AOIs, the daily precipitation histograms are also shown in this figure since vegetation wetness and soil moisture conditions could impact the SAR backscatter. Figure 3(a,b) show the SAR backscatter variations over time for forest and grassland respectively in Elephant Hill Fire site while (c) and (d) display the temporal backscatter variations in the Camp Fire site. Figure 3(a) shows that, before the wildfire event on July 6, 2017, the AOIs of burnt and unburnt forest share the same SAR backscatter patterns, indicating that the forested areas have very similar vegetation and topographic conditions before the fire. After July 6th, however, the mean backscatters for both C-VH and C-VV polarization in the burnt AOI decreased approximately 2-3.5 dBs, with a larger standard deviation while the SAR mean backscatters in the unburnt AOI remain rather stable. The temporal trends of C-HVs and C-VV backscatter are very similar even though C-VV backscatter coefficients are several dBs higher than that of C-VH. The precipitation events did not affect the backscatter for either burnt or unburnt forest as the rain events were not right before the SAR data acquisitions. Similarly, burnt AOI shares similar backscatter patterns to unburnt AOI before fire in the  www.nature.com/scientificreports www.nature.com/scientificreports/ forest AOIs of the Camp Fire site. However, both C-VV and C-VH backscatter coefficients increased after the fire event, as shown in Fig. 3(c). The increase in backscatter of burnt forest is likely caused by rain events immediately before the SAR data acquisitions resulting in higher soil moisture contents in burnt areas.
For the grassland AOIs in the Elephant Hill Fire site, similar temporal backscatter patterns to forest are observed, as shown in Fig. 3(b). The mean backscatters of the burnt AOI decreased while that of unburnt AOI remain relatively stable. Again, the precipitation events did not affect the SAR backscatter as the rain events were not right before the SAR data acquisitions. For the grassland AOIs in the Camp Fire site, as shown in Fig. 3(d), obvious increases in C-VV backscatters after the start of the wildfire are observed due to rain events, similar to forest backscatter increase in the Camp Fire site. Several heavy rain events occurred between Nov. 21 and Nov. 28 of 2018, and VV is more sensitive to the soil moisture than VH in burnt areas. However, the C-VH backscatter patterns of unburnt and burnt grassland remain the same after the fire. The reasons for this need further investigation as C-band radar signature is usually not significantly influenced by dry grass subject to fire event according to literature. This observation further confirms the previous findings that identification of the burnt area for non-forest vegetation (i.e., grassland) is not so straightforward.
Sentinel-1 SAR for near real-time wildfire progression monitoring. Elephant hill fire. The SAR-based wildfire progression maps of the Elephant Hill Fire are presented in Fig. 4, estimated by different methods, including the classical log-ratio (logRt) 35 , the ratio between the logRt and the corresponding historical stdDev map (kmap) and CNN-based framework. The first three rows show the estimated change maps, while the fourth row shows the binary progression maps corresponding to the burn confidence maps predicted by CNN in the third row. Row 5, labeled as CNN_mrg, presents the merged progression maps for each date, produced by accumulating all the progression maps before a date. To reduce unnecessary noises, the merging operation was not applied on the first two dates, i.e., July 8 and July 20, 2017. In the last row, CNN_tsc_mrg denotes that a simple time series correction (TSC) was applied on the progression maps in the same orbit before merging them. It is assumed the burnt area would not disappear in the later progression once it appeared before, TSC can be used to reduce the noisy pixels in the earlier progression maps based on the later progression maps. However, with TSC, the CNN_tsc_mrg is not a near real-time approach any longer because it depends the future progression maps. For any date in the Elephant Hill Fire, the TSC was implemented by multiplying it with the future progression maps in the same orbit.
To quantitatively assess SAR-based burnt area results, Sentinel-2 dNBR is segmented into a binary map of burnt and unburnt areas and used as the reference maps together field data and WorldView-3 imagery. 10,000 validation points are randomly selected from burnt and unburnt areas respectively. Table 2 presents the quantitative evaluation of SAR-based wildfire progression results for the Elephant Hill Fire. Among logRt, kmap, CNN_mrg and CNN_tsc_mrg, CNN_mrg achieves the highest values in Precision, Recall, OA, Kappa and F 1 , CNN_tsc_mrg ranks second, and both of them are much higher than logRt and kmap-based results. It is worth noting that CNN_tsc_mrg reaches a very high value in Recall (0.9952), which implies that TSC greatly reduces the false alarm rate, compared to CNN_mrg's Recall (0.9336).
Due to lack of field data and optical images acquired on the same date as the SAR imagery during the wildfire, the progression maps are validated visually by overlaying CNN_mrg on the Sentinel-2 false color composite (R = SWIR 2 , G = SWIR 1 , B = SWIR 2 ). For each map in CNN_mrg_overlay, the Sentinel-2 image with the closest cloud-free date after the SAR acquisition. Visual observation shows that there is a high level of agreement between Sentinel-1 SAR progression map and Sentinel 2 burnt area in the full time series. The examples of the overlays are presented in Fig. 5.
Camp fire. The SAR-based wildfire progression maps of the Camp Fire are presented in Fig. 6, estimated by different methods, including logRt with absolute operation (denoted as logRt_abs), kmap and CNN burnt Confidence map (CNN burnConf). Compared with logRt_abs and kmap, we can find that CNN burnConf highlights the burnt areas very well, that is critical for subsequent segmentation. The row marked with CNN burnMap shows the corresponding binary map of CNN burnConf maps with Otsu thresholding 52 , and CNN_mrg and CNN_tsc_mrg are produced with similar procedures for the Elephant Hill Fire. For the Camp Fire, the logRt_abs is exploited to detect both positive and negative backscatter changes, due to the fact that there exist both increased and decreased backscatter changes in the fire related areas. In the logRt_abs maps, as shown in the first row in Fig. 6, are the false color composite (R = |Δβ VH |, G = |Δβ VV |, B = |Δβ VH |).
In the first four stages, most of the burnt areas are in purple, indicating that VH backscatter is more sensitive than VV to the changes caused by the wildfire event, while the white pixels indicate that VH and VV show similar sensitivity to the changes. However, in the last four stages, the burnt area appears very different, in green instead of purple. The green pixels show that VV backscatter are more sensitive than VH to the fire-induced changes. This is because several heavy precipitations occurred between Nov. 21 and Nov. 28 of 2018, and VV is more sensitive to the soil moisture than VH in burnt areas. As shown in the CNN burnConf maps, the first two stages (a) and (b) indicate that CNN helps enhance the difference between burnt and unburnt areas, but significant over and under estimations are observed. As new SAR data comes, the CNN model is fine-tuned further, then the predicted CNN burnConf maps show much better contrast between burnt and unburnt pixels than the first two stages. With Gaussian filtering followed by Otsu thresholding 52 , CNN burnConf maps in range [0, 1] are binarized into CNN burnMap. As expected, the first stage (a) looks rather noisy and the later stages detect most of the burnt areas with less noise. CNN_mrg combines all detected burnt areas on different orbits before current dates, except for the first date, Nov. 11, 2017, which is too noisy. CNN_tsc_mrg provides non near real-time results with less false alarm pixels by applying time series correction on the same orbit. By overlaying Senintel-1 SAR-based progression results (in transparent red) and Sentinel-2 SWIR composite, the bottom row demonstrates that there is a certain degree (2020) 10:1322 | https://doi.org/10.1038/s41598-019-56967-x www.nature.com/scientificreports www.nature.com/scientificreports/ of agreement between Senintel-1 SAR-based progression maps and Senintel-2 fire scars. Compared to the visual observations in the Elephant Hill Fire, the agreement on the Camp Fire is not as good.
Quantitative evaluations of SAR-based wildfire progression results of the Camp Fire are presented in Table 3. With kmap, C-VV achieves a much higher value than C-VH in OA, Kappa and F 1 score, and combining C-VV and C-VH can reach a higher accuracy than VV or VH alone. By combining VH and VV, CNN_mrg achieves an overall accuracy of 83.58% (Kappa: 0.6716, F 1 : 0.8139), and CNN_tsc_mrg reaches a higher Recall value than CNN_mrg, i.e., a lower false alarm rate.  Figure 7 presents the SAR-based wildfire progression maps of the Chunkegg Creek Fire, estimated by logRt, kmap and the proposed CNN-based deep learning framework. Different from the Elephant Hill Fire and the Camp Fire, Sentinel-1 SAR acquired images every six days in the same orbit (ASC20) over the Chuckegg Creek Fire, a higher imaging frequency. The logRt-based progression maps showed that the VV and VH backscattering have similar sensitivity to changes caused by fire, thus the burnt areas appear white. However, they have very different responses to changes caused by agricultural activities. While VH increases slightly, VV decreases significantly over the agricultural areas (in green). Similar to a wildfire event, the agricultural activities may cause a significant decrease in VV backscattering, which would result in false alarms. Owning to the fact that the agricultural fields often have a high standard deviation in the historical time series, kmap can suppress the agricultural activities-related changes better than log ratio, as shown in the second row. Trained with samples from the binarized kmap, the CNN-based framework can highlight the burnt areas and suppress false alarms due to agricultural activities, as shown in CNN_burnConf and CNN_burnMap. CNN_mrg and CNN_tsc_mrg show the merged results of the burnt areas without or with TSC respectively. The bottom row shows the visual comparison between optical images and SAR-based results, which indicate that SAR data has the potential to detect most of the burnt areas, but some low burn severity areas without structural changes may be missed. Table 4 summarizes the quantitative analysis of the SAR-based wildfire progression mapping results, and these statistics are based on 10,000 samples randomly selected from burnt areas and unburnt areas respectively. With kmap, both VH and VV reach a very high Recall value but a low Precision value, indicating that both of them have a very low false negative rate and high false positive rate. By combining VH and VV together, kmap achieves a much higher precision without a significant decrease in Recall, resulting in the increase in OA (73.05%), Kappa (0.4609) and F1 (0.6339). By applying the proposed CNN-based framework using both VH and VV data, CNN_ mrg can achieve a significant improvement in Precision with a minor decrease (0.3%) in Recall, leading to 88.09% in OA, 0.7618 in Kappa and 0.8666 in F1 score. By exploiting TSC, CNN_tsc_mrg can reduce the noisy pixels very well, but the accuracy decreases slightly. www.nature.com/scientificreports www.nature.com/scientificreports/

Conclusions
In this paper, we evaluated Senitnel-1 SAR time series for near real-time wildfire progression monitoring using a novel and fully automatic deep learning framework based on CNN. The analysis of SAR temporal backscatter profiles showed that significant differences between burnt forest and grassland can be observed in both the Elephant Hill Fire and the Camp Fire sites (except C-VH over the Camp Fire site). The CNN-based deep learning framework performed much better than log-ratio based kmap in detecting burnt areas, achieving a significant improvement in Kappa over these three study areas: (0.11 for the Elephant Hill Fire, 0.27 for the Camp Fire and 0.30 for the Chuckegg Creek Fire, respectively). By fine-tuning with local data, we demonstrate the proposed CNN framework is effective in monitoring the progressions of three large wildfires in different geographic regions      www.nature.com/scientificreports www.nature.com/scientificreports/ in various topographic conditions. Additional studies are planned to further demonstrate the transferability of the CNN framework to other wildfire events via pixel-wise network forward propagation. By exploiting all the available SAR data acquired before the wildfire event to characterize the area in terms of backscatter variations due to different environmental conditions, the time series based anomaly detection method is effective in producing coarse burnt area maps that are essential for automatic training of the CNN framework. This research is the first attempt on wildfire progression monitoring using SAR time series and deep learning in challenging topographic conditions. The findings demonstrates that, using a fully automatic deep learning framework, spaceborne SAR data can play a significant role for real-time wildfire progression monitoring when the data becomes available at daily and hourly intervals with the launches of RADARSAT Constellation Missions and SAR CubeSat constellations.

Methodology
The main goal of the methodology is to develop a novel and fully automatic procedure based on a deep learning framework that utilizes every new Sentinel-1 SAR image acquired during the wildfire event to monitor the fire progression in near real-time. When a wildfire occurs, pre-fire SAR dense time series of the study area are collected from the archive and new SAR images are acquired in near real-time during the wildfire event. In particular, the proposed method has two innovative aspects, one is to exploit all available SAR data acquired before the wildfire event to characterize the area in terms of SAR backscatter variations due to different environmental conditions (e.g., seasonal effect, different land cover, weather conditions, etc.) while the other is to automatically train an implicit deep learning framework to estimate the changes in the SAR images acquired during the wildfire. The methodology includes four major processing steps. First, log-ratio of the pre-fire and post-fire SAR images is performed to detect changes caused by wildfire. Then the coarse binary map of burnt and unburnt areas is generated using a time series based anomaly detection technique. Using training samples automatically generated from the coarse binary change map, the CNN is trained and fitted to refine the burnt area detection and to generate the burnt confidence maps. The last step is to binarize the confidence maps using the Otsu automatic thresholding approach and to combine the individual wildfire progression maps progressively to improve their reliability and consistency. The overview of the methodology is presented in Fig. 8. In the following sub-sections, a full description of the different steps is reported.
Log-ratio based change measurement. To detect changes caused by the wildfire. comparison of preand post-fire SAR images is performed. For each new SAR image acquired after the start of the wildfire, a pre-fire image is selected as a master image for each available ascending (ASC) or descending (DSC) orbit. By applying log-ratio operator on the master (pre-fire) and slave (after the start of the fire) image, a change map can be derived for C-VV and C-VH respectively, and the corresponding log-ratio time series can be established. The optimal master image is selected taking into account both minimizing the seasonal effects and avoiding master images acquired after heavy rain events. Log-Ratio based change measurement is defined accordingly with the following formula: where r ∈ {ASC-μ, DSC-ν} denote the corresponding orbit direction (ASC or DSC) and relative start orbit number (μ or ν), and β r is the radar backscattering value, m and s represent pre-fire (master) and post-fire (slave) image, respectively. By applying log-ratio on the master and slave images, the change map Δβ r can be derived, which estimates the difference degree between master and slave images. The change map Δβ r are subsequently binarized using the StdDev map as a reference estimation of the regular oscillation of the SAR backscatter.
Hereafter, Δβ r and σ r are rewritten as Δβ and σ for convenient and compact mathematical notation.

Time series based anomaly detection.
Based on the pre-fire SAR time series, the corresponding mean and standard deviation (StdDev) are computed with respect to every pixel, forming a mean map and a StdDev map. Over the same study area, the StdDev maps σ are computed based on the historical SAR time series for ASC and DSC orbit respectively, including both VH and VV polarizations. The StdDev maps estimate the normal variance of SAR backscatter with seasonal changes over time, which would provide a pixel-wise reference level for different land cover types in the study area.
where β represents the mean image over the available SAR time series on the same orbit, and the length is denoted as N, i.e., the total number of available images acquired on the same orbit.
k In order to detect the abnormal variance caused by wildfire events, a k-Map can be computed by dividing σ from |Δβ(i, j)|, as shown in Eq. 5. The k-map I k estimates the times that |Δβ(i, j)| is larger than the corresponding σ for each pixel in the study area: a higher value in the k-Map means it is abnormal variation corresponding to a higher probability of changes. As illustrated in Eq. (6), the I k map can be transformed into a binary map I with a threshold k 0 , which means the pixels will be considered as abnormal ones (i.e., burnt pixels) if they are larger than the k 0 value in the estimated log-ratio map, otherwise, they will be taken as the normal ones, i.e., unburnt pixels. In practice, k 0 = 2 is a good trade-off between detecting abnormal changes and suppressing noise. The produced binary maps are the main input in the next CNN refinement step as reference data to automatically select the training samples.
Deep learning-based burnt area refinement. As shown in Fig. 8, we use the binary logRt map time series to select burnt and unburnt samples for training a CNN model to detect burnt area automatically. By iterating over the available dates, each image is stacked with the corresponding master image and StdDev map in the same orbit, and the DEM products such as elevation, slope and aspects can also be stacked on them. The same number of training samples are randomly chosen from the burnt areas and unburnt areas identified by the log-ratio algorithm with StdDev binarization, and these training samples are stored into a database, used to train a CNN model to further refine the burnt areas. In the testing phase, when the image stack is fed into this CNN model, the corresponding burnt area mapping will be generated automatically.
The CNN framework is designed to produce a confidence map characterized by a bi-modal distribution of burn and un-burnt pixels. Let F(θ) denotes the learnable deep network for detecting burnt areas, we can derive the output O i l of the first l layers by forward passing patch P i in network F(θ) (short for F L (θ), where L is the total number of network layers), and the forward passing is denoted as ⊗ in Eq. (7).
where W l and b l represent weights and bias of l-th layer in F(θ), respectively, and δ is the ReLU activation function 38 . Table 5 lists the CNN architecture used, it has 18 layers of neural networks, and each convolutional layer is followed by a ReLU activation (sigmoid for the last layer). The CNN burnConf map is derived by applying the sigmoid activation on the output of the last layer, since burnt area detection is actually a binary classification problem, and the sigmoid activation is a good choice to scale the predicted confidence into the range [0, 1].
With randomly sampled data (P i , y i ), a CNN-based non-linear change indicator F(θ) can be learnt for highlighting the burnt areas based on the SAR data. The predicted burn confidence vector ∈ × R y i 2 1 can be derived by . Therefore, the loss function can be formulated as: here n is the number of training samples, θ is the learn-able network parameter over all layers, including weights and bias, and λ controls the weight decay rate. In our experiment, we set λ = 0.001. Once trained, the corresponding burn confidence map can be obtained, which can be used to update the binary logRt time series for next training. Like this, the Pseudo label updating can contribute to providing more reliable ones, but it is not necessary. Moreover, Digital Elevation Model (DEM) products can be integrated to take topography into consideration as additional input layers. CNN BurnConf maps: binarization and time series merging. The outputs of the CNN refinement are CNN burnConf maps, where the pixel values are ranging from 0 to 1 and they are proportional to the probability that each pixel represent a burnt area (0: unburnt, 1: burnt). The main advantage of using the proposed CNN framework is that the differences in term of backscatter variation between burnt and unburnt pixel are represented by a clear bi-modal distribution (see Fig. 9) with respect to unimodal distribution of the log-ratio based results (scaled to [0, 1] by dividing the maximum). Consequently, the CNN burnConf maps are easy to be binarized using an Otsu automatic thresholding technique 52 .
To produce more reliable and consistent fire progression maps, the binary wildfire progression maps at different stages are combined using two different methods. The first method is for near-real time wildfire progression monitoring and it simply combines the new wildfire progression map with the previous ones to generate the latest burnt area map. This method does not use the later burnt map to improve the results of the previous ones. This method has been investigated to highlight the potential of the SAR-based CNN framework for near real-time wildfire monitoring. The second method is a post-processing step that uses all the generated binary maps to update the fire progression maps exploiting the available multitemporal information. The method reduces the noise using a gaussian temporal filtering of the produced burnt map time series and it can be used to obtain a more reliable delineation of the fire progression for post-fire analysis (i.e. calibration of fire progression models).

Data availability
The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.