Process-based modelling offers interpretability and physical consistency in many domains of geosciences but struggles to leverage large datasets efficiently. Machine-learning methods, especially deep networks, have strong predictive skills yet are unable to answer specific scientific questions. In this Perspective, we explore differentiable modelling as a pathway to dissolve the perceived barrier between process-based modelling and machine learning in the geosciences and demonstrate its potential with examples from hydrological modelling. ‘Differentiable’ refers to accurately and efficiently calculating gradients with respect to model variables or parameters, enabling the discovery of high-dimensional unknown relationships. Differentiable modelling involves connecting (flexible amounts of) prior physical knowledge to neural networks, pushing the boundary of physics-informed machine learning. It offers better interpretability, generalizability, and extrapolation capabilities than purely data-driven machine learning, achieving a similar level of accuracy while requiring less training data. Additionally, the performance and efficiency of differentiable models scale well with increasing data volumes. Under data-scarce scenarios, differentiable models have outperformed machine-learning models in producing short-term dynamics and decadal-scale trends owing to the imposed physical constraints. Differentiable modelling approaches are primed to enable geoscientists to ask questions, test hypotheses, and discover unrecognized physical relationships. Future work should address computational challenges, reduce uncertainty, and verify the physical significance of outputs.
This is a preview of subscription content, access via your institution
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 digital issues and online access to articles
$99.00 per year
only $8.25 per issue
Rent or buy this article
Prices vary by article type
Prices may be subject to local taxes which are calculated during checkout
Ajami, N. K., Gupta, H., Wagener, T. & Sorooshian, S. Calibration of a semi-distributed hydrologic model for streamflow estimation along a river system. J. Hydrol. 298, 112–135 (2004).
van Griensven, A. & Meixner, T. A global and efficient multi-objective auto-calibration and uncertainty estimation method for water quality catchment models. J. Hydroinform. 9, 277–291 (2007).
Barendrecht, M. H. et al. The value of empirical data for estimating the parameters of a sociohydrological flood risk model. Water Resour. Res. 55, 1312–1336 (2019).
Post, H., Vrugt, J. A., Fox, A., Vereecken, H. & Franssen, H.-J. H. Estimation of Community Land Model parameters for an improved assessment of net carbon fluxes at European sites. J. Geophys. Res. Biogeosci. 122, 661–689 (2017).
Aumont, O., Ethé, C., Tagliabue, A., Bopp, L. & Gehlen, M. PISCES-v2: an ocean biogeochemical model for carbon and ecosystem studies. Geosci. Model. Dev. 8, 2465–2513 (2015).
Ahmed, M. et al. Calibration and validation of APSIM-Wheat and CERES-Wheat for spring wheat under rainfed conditions: models evaluation and application. Comput. Electron. Agric. 123, 384–401 (2016).
Lepore, C., Arnone, E., Noto, L. V., Sivandran, G. & Bras, R. L. Physically based modeling of rainfall-triggered landslides: a case study in the Luquillo forest, Puerto Rico. Hydrol. Earth Syst. Sci. 17, 3371–3387 (2013).
Shirzaei, M. et al. Measuring, modelling and projecting coastal land subsidence. Nat. Rev. Earth Environ. 2, 40–58 (2021).
Biemans, H. et al. Importance of snow and glacier meltwater for agriculture on the Indo-Gangetic Plain. Nat. Sustain. 2, 594–601 (2019).
Li, L. et al. Toward catchment hydro-biogeochemical theories. WIREs Water 8, e1495 (2021).
Steefel, C. I. et al. Reactive transport codes for subsurface environmental simulation. Comput. Geosci. 19, 445–478 (2015).
Li, L. et al. Expanding the role of reactive transport models in critical zone processes. Earth Sci. Rev. 165, 280–301 (2017).
Flato, G. M. Earth system models: an overview. WIREs Clim. Change 2, 783–800 (2011).
Danabasoglu, G. et al. The Community Earth System Model version 2 (CESM2). J. Adv. Modeling Earth Syst. 12, e2019MS001916 (2020).
Eyring, V. et al. Overview of the Coupled Model Intercomparison Project phase 6 (CMIP6) experimental design and organization. Geosci. Model. Dev. 9, 1937–1958 (2016).
Calvin, K. et al. GCAM v5.1: representing the linkages between energy, water, land, climate, and economic systems. Geosci. Model. Dev. 12, 677–698 (2019).
ISIMIP. The Inter-Sectoral Impact Model Intercomparison Project (ISIMIP). https://www.isimip.org/ (2022).
Lange, S. Trend-preserving bias adjustment and statistical downscaling with ISIMIP3BASD (v1.0). Geosci. Model. Dev. 12, 3055–3070 (2019).
Weyant, J. et al. in Climate Change 1995: Social and Economic Dimensions of Climate Change (eds Bruce, J. P., Lee, H. & Haites, E. F.) 367–396 (IPCC, Cambridge Univ. Press, 1996).
IPCC. Climate Change 2021: The Physical Science Basis (eds Masson-Delmotte, V. et al.) (Cambridge Univ. Press, 2021).
Clark, M. P. et al. Improving the representation of hydrologic processes in Earth system models. Water Resour. Res. 51, 5929–5956 (2015).
Geary, W. L. et al. A guide to ecosystem models and their environmental applications. Nat. Ecol. Evol. 4, 1459–1471 (2020).
Fatichi, S. et al. An overview of current applications, challenges, and future trends in distributed process-based models in hydrology. J. Hydrol. 537, 45–60 (2016).
Wagener, T. et al. On doing hydrology with dragons: realizing the value of perceptual models and knowledge accumulation. WIREs Water 8, e1550 (2021).
Hood, R. R. et al. The Chesapeake Bay program modeling system: overview and recommendations for future development. Ecol. Model. 456, 109635 (2021).
Fan, Y. et al. Hillslope hydrology in global change research and Earth system modeling. Water Resour. Res. 55, 1737–1772 (2019).
van Kampenhout, L. et al. Improving the representation of polar snow and firn in the Community Earth System Model. J. Adv. Modeling Earth Syst. 9, 2583–2600 (2017).
Medlyn, B. E. et al. Using ecosystem experiments to improve vegetation models. Nat. Clim. Change 5, 528–534 (2015).
Nearing, G. S. et al. What role does hydrological science play in the age of machine learning? Water Resour. Res. 57, e2020WR028091 (2021).
Shen, C. et al. HESS Opinions: incubating deep-learning-powered hydrologic science advances as a community. Hydrol. Earth Syst. Sci. 22, 5639–5656 (2018).
Hunt, R. J., Fienen, M. N. & White, J. T. Revisiting ‘An exercise in groundwater model calibration and prediction’ after 30 years: insights and new directions. Groundwater 58, 168–182 (2020).
Addor, N. & Melsen, L. A. Legacy, rather than adequacy, drives the selection of hydrological models. Water Resour. Res. 55, 378–390 (2019).
Clark, M. P., Kavetski, D. & Fenicia, F. Pursuing the method of multiple working hypotheses for hydrological modeling. Water Resour. Res. 47, WR009827 (2011).
Jakeman, A. J. & Hornberger, G. M. How much complexity is warranted in a rainfall-runoff model? Water Resour. Res. 29, 2637–2649 (1993).
Wagener, T., Wheater, H. S. & Gupta, H. V. in Calibration of Watershed Models 29–47 (Wiley, 2003).
Young, P., Jakeman, A. & McMurtrie, R. An instrumental variable method for model order identification. Automatica 16, 281–294 (1980).
Shen, C. A transdisciplinary review of deep learning research and its relevance for water resources scientists. Water Resour. Res. 54, 8558–8593 (2018).
Abbott, B. W. et al. Human domination of the global water cycle absent from depictions and perceptions. Nat. Geosci. 12, 533–540 (2019).
Lemordant, L., Gentine, P., Swann, A. S., Cook, B. I. & Scheff, J. Critical impact of vegetation physiology on the continental hydrologic cycle in response to increasing CO2. Proc. Natl Acad. Sci. USA 115, 4093–4098 (2018).
Trancoso, R., Larsen, J. R., McVicar, T. R., Phinn, S. R. & McAlpine, C. A. CO2–vegetation feedbacks and other climate changes implicated in reducing base flow. Geophys. Res. Lett. 44, 2310–2318 (2017).
Yu, D. et al. Socio-hydrology: an interplay of design and self-organization in a multilevel world. Ecol. Soc. 25, 22 (2020).
LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).
Hochreiter, S. & Schmidhuber, J. Long short-term memory. Neural Comput. 9, 1735–1780 (1997).
Yin, H., Guo, Z., Zhang, X., Chen, J. & Zhang, Y. RR-Former: rainfall-runoff modeling based on transformer. J. Hydrol. 609, 127781 (2022).
Amanambu, A. C., Mossa, J. & Chen, Y.-H. Hydrological drought forecasting using a deep transformer model. Water 14, 3611 (2022).
Sun, A. Y., Jiang, P., Mudunuru, M. K. & Chen, X. Explore spatio-temporal learning of large sample hydrology using graph neural networks. Water Resour. Res. 57, e2021WR030394 (2021).
Krizhevsky, A., Sutskever, I. & Hinton, G. E. in Advances in Neural Information Processing Systems vol. 25 (eds Bartlett, P. et al.) 1097–1105 (Curran Associates, 2012).
Lecun, Y. & Bengio, Y. Convolutional networks for images, speech, and time-series. in The Handbook of Brain Theory and Neural Networks (ed. Arbib, M. A.) 255–258 (MIT Press, 1995).
Khaki, S. & Wang, L. Crop yield prediction using deep neural networks. Front. Plant Sci. 10, 00621 (2019).
Wang, A. X., Tran, C., Desai, N., Lobell, D. & Ermon, S. Deep transfer learning for crop yield prediction with remote sensing data. In Proceedings of the 1st ACM SIGCAS Conference on Computing and Sustainable Societies, 1–5 (ACM, 2018).
Pan, B. et al. Improving seasonal forecast using probabilistic deep learning. J. Adv. Model. Earth Syst. 14, e2021MS002766 (2022).
Shi, X. et al. Convolutional LSTM network: a machine learning approach for precipitation nowcasting. In Proceedings of the 28th International Conference on Neural Information Processing Systems Vol. 1, 802–810 (MIT Press, 2015).
Bhowmik, M., Singh, M., Rao, S. & Paul, S. DeepClouds.ai: deep learning enabled computationally cheap direct numerical simulations. Preprint at https://doi.org/10.48550/arXiv.2208.08956 (2022).
Lin, G.-Y., Chen, H.-W., Chen, B.-J. & Yang, Y.-C. Characterization of temporal PM2.5, nitrate, and sulfate using deep learning techniques. Atmos. Pollut. Res. 13, 101260 (2022).
Varadharajan, C. et al. Can machine learning accelerate process understanding and decision-relevant predictions of river water quality? Hydrol. Process. 36, e14565 (2022).
Jia, X. et al. Physics-guided recurrent graph model for predicting flow and temperature in river networks. In Proceedings of the 2021 SIAM International Conference on Data Mining (SDM), 612–620 (Society for Industrial and Applied Mathematics, 2021).
Rahmani, F. et al. Exploring the exceptional performance of a deep learning stream temperature model and the value of streamflow data. Environ. Res. Lett. https://doi.org/10.1088/1748-9326/abd501 (2021).
Rahmani, F., Shen, C., Oliver, S., Lawson, K. & Appling, A. Deep learning approaches for improving prediction of daily stream temperature in data-scarce, unmonitored, and dammed basins. Hydrol. Process. 35, e14400 (2021).
Read, J. S. et al. Process-guided deep learning predictions of lake water temperature. Water Resour. Res. 55, 9173–9190 (2019).
Zhi, W. et al. From hydrometeorology to river water quality: can a deep learning model predict dissolved oxygen at the continental scale? Environ. Sci. Technol. 55, 2357–2368 (2021).
Zhi, W., Ouyang, W., Shen, C. & Li, L. Temperature outweighs light and flow as the predominant driver of dissolved oxygen in US rivers. Nat. Water 1, 249–260 (2023).
He, M., Wu, S., Huang, B., Kang, C. & Gui, F. Prediction of total nitrogen and phosphorus in surface water by deep learning methods based on multi-scale feature extraction. Water 14, 1643 (2022).
Hrnjica, B., Mehr, A. D., Jakupović, E., Crnkić, A. & Hasanagić, R. Application of deep learning neural networks for nitrate prediction in the Klokot River, Bosnia and Herzegovina. In 7th International Conference on Control, Instrumentation and Automation (ICCIA) 1–6 (IEEE, 2021).
Xiong, R. et al. Predicting dynamic riverine nitrogen export in unmonitored watersheds: leveraging insights of AI from data-rich regions. Environ. Sci. Technol. 56, 10530–10542 (2022).
Shen, C., Chen, X. & Laloy, E. Editorial: broadening the use of machine learning in hydrology. Front. Water, https://doi.org/10.3389/frwa.2021.681023 (2021).
Fang, K., Shen, C., Kifer, D. & Yang, X. Prolongation of SMAP to spatiotemporally seamless coverage of continental U.S. using a deep learning neural network. Geophys. Res. Lett. 44, 11030–11039 (2017).
Fang, K., Pan, M. & Shen, C. The value of SMAP for long-term soil moisture estimation with the help of deep learning. IEEE Trans. Geosci. Remote. Sens. 57, 2221–2233 (2019).
Fang, K. & Shen, C. Near-real-time forecast of satellite-based soil moisture using long short-term memory with an adaptive data integration kernel. J. Hydrometeorol. 21, 399–413 (2020).
Feng, D., Fang, K. & Shen, C. Enhancing streamflow forecast and extracting insights using long-short term memory networks with data integration at continental scales. Water Resour. Res. 56, e2019WR026793 (2020).
Kratzert, F. et al. Towards learning universal, regional, and local hydrological behaviors via machine learning applied to large-sample datasets. Hydrol. Earth Syst. Sci. 23, 5089–5110 (2019).
Xiang, Z. & Demir, I. Distributed long-term hourly streamflow predictions using deep learning — a case study for State of Iowa. Environ. Model. Softw. 131, 104761 (2020).
Alemohammad, S. H. et al. Water, energy, and carbon with artificial neural networks (WECANN): a statistically based estimate of global surface turbulent fluxes and gross primary productivity using solar-induced fluorescence. Biogeosciences 14, 4101–4124 (2017).
Jung, M. et al. The FLUXCOM ensemble of global land-atmosphere energy fluxes. Sci. Data 6, 74 (2019).
Zhao, W. L. et al. Physics-constrained machine learning of evapotranspiration. Geophys. Res. Lett. 46, 14496–14507 (2019).
Afzaal, H., Farooque, A. A., Abbas, F., Acharya, B. & Esau, T. Groundwater estimation from major physical hydrology components using artificial neural networks and deep learning. Water 12, 5 (2020).
Meyal, A. Y. et al. Automated cloud based long short-term memory neural network based SWE prediction. Front. Water 2, 574917 (2020).
McDonnell, J. J. & Beven, K. Debates — the future of hydrological sciences: a (common) path forward? A call to action aimed at understanding velocities, celerities and residence time distributions of the headwater hydrograph. Water Resour. Res. 50, 5342–5350 (2014).
Appling, A. P., Oliver, S. K., Read, J. S., Sadler, J. M. & Zwart, J. Machine learning for understanding inland water quantity, quality, and ecology. earthArXiv preprint at https://doi.org/10.1016/B978-0-12-819166-8.00121-3 (2022).
Fang, K., Kifer, D., Lawson, K., Feng, D. & Shen, C. The data synergy effects of time-series deep learning models in hydrology. Water Resour. Res. 58, e2021WR029583 (2022).
McGovern, A., Ebert-Uphoff, I., Gagne, D. J. & Bostrom, A. Why we need to focus on developing ethical, responsible, and trustworthy artificial intelligence approaches for environmental science. Environ. Data Sci. 1, e6 (2022).
Schölkopf, B. in Probabilistic and Causal Inference: The Works of Judea Pearl vol. 36, 765–804 (Association for Computing Machinery, 2022).
Bach, S. et al. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE 10, e0130140 (2015).
Montavon, G., Samek, W. & Müller, K.-R. Methods for interpreting and understanding deep neural networks. Digit. Signal Process. 73, 1–15 (2018).
Toms, B. A., Barnes, E. A. & Ebert-Uphoff, I. Physically interpretable neural networks for the geosciences: applications to Earth system variability. J. Adv. Modeling Earth Syst. 12, e2019MS002002 (2020).
Fleming, S. W., Watson, J. R., Ellenson, A., Cannon, A. J. & Vesselinov, V. C. Machine learning in Earth and environmental science requires education and research policy reforms. Nat. Geosci. 14, 878–880 (2021).
Hornik, K. Approximation capabilities of multilayer feedforward networks. Neural Netw. 4, 251–257 (1991).
Hornik, K., Stinchcombe, M. & White, H. Multilayer feedforward networks are universal approximators. Neural Netw. 2, 359–366 (1989).
Bubeck, S. et al. Sparks of artificial general intelligence: early experiments with GPT-4. Preprint at https://arxiv.org/abs/2303.12712 (2023).
Deb, K., Pratap, A., Agarwal, S. & Meyarivan, T. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evolut. Comput. 6, 182–197 (2002).
Duan, Q., Sorooshian, S. & Gupta, V. Effective and efficient global optimization for conceptual rainfall-runoff models. Water Resour. Res. 28, 1015–1031 (1992).
Zitzler, E., Laumanns, M. & Thiele, L. SPEA2: Improving the Strength Pareto Evolutionary Algorithm. TIK Report vol. 103 (ETH Zurich, 2001); https://www.research-collection.ethz.ch/handle/20.500.11850/145755.
Liu, S. et al. A hybrid approach of support vector regression with genetic algorithm optimization for aquaculture water quality prediction. Math. Comput. Model. 58, 458–465 (2013).
Zambrano-Bigiarini, M. & Rojas, R. A model-independent particle swarm optimisation software for model calibration. Environ. Model. Softw. 43, 5–25 (2013).
Baydin, A. G., Pearlmutter, B. A., Radul, A. A. & Siskind, J. M. Automatic differentiation in machine learning: a survey. J. Mach. Learn. Res. 18, 1–43 (2018).
Innes, M. et al. A differentiable programming system to bridge machine learning and scientific computing. Preprint at https://arxiv.org/abs/1907.07587 (2019).
Goodfellow, I., Bengio, Y. & Courville, A. in Deep Learning (ed. Goodfellow, I.) Ch. 4 (MIT Press, 2016).
Paszke, A. et al. Automatic differentiation in PyTorch. In 31st Conference on Neural Information Processing Systems (NIPS 2017) (NIPS, 2017).
Bradbury, J. et al. JAX: Autograd and XLA. Astrophysics Source Code Library record ascl:2111.002 (Astrophysics Source Code Library, 2021).
Bezanson, J., Edelman, A., Karpinski, S. & Shah, V. B. Julia: a fresh approach to numerical computing. SIAM Rev. 59, 65–98 (2017).
Abadi, M. et al. Tensorflow: a system for large-scale machine learning. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16) 265–283 (USENIX Association, 2016).
Errico, R. M. What is an adjoint model? Bull. Am. Meteorol. Soc. 78, 2577–2592 (1997).
Johnson, S. G. Notes on Adjoint Methods for 18.335 (2021).
Pal, A., Edelman, A. & Rackauckas, C. Mixing implicit and explicit deep learning with skip DEQs and infinite time neural ODEs (continuous DEQs). Preprint at https://doi.org/10.48550/arXiv.2201.12240 (2022).
Ghattas, O. & Willcox, K. Learning physics-based models from data: perspectives from inverse problems and model reduction. Acta Numerica 30, 445–554 (2021).
Baker, N. et al. Workshop Report on Basic Research Needs for Scientific Machine Learning: Core Technologies for Artificial Intelligence. https://www.osti.gov/biblio/1478744 (2019).
Rackauckas, C. et al. Universal differential equations for scientific machine learning. Preprint at https://doi.org/10.48550/arXiv.2001.04385 (2021).
Feng, D., Liu, J., Lawson, K. & Shen, C. Differentiable, learnable, regionalized process-based models with multiphysical outputs can approach state-of-the-art hydrologic prediction accuracy. Water Resour. Res. 58, e2022WR032404 (2022).
Huang, D. Z., Xu, K., Farhat, C. & Darve, E. Learning constitutive relations from indirect observations using deep neural networks. J. Comput. Phys. 416, 109491 (2020).
Tartakovsky, A. M., Marrero, C. O., Perdikaris, P., Tartakovsky, G. D. & Barajas‐Solano, D. Physics-informed deep neural networks for learning parameters and constitutive relationships in subsurface flow problems. Water Resour. Res. 56, e2019WR026731 (2020).
Padarian, J., McBratney, A. B. & Minasny, B. Game theory interpretation of digital soil mapping convolutional neural networks. Soil 6, 389–397 (2020).
Udrescu, S.-M. & Tegmark, M. AI Feynman: a physics-inspired method for symbolic regression. Sci. Adv. 6, eaay2631 (2020).
Ma, Y., Tsao, D. & Shum, H.-Y. On the principles of parsimony and self-consistency for the emergence of intelligence. Front. Inf. Technol. Electron. Eng. 23, 1298–1323 (2022).
Haber, E. & Ruthotto, L. Stable architectures for deep neural networks. Inverse Probl. 34, 014004 (2018).
Chen, R. T. Q., Rubanova, Y., Bettencourt, J. & Duvenaud, D. Neural ordinary differential equations. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, 6572–6583 (Curran Associates, 2018).
Mahecha, M. D. et al. Earth system data cubes unravel global multivariate dynamics. Earth Syst. Dyn. 11, 201–234 (2020).
Myneni, R., Knyazikhin, Y. & Park, T. MCD15A2H MODIS/Terra+Aqua Leaf Area Index/FPAR 8-day L4 Global 500m SIN Grid V006 (2015); https://doi.org/10.5067/MODIS/MCD15A2H.006.
About SMOS — Soil Moisture and Ocean Salinity Mission (ESA, 2022); https://earth.esa.int/eogateway/missions/smos.
O’Neill, P. E. et al. SMAP enhanced L3 radiometer global and polar grid daily 9 km EASE-grid soil moisture, Version 5 (SPL3SMP_E) (2021); https://doi.org/10.5067/4DQ54OUIJ9DL.
Lin, Y.-S. et al. Optimal stomatal behaviour around the world. Nat. Clim. Change 5, 459–464 (2015).
Feng, D., Lawson, K. & Shen, C. Mitigating prediction error of deep learning streamflow models in large data-sparse regions with ensemble modeling and soft data. Geophys. Res. Lett. 48, e2021GL092999 (2021).
Feng, D., Beck, H., Lawson, K. & Shen, C. The suitability of differentiable, learnable hydrologic models for ungauged regions and climate change impact assessment. Hydrology and Earth System Sciences Discussions 1–28 (European Geoscience Union, 2022); https://doi.org/10.5194/hess-2022-245.
Wagener, T. et al. The future of hydrology: an evolving science for a changing world. Water Resour. Res. 46, 1–10 (2010).
Liu, J., Hughes, D., Rahmani, F., Lawson, K. & Shen, C. Evaluating a global soil moisture dataset from a multitask model (GSM3 v1.0) with potential applications for crop threats. Geosci. Model. Dev. 16, 1553–1567 (2023).
Tsai, W.-P. et al. From calibration to parameter learning: harnessing the scaling effects of big data in geoscientific modeling. Nat. Commun. 12, 5988 (2021).
Jiang, S., Zheng, Y. & Solomatine, D. Improving AI system awareness of geoscience knowledge: Symbiotic integration of physical approaches and deep learning. Geophys. Res. Lett. 47, e2020GL088229 (2020).
Beven, K. A manifesto for the equifinality thesis. J. Hydrol. 320, 18–36 (2006).
Pokhrel, P., Gupta, H. V. & Wagener, T. A spatial regularization approach to parameter estimation for a distributed watershed model. Water Resour. Res. 44, WR006615 (2008).
Wagener, T., McIntyre, N., Lees, M. J., Wheater, H. S. & Gupta, H. V. Towards reduced uncertainty in conceptual rainfall-runoff modelling: dynamic identifiability analysis. Hydrol. Process. 17, 455–476 (2003).
Onken, D. & Ruthotto, L. Discretize-optimize vs. optimize-discretize for time-series regression and continuous normalizing flows. Preprint at https://arxiv.org/abs/2005.13420 (2020).
Mitusch, S. K., Funke, S. W. & Kuchta, M. Hybrid FEM-NN models: combining artificial neural networks with the finite element method. J. Comput. Phys. 446, 110651 (2021).
Farrell, P. E., Ham, D. A., Funke, S. W. & Rognes, M. E. Automated derivation of the adjoint of high-level transient finite element programs. SIAM J. Sci. Comput. 35, C369–C393 (2013).
Fisher, M. & Andersson, E. Developments in 4D-Var and Kalman Filtering. https://www.ecmwf.int/sites/default/files/elibrary/2001/9409-developments-4d-var-and-kalman-filtering.pdf (2001).
Neupauer, R. M. & Wilson, J. L. Adjoint-derived location and travel time probabilities for a multidimensional groundwater system. Water Resour. Res. 37, 1657–1668 (2001).
Clark, M. P. & Kavetski, D. Ancient numerical daemons of conceptual hydrological modeling: 1. Fidelity and efficiency of time stepping schemes. Water Resour. Res. 46, W10510 (2010).
Kavetski, D. & Clark, M. P. Ancient numerical daemons of conceptual hydrological modeling: 2. Impact of time stepping schemes on model analysis and prediction. Water Resour. Res. 46, W10511 (2010).
Höge, M., Scheidegger, A., Baity-Jesi, M., Albert, C. & Fenicia, F. Improving hydrologic models for predictions and process understanding using neural ODEs. Hydrol. Earth Syst. Sci. 26, 5085–5102 (2022).
Aboelyazeed, D. et al. A differentiable ecosystem modeling framework for large-scale inverse problems: demonstration with photosynthesis simulations. Biogeosci. Discuss. https://doi.org/10.5194/bg-2022-211 (2022).
Bao, T. et al. Partial differential equation driven dynamic graph networks for predicting stream water temperature. in 2021 IEEE International Conference on Data Mining (ICDM) 11–20 (IEEE, 2021); https://doi.org/10.1109/ICDM51629.2021.00011.
Bindas, T. et al. Improving large-basin streamflow simulation using a modular, differentiable, learnable graph model for routing. Preprint at https://doi.org/10.1002/essoar.10512512.1 (2023).
Forghani, M. et al. Application of deep learning to large scale riverine flow velocity estimation. Stoch. Environ. Res. Risk Assess. 35, 1069–1088 (2021).
Forghani, M. et al. Variational encoder geostatistical analysis (VEGAS) with an application to large scale riverine bathymetry. Adv. Water Resour. 170, 104323 (2022).
Asher, M. J., Croke, B. F. W., Jakeman, A. J. & Peeters, L. J. M. A review of surrogate models and their application to groundwater modeling. Water Resour. Res. 51, 5957–5973 (2015).
Blechschmidt, J. & Ernst, O. G. Three ways to solve partial differential equations with neural networks — a review. GAMM-Mitteilungen 44, e202100006 (2021).
Lu, L., Meng, X., Mao, Z. & Karniadakis, G. E. DeepXDE: a deep learning library for solving differential equations. SIAM Rev. 63, 208–228 (2021).
Takamoto, M. et al. PDEBENCH: an extensive benchmark for scientific machine learning. Preprint at https://arxiv.org/abs/2210.07182 (2022).
Maxwell, R. M., Condon, L. E. & Melchior, P. A physics-informed, machine learning emulator of a 2D surface water model: what temporal networks and simulation-based inference can help us learn about hydrologic processes. Water 13, 3633 (2021).
Liu, X., Song, Y. & Shen, C. Bathymetry inversion using a deep-learning-based surrogate for shallow water equations solvers. Preprint at https://doi.org/10.48550/arXiv.2203.02821 (2022).
Raissi, M., Perdikaris, P. & Karniadakis, G. E. Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686–707 (2019).
He, Q., Barajas-Solano, D., Tartakovsky, G. & Tartakovsky, A. M. Physics-informed neural networks for multiphysics data assimilation with application to subsurface transport. Adv. Water Resour. 141, 103610 (2020).
Wang, N., Zhang, D., Chang, H. & Li, H. Deep learning of subsurface flow via theory-guided neural network. J. Hydrol. 584, 124700 (2020).
Brown, T. B. et al. Language models are few-shot learners. In Proc. of the 34th International Conference on Neural Information Processing Systems, 1877–1901 (2020).
Kraft, B., Jung, M., Körner, M. & Reichstein, M. in The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences vol. XLIII-B2-2020, 1537–1544 (Copernicus, 2020).
Kraft, B., Jung, M., Körner, M., Koirala, S. & Reichstein, M. Towards hybrid modeling of the global hydrological cycle. Hydrol. Earth Syst. Sci. 26, 1579–1614 (2022).
Liu, J., Rahmani, F., Lawson, K. & Shen, C. A multiscale deep learning model for soil moisture integrating satellite and in situ data. Geophys. Res. Lett. 49, e2021GL096847 (2022).
Karniadakis, G. E. et al. Physics-informed machine learning. Nat. Rev. Phys. 3, 422–440 (2021).
Karpatne, A. et al. Theory-guided data science: a new paradigm for scientific discovery from data. IEEE Trans. Knowl. Data Eng. 29, 2318–2331 (2017).
Khandelwal, A. et al. Physics guided machine learning methods for hydrology. https://arxiv.org/abs/2012.02854 (2020).
Pawar, S., San, O., Aksoylu, B., Rasheed, A. & Kvamsdal, T. Physics guided machine learning using simplified theories. Phys. Fluids 33, 011701 (2021).
Reichstein, M. et al. Deep learning and process understanding for data-driven Earth system science. Nature 566, 195–204 (2019).
Bennett, A. & Nijssen, B. Deep learned process parameterizations provide better representations of turbulent heat fluxes in hydrologic models. Water Resour. Res. 57, e2020WR029328 (2021).
Schaap, M. G., Leij, F. J. & van Genuchten, M. Th. Rosetta: a computer program for estimating soil hydraulic parameters with hierarchical pedotransfer functions. J. Hydrol. 251, 163–176 (2001).
Rasp, S., Pritchard, M. S. & Gentine, P. Deep learning to represent subgrid processes in climate models. Proc. Natl Acad. Sci. USA 115, 9684–9689 (2018).
Zhu, Y. et al. Physics-informed deep-learning parameterization of ocean vertical mixing improves climate simulations. Natl Sci. Rev. 9, nwac044 (2022).
Koppa, A., Rains, D., Hulsman, P., Poyatos, R. & Miralles, D. G. A deep learning-based hybrid model of global terrestrial evaporation. Nat. Commun. 13, 1912 (2022).
Liu, B. et al. Physics-guided long short-term memory network for streamflow and flood simulations in the Lancang–Mekong river basin. Water 14, 1429 (2022).
Li, D., Marshall, L., Liang, Z., Sharma, A. & Zhou, Y. Bayesian LSTM with stochastic variational inference for estimating model uncertainty in process-based hydrological models. Water Resour. Res. 57, e2021WR029772 (2021).
Frame, J. M. et al. Post-processing the national water model with long short-term memory networks for streamflow predictions and model diagnostics. J. Am. Water Resour. Assoc. 57, 885–905 (2021).
Sun, A. Y., Jiang, P., Yang, Z.-L., Xie, Y. & Chen, X. A graph neural network (GNN) approach to basin-scale river network learning: the role of physics-based connectivity and data fusion. Hydrol. Earth Syst. Sci. Discuss. 26, 5163–5184 (2022).
Hochreiter, S. The vanishing gradient problem during learning recurrent neural nets and problem solutions. Int. J. Uncertain. Fuzziness Knowl. Syst. 06, 107–116 (1998).
Hochreiter, S., Bengio, Y., Frasconi, P. & Jürgen S. in A Field Guide to Dynamical Recurrent Neural Networks (eds Kremer, S. C. & Kolen, J. F.) 237–244 (IEEE, 2001).
Kochkov, D. et al. Machine learning–accelerated computational fluid dynamics. Proc. Natl Acad. Sci. USA 118, e2101784118 (2021).
Fang, K., Kifer, D., Lawson, K. & Shen, C. Evaluating the potential and challenges of an uncertainty quantification method for long short-term memory models for soil moisture predictions. Water Resour. Res. 56, e2020WR028095 (2020).
Tabas, S. S. & Samadi, S. Variational Bayesian dropout with a Gaussian prior for recurrent neural networks application in rainfall–runoff modeling. Environ. Res. Lett. 17, 065012 (2022).
Krapu, C. & Borsuk, M. A differentiable hydrology approach for modeling with time-varying parameters. Water Resour. Res. 58, e2021WR031377 (2022).
The authors are grateful for the discussion at the HydroML symposium, University Park, PA, May 2022, https://bit.ly/3g3DQNX, sponsored by National Science Foundation EAR #2015680 and Penn State Institute for Computational and Data Sciences. C.S., Y.S. and T.B. were supported by National Science Foundation Award EAR-2221880, Office of Science, US Department of Energy under award DE-SC0016605, and Cooperative Institute for Research to Operations in Hydrology (CIROH), award number A22-0307-S003, respectively. P.G. acknowledges funding from the National Science Foundational Science and Technology Center, Learning the Earth with Artificial intelligence and Physics (LEAP), award #2019625, and USMILE European Research Council grant. M. Wernimont at the US Geological Survey (USGS) greatly improved the presentation of Figs. 1 and 2. A.P.A. was supported by the USGS Water Mission Area, Water Availability and Use Science Program. Any use of trade, firm or product names is for descriptive purposes only and does not imply endorsement by the US Government.
K.L. and C.S. have financial interests in HydroSapient, Inc.
Peer review information
Nature Reviews Earth & Environment thanks A. McGovern, G. Varando and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Shen, C., Appling, A.P., Gentine, P. et al. Differentiable modelling to unify machine learning and physical models for geosciences. Nat Rev Earth Environ 4, 552–567 (2023). https://doi.org/10.1038/s43017-023-00450-9