Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

A framework for data-driven solution and parameter estimation of PDEs using conditional generative adversarial networks

Nature Computational Science volume 1, pages 819–829 (2021)Cite this article

Subjects

A preprint version of the article is available at arXiv.

Abstract

Here we employ and adapt the image-to-image translation concept based on conditional generative adversarial networks (cGAN) for learning a forward and an inverse solution operator of partial differential equations (PDEs). We focus on steady-state solutions of coupled hydromechanical processes in heterogeneous porous media and present the parameterization of the spatially heterogeneous coefficients, which is exceedingly difficult using standard reduced-order modeling techniques. We show that our framework provides a speed-up of at least 2,000 times compared to a finite-element solver and achieves a relative root-mean-square error (r.m.s.e.) of less than 2% for forward modeling. For inverse modeling, the framework estimates the heterogeneous coefficients, given an input of pressure and/or displacement fields, with a relative r.m.s.e. of less than 7%, even for cases where the input data are incomplete and contaminated by noise. The framework also provides a speed-up of 120,000 times compared to a Gaussian prior-based inverse modeling approach while also delivering more accurate results.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Illustration of strongly heterogeneous input and output fields.
Fig. 2: The results of three test cases using the W model.
Fig. 3: Relative r.m.s.e. of the test set using the W model.
Fig. 4: Results of equation (4) for the W model using different numbers of training samples.
Fig. 5: The results of three test cases of the W model and pyPCGA.
Fig. 6: The results of three test cases of the W model and pyPCGA.

Similar content being viewed by others

Data availability

We provide the training, validation and testing data or the scripts used to generate them at 10.24433/CO.6650973.v161. We also discuss these data characteristics in Supplementary Section 4.1. Source data are provided with this paper.

Code availability

The scripts used to produce these results as well as training data are available at 10.24433/CO.6650973.v161. The finite-element source codes used to generate the training, validation and testing data are available at https://github.com/teeratornk/jcp_YJCPH_110030_git, and a tutorial for the multiphenics package is available at https://github.com/multiphenics/multiphenics/tree/master/tutorials/09_multiphysics_examples. Scripts of the pyPCGA used in the paper are available at https://github.com/jonghyunharrylee/pyPCGA/tree/master/examples/fenics_hydromechanics.

References

  1. Schilders, W., Van der Vorst, H. & Rommes, J. Model Order Reduction: Theory, Research Aspects and Applications Vol. 13 (Springer, 2008).

  2. Choi, Y., Boncoraglio, G., Anderson, S., Amsallem, D. & Farhat, F. Gradient-based constrained optimization using a database of linear reduced-order models. J. Comput. Phys. 423, 109787 (2020).

    Article  MathSciNet  Google Scholar 

  3. McBane, S. & Choi, Y. Component-wise reduced order model lattice-type structure design. Comput. Methods Appl. Mech. Eng 381, 113813 (2021).

    Article  MathSciNet  Google Scholar 

  4. Choi, Y., Oxberry, G., White, D. & Kirchdoerfer, T. Accelerating design optimization using reduced order models. Preprint at https://arxiv.org/abs/1909.11320 (2019).

  5. Ballarin, F., D’amario, A., Perotto, S. & Rozza, G. A POD-selective inverse distance weighting method for fast parametrized shape morphing. Int. J. Numer. Methods Eng. 117, 860–884 (2019).

    Article  MathSciNet  Google Scholar 

  6. Hesthaven, J. et al. Certified Reduced Basis Methods for Parametrized Partial Differential Equations (Springer, 2016).

  7. Kadeethum, T., Ballarin, F. & Bouklas, N. Data-driven reduced order modeling of poroelasticity of heterogeneous media based on a discontinuous Galerkin approximation. GEM Int. J. Geomath. 12, 1–45 (2021).

    Article  MathSciNet  Google Scholar 

  8. Matthai, S. & Nick, H. Upscaling two-phase flow in naturally fractured reservoirs. AAPG Bull. 93, 1621–1632 (2009).

    Article  Google Scholar 

  9. Flemisch, B. et al. Benchmarks for single-phase flow in fractured porous media. Adv. Water Resour. 111, 239–258 (2018).

    Article  Google Scholar 

  10. Kadeethum, T., Salimzadeh, S. & Nick, H. An investigation of hydromechanical effect on well productivity in fractured porous media using full factorial experimental design. J. Pet. Sci. Eng. 181, 106233 (2019).

    Article  Google Scholar 

  11. Kadeethum, T., Salimzadeh, S. & Nick, H. Well productivity evaluation in deformable single-fracture media. Geothermics 87, 101839 (2020).

    Article  Google Scholar 

  12. Hesthaven, J. & Ubbiali, S. Non-intrusive reduced order modeling of nonlinear problems using neural networks. J. Comput. Phys. 363, 55–78 (2018).

    Article  MathSciNet  Google Scholar 

  13. Choi, Y., Brown, P., Arrighi, W., Anderson, R. & Huynh, K. Space-time reduced order model for large-scale linear dynamical systems with application to Boltzmann transport problems. J. Comput. Phys. 424, 109845 (2021).

    Article  MathSciNet  Google Scholar 

  14. Raissi, M., Perdikaris, P. & Karniadakis, G. Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686–707 (2019).

    Article  MathSciNet  Google Scholar 

  15. Kadeethum, T., Jørgensen, T. M. & Nick, H. M. Physics-informed neural networks for solving nonlinear diffusivity and Biot’s equations. PLoS ONE 15, e0232683 (2020).

    Article  Google Scholar 

  16. Fuhg, J. N. & Bouklas, N. The mixed deep energy method for resolving concentration features in finite strain hyperelasticity. Preprint at https://arxiv.org/abs/2104.09623 (2021).

  17. Jagtap, A. & Karniadakis, G. Extended physics-informed neural networks (XPINNs): a generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations. Commun. Comput. Phys. 28, 2002–2041 (2020).

    Article  MathSciNet  Google Scholar 

  18. Lu, L., Jin, P., Pang, G., Zhang, Z. & Karniadakis, G. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nat. Mach. Intell. 3, 218–229 (2021).

    Article  Google Scholar 

  19. Li, Z. et al. Neural operator: graph kernel network for partial differential equations. Preprint at https://arxiv.org/abs/2003.03485 (2020).

  20. Goodfellow, I. et al. Generative adversarial networks. Commun. ACM 63, 139–144 (2020).

    Article  Google Scholar 

  21. Liu, M. & Tuzel, O. Coupled generative adversarial networks. Preprint at https://arxiv.org/abs/1606.07536 (2016).

  22. Odena, A., Dumoulin, V. & Olah, C. Deconvolution and checkerboard artifacts. Distill 1, e3 (2016).

    Article  Google Scholar 

  23. Radford, A., Metz, L. & Chintala, S. Unsupervised representation learning with deep convolutional generative adversarial networks. Preprint at https://arxiv.org/abs/1511.06434 (2015).

  24. Yoon, J., Jarrett, D. & van der Schaar, M. Time-series generative adversarial networks. In Advances in Neural Information Processing Systems Vol. 32 (NeurIPS 2019) (NIPS, 2019).

  25. Shen, Y., Gu, J., Tang, X. & Zhou, B. Interpreting the latent space of GANs for semantic face editing. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 9243–9252 (IEEE, 2020).

  26. Mirza, M. & Osindero, S. Conditional generative adversarial nets. Preprint at https://arxiv.org/abs/1411.1784 (2014).

  27. Chen, X. et al. InfoGAN: interpretable representation learning by information maximizing generative adversarial nets. In Proc 30th International Conference on Neural Information Processing Systems 2180–2188 (2016).

  28. Lee, X. et al. Fast inverse design of microstructures via generative invariance networks. Nat. Comput. Sci. 1, 229–238 (2021).

    Article  Google Scholar 

  29. Isola, P., Zhu, J., Zhou, T. & Efros, A. Image-to-image translation with conditional adversarial networks. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 1125–1134 (IEEE, 2017).

  30. Wang, T. et al. High-resolution image synthesis and semantic manipulation with conditional GANs. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 8798–8807 (IEEE, 2018).

  31. Ledig, C. et al. Photo-realistic single image super-resolution using a generative adversarial network. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 4681–4690 (IEEE, 2017).

  32. Xu, T. et al. AttnGAN: fine-grained text to image generation with attentional generative adversarial networks. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 1316–1324 (IEEE, 2018).

  33. Zhu, J., Park, T., Isola, P. & Efros, A. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proc. IEEE International Conference on Computer Vision 2223–2232 (IEEE, 2017).

  34. Liu, M., Breuel, T. & Kautz, J. Unsupervised image-to-image translation networks. Preprint at https://arxiv.org/abs/1703.00848 (2017).

  35. Huang, X., Liu, M., Belongie, S. & Kautz, J. Multimodal unsupervised image-to-image translation. In Proc. European Conference on Computer Vision (ECCV) 172–189 (IEEE, 2018).

  36. Park, T., Liu, M., Wang, T. & Zhu, J. Semantic image synthesis with spatially-adaptive normalization. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 2337–2346 (IEEE, 2019).

  37. Ronneberger, O., Fischer, P. & Brox, T. U-Net: convolutional networks for biomedical image segmentation. In Proc. International Conference on Medical Image Computing and Computer-Assisted Intervention 234–241 (Springer, 2015).

  38. Demir, U. & Unal, G. Patch-based image inpainting with generative adversarial networks. Preprint at https://arxiv.org/abs/1803.07422 (2018).

  39. Miyato, T., Kataoka, T., Koyama, M. & Yoshida, Y. Spectral normalization for generative adversarial networks. Preprint at https://arxiv.org/abs/1802.05957 (2018).

  40. Arjovsky, M., Chintala, S. & Bottou, L. Wasserstein generative adversarial networks. In Proc. International Conference on Machine Learning 214–223 (PMLR, 2017).

  41. Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V. & Courville, A. Improved training of Wasserstein GANs. Preprint at https://arxiv.org/abs/1704.00028 (2017).

  42. Kadeethum, T., Nick, H., Lee, S. & Ballarin, F. Enriched Galerkin discretization for modeling poroelasticity and permeability alteration in heterogeneous porous media. J. Comput. Phys. 427, 110030 (2021).

    Article  MathSciNet  Google Scholar 

  43. Kadeethum, T., Lee, S. & Nick, H. Finite element solvers for Biot’s poroelasticity equations in porous media. Math. Geosci. 52, 977–1015 (2020).

    Article  MathSciNet  Google Scholar 

  44. Zinn, B. & Harvey, C. When good statistical models of aquifer heterogeneity go bad: a comparison of flow, dispersion, and mass transfer in connected and multivariate Gaussian hydraulic conductivity fields. Water Resour. Res. 39, 1051 (2003).

    Article  Google Scholar 

  45. Paul-Dubois-Taine, A. & Amsallem, D. An adaptive and efficient greedy procedure for the optimal training of parametric reduced-order models. Int. J. Numer. Methods Eng. 102, 1262–1292 (2015).

    Article  MathSciNet  Google Scholar 

  46. Guénot, M., Lepot, I., Sainvitu, C., Goblet, J. & Filomeno Coelho, R. Adaptive sampling strategies for non-intrusive POD-based surrogates. Eng. Comput. 30, 521–547 (2013).

    Article  Google Scholar 

  47. O’Malley, D., Karra, S., Hyman, J., Viswanathan, H. & Srinivasan, G. Efficient Monte Carlo with graph-based subsurface flow and transport models. Water Resour. Res. 54, 3758–3766 (2018).

    Article  Google Scholar 

  48. Peherstorfer, B., Willcox, K. & Gunzburger, M. Optimal model management for multifidelity Monte Carlo estimation. SIAM J. Sci. Comput. 38, A3163–A3194 (2016).

    Article  MathSciNet  Google Scholar 

  49. Lee, J. & Kitanidis, P. Large-scale hydraulic tomography and joint inversion of head and tracer data using the principal component geostatistical approach (PCGA). Water Resour. Res. 50, 5410–5427 (2014).

    Article  Google Scholar 

  50. Lee, J., Yoon, H., Kitanidis, P., Werth, C. & Valocchi, A. Scalable subsurface inverse modeling of huge data sets with an application to tracer concentration breakthrough data from magnetic resonance imaging. Water Resour. Res. 52, 5213–5231 (2016).

    Article  Google Scholar 

  51. Ghorbanidehno, H., Kokkinaki, A., Lee, J. & Darve, E. Recent developments in fast and scalable inverse modeling and data assimilation methods in hydrology. J. Hydrol. 591, 125266 (2020).

    Article  Google Scholar 

  52. Wang, R., Chen, C., Lee, J. & Darve, E. PBBFMM3D: a parallel black-box algorithm for kernel matrix-vector multiplication. J. Parallel Distrib. Comput. 154, 64–73 (2021).

    Article  Google Scholar 

  53. Kang, P. K. et al. Improved characterization of heterogeneous permeability in saline aquifers from transient pressure data during freshwater injection. Water Resour. Res. 53, 4444–4458 (2017).

    Article  Google Scholar 

  54. Lee, J., Kokkinaki, A. & Kitanidis, P. K. Fast large-scale joint inversion for deep aquifer characterization using pressure and heat tracer measurements. Transport Porous Med. 123, 533–543 (2018).

    Article  MathSciNet  Google Scholar 

  55. Kadeethum, T., Jørgensen, T. & Nick, H. Physics-informed neural networks for solving nonlinear diffusivity and Biot’s equations. PLoS ONE 15, e0232683 (2020).

    Article  Google Scholar 

  56. Kadeethum, T., Jørgensen, T. & Nick, H. Physics-informed neural networks for solving inverse problems of nonlinear Biot’s equations: batch training. In Proc. 54th US Rock Mechanics/Geomechanics Symposium (American Rock Mechanics Association, 2020).

  57. Qian, Y. et al. Application of deep learning-based interpolation methods to nearshore bathymetry. Preprint at https://arxiv.org/abs/2011.09707 (2020).

  58. Patel, D. & Oberai, A. GAN-based priors for quantifying uncertainty. Preprint at https://arxiv.org/abs/2003.12597 (2020).

  59. Ronneberger, O., Fischer, P. & Brox, T. U-Net: convolutional networks for biomedical image segmentation. In Proc. International Conference on Medical Image Computing and Computer-Assisted Intervention 234–241 (Springer, 2015).

  60. Demir, U. & Unal, G. Patch-based image inpainting with generative adversarial networks. Preprint at https://arxiv.org/abs/1803.07422 (2018).

  61. Kadeethum, T. et al. scripts: a framework for data-driven solution and parameter estimation of PDEs using conditional generative adversarial networks [source code]. (Code Ocean, 2021); https://doi.org/10.24433/CO.6650973.v1

Download references

Acknowledgements

D.O. acknowledges support from Los Alamos National Laboratory’s Laboratory Directed Research and Development Early Career Award (20200575ECR). H.V. is grateful for funding support from the US Department of Energy (DOE) Basic Energy Sciences (LANLE3W1). N.B. acknowledges start-up funds from Cornell University. Y.C. acknowledges LDRD funds (21-FS-042) from Lawrence Livermore National Laboratory. Lawrence Livermore National Laboratory is operated by Lawrence Livermore National Security, LLC, for the US DOE, National Nuclear Security Administration, under contract no. DE-AC52-07NA27344 (LLNL-JRNL-823007).

Author information

Authors and Affiliations

  1. Sibley School of Mechanical and Aerospace, EngineeringCornell University, Ithaca, NY, USA

    Teeratorn Kadeethum, Jan Niklas Fuhg & Nikolaos Bouklas

  2. Computational Earth Science, Los Alamos National Laboratory, Los Alamos, NM, USA

    Daniel O’Malley & Hari S. Viswanathan

  3. Center for Applied Scientific Computing, Lawrence Livermore National Laboratory, Livermore, CA, USA

    Youngsoo Choi

  4. Civil & Environmental Engineering/Water Resources Research Center, University of Hawai’i at Manoa, Honolulu, HI, USA

    Jonghyun Lee

  5. Center for Applied Mathematics, Cornell University, Ithaca, NY, USA

    Nikolaos Bouklas

Authors
  1. Teeratorn Kadeethum
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel O’Malley
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jan Niklas Fuhg
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Youngsoo Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jonghyun Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hari S. Viswanathan
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Nikolaos Bouklas
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

T.K. contributed conceptualization, formal analysis, software and validation, wrote the original draft, and reviewed and edited the manuscript. D.O. contributed conceptualization, formal analysis, supervision and validation, and reviewed and edited the manuscript. J.N.F. contributed software and validation, and reviewed and edited the manuscript. Y.C. contributed conceptualization, formal analysis, supervision and validation, and reviewed and edited the manuscript. J.L. contributed software, formal analysis and supervision, and reviewed and edited the manuscript. H.S.V. contributed conceptualization and supervision, and reviewed and edited the manuscript. N.B. contributed conceptualization, formal analysis, funding acquisition and supervision, and reviewed and edited the manuscript.

Corresponding author

Correspondence to Nikolaos Bouklas.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Editor recognition statement Handling editor: Fernando Chirigati, in collaboration with the Nature Computational Science team. Nature Computational Science thanks the anonymous reviewers for their contribution to the peer review of this work.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary sections 1–5, Figs. 1–28 and Tables 1–4.

Source data

Source Data Fig. 2

Statistical source data.

Source Data Fig. 3

Statistical source data.

Source Data Fig. 4

Statistical source data.

Source Data Fig. 5

Statistical source data.

Source Data Fig. 6

Statistical source data.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kadeethum, T., O’Malley, D., Fuhg, J.N. et al. A framework for data-driven solution and parameter estimation of PDEs using conditional generative adversarial networks. Nat Comput Sci 1, 819–829 (2021). https://doi.org/10.1038/s43588-021-00171-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s43588-021-00171-3

This article is cited by

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics