Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Super-resolution generative adversarial networks of randomly-seeded fields

Nature Machine Intelligence volume 4, pages 1165–1173 (2022)Cite this article

Subjects

A preprint version of the article is available at arXiv.

Abstract

Reconstruction of field quantities from sparse measurements is a problem arising in a broad spectrum of applications. This task is particularly challenging when the mapping between sparse measurements and field quantities is performed in an unsupervised manner. Further complexity is added for moving sensors and/or random on–off status. Under such conditions, the most straightforward solution is to interpolate the scattered data onto a regular grid. However, the spatial resolution achieved with this approach is ultimately limited by the mean spacing between the sparse measurements. In this work, we propose a super-resolution generative adversarial network framework to estimate field quantities from random sparse sensors. The algorithm exploits random sampling to provide incomplete views of the high-resolution underlying distributions. It is hereby referred to as the randomly seeded super-resolution generative adversarial network (RaSeedGAN). The proposed technique is tested on synthetic databases of fluid flow simulations, ocean surface temperature distribution measurements and particle-image velocimetry data of a zero-pressure-gradient turbulent boundary layer. The results show excellent performance even in cases with high sparsity or noise level. This generative adversarial network algorithm provides full-field high-resolution estimation from randomly seeded fields with no need of full-field high-resolution representations for training.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Schematic illustration of RaSeedGAN architecture.
Fig. 2: Panel of synthetic test cases, with examples of instantaneous field reconstructions.
Fig. 3: Measurement of ocean surface temperature from NOAA sea surface data.
Fig. 4: Experimental turbulent boundary layer at Reτ ≈ 1,000.
Fig. 5: Turbulent statistics for test case 2 (DNS turbulent channel flow) and test case 4 (experimental turbulent boundary layer).

Similar content being viewed by others

Data availability

All datasets used in this work are openly available in Zenodo, accessible at https://doi.org/10.5281/zenodo.7191210

Code availability

All codes developed in this work are openly available in GitHub, accessible through the link https://github.com/eaplab/RaSeedGAN

References

  1. Bolton, T. & Zanna, L. Applications of deep learning to ocean data inference and subgrid parameterization. J. Adv. Model. Earth Syst. 11, 376–399 (2019).

    Article  Google Scholar 

  2. Akiyama, K. et al. First M87 event horizon telescope results. III. Data processing and calibration. Astrophys. J. Lett. 875, L3 (2019).

    Article  Google Scholar 

  3. Yakhot, A., Anor, T. & Karniadakis, G. E. A reconstruction method for gappy and noisy arterial flow data. IEEE Trans. Med. Imaging 26, 1681–1697 (2007).

    Article  Google Scholar 

  4. Manohar, K., Brunton, B. W., Kutz, J. N. & Brunton, S. L. Data-driven sparse sensor placement for reconstruction: demonstrating the benefits of exploiting known patterns. IEEE Control Syst. Mag. 38, 63 (2018).

    Article  MathSciNet  MATH  Google Scholar 

  5. Cortina-Fernández, J., Sanmiguel Vila, C., Ianiro, A. & Discetti, S. From sparse data to high-resolution fields: ensemble particle modes as a basis for high-resolution flow characterization. Exp. Therm. Fluid Sci. 120, 110178 (2020).

    Article  Google Scholar 

  6. Fukami, K., Maulik, R., Ramachandra, N., Fukagata, K. & Taira, K. Global field reconstruction from sparse sensors with Voronoi tessellation-assisted deep learning. Nat. Mach. Intell. 3, 945 (2021).

    Article  Google Scholar 

  7. Gundersen, K., Oleynik, A., Blaser, N. & Alendal, G. Semi-conditional variational auto-encoder for flow reconstruction and uncertainty quantification from limited observations. Phys. Fluids 33, 017119 (2021).

    Article  Google Scholar 

  8. Shen, H. et al. Missing information reconstruction of remote sensing data: a technical review. IEEE Geosci. Remote Sens. Mag. 3, 61 (2015).

    Article  Google Scholar 

  9. Callaham, J. L., Maeda, K. & Brunton, S. L. Robust flow reconstruction from limited measurements via sparse representation. Phys. Rev. Fluids 4, 103907 (2019).

    Article  Google Scholar 

  10. Berkooz, G., Holmes, P. & Lumley, J. L. The proper orthogonal decomposition in the analysis of turbulent flows. Annu. Rev. Fluid Mech. 25, 539 (1993).

    Article  MathSciNet  Google Scholar 

  11. Schmid, P. J. Dynamic mode decomposition of numerical and experimental data. J. Fluid Mech. 656, 5 (2010).

    Article  MathSciNet  MATH  Google Scholar 

  12. Everson, R. & Sirovich, L. Karhunen-Loève procedure for gappy data. J. Opt. Soc. Am. A 12, 1657 (1995).

    Article  Google Scholar 

  13. Venturi, D. & Karniadakis, G. E. Gappy data and reconstruction procedures for flow past a cylinder. J. Fluid Mech. 519, 315 (2004).

    Article  MathSciNet  MATH  Google Scholar 

  14. Raben, S. G., Charonko, J. J. & Vlachos, P. P. Adaptive gappy proper orthogonal decomposition for particle image velocimetry data reconstruction. Meas. Sci. Technol. 23, 025303 (2012).

    Article  Google Scholar 

  15. Huang, X. Compressive sensing and reconstruction in measurements with an aerospace application. AIAA J. 51, 1011 (2013).

    Article  Google Scholar 

  16. Maulik, R., Fukami, K., Ramachandra, N., Fukagata, K. & Taira, K. Probabilistic neural networks for fluid flow surrogate modeling and data recovery. Phys. Rev. Fluids 5, 104401 (2020).

    Article  Google Scholar 

  17. Gao, H., Sun, L. & Wang, J.-X. Super-resolution and denoising of fluid flow using physics-informed convolutional neural networks without high-resolution labels. Phys. Fluids 33, 073603 (2021).

    Article  Google Scholar 

  18. Erichson, N. B. et al. Shallow neural networks for fluid flow reconstruction with limited sensors. Proc. R. Soc. A: Math. Phys. Eng. Sci. 476, 20200097 (2020).

    Article  MathSciNet  MATH  Google Scholar 

  19. Sun, L. & Wang, J.-X. Physics-constrained bayesian neural network for fluid flow reconstruction with sparse and noisy data. Theor. Appl. Mech. Lett. 10, 161 (2020).

    Article  Google Scholar 

  20. Arzani, A., Wang, J.-X. & D’Souza, R. M. Uncovering near-wall blood flow from sparse data with physics-informed neural networks. Phys. Fluids 33, 071905 (2021).

    Article  Google Scholar 

  21. Brunton, S. L., Noack, B. R. & Koumoutsakos, P. Machine learning for fluid mechanics. Annu. Rev. Fluid Mech. 52, 477 (2020).

  22. Fukami, K., Fukagata, K. & Taira, K. Super-resolution reconstruction of turbulent flows with machine learning. J. Fluid Mech. 870, 106 (2019).

    Article  MathSciNet  MATH  Google Scholar 

  23. Kim, H., Kim, J., Won, S. & Lee, C. Unsupervised deep learning for super-resolution reconstruction of turbulence. J. Fluid Mech. 910, A29 (2021).

    Article  MathSciNet  MATH  Google Scholar 

  24. Güemes, A. et al. From coarse wall measurements to turbulent velocity fields through deep learning. Phys. Fluids 33, 075121 (2021).

    Article  Google Scholar 

  25. Deng, Z., He, C., Liu, Y. & Kim, K. C. Super-resolution reconstruction of turbulent velocity fields using a generative adversarial network-based artificial intelligence framework. Phys. Fluids 31, 125111 (2019).

    Article  Google Scholar 

  26. Stengel, K., Glaws, A., Hettinger, D. & King, R. N. Adversarial super-resolution of climatological wind and solar data. Proc. Natl Acad. Sci. USA 117, 16805 (2020).

    Article  Google Scholar 

  27. Ledig, C. et al. Photo-realistic single image super-resolution using a generative adversarial network. In Proc. IEEE CVPR 4681–4690 (IEEE, 2017).

  28. Deng, N., Noack, B. R., Morzyński, M. & Pastur, L. R. Low-order model for successive bifurcations of the fluidic pinball. J. Fluid Mech. 884, A37 (2020).

    Article  MathSciNet  MATH  Google Scholar 

  29. Scarano, F. Iterative image deformation methods in PIV. Meas. Sci. Technol. 13, R1 (2001).

    Article  Google Scholar 

  30. Radford, A., Metz, L. & Chintala, S. Unsupervised representation learning with deep convolutional generative adversarial networks. Preprint at https://arxiv.org/abs/1511.06434 (2015).

  31. Sønderby, C. K., Caballero, J., Theis, L., Shi, W. & Huszár, F. Amortised map inference for image super-resolution. Preprint at https://arxiv.org/abs/1610.04490 (2016).

  32. Lozano-Durán, A., Flores, O. & Jiménez, J. The three-dimensional structure of momentum transfer in turbulent channels. J. Fluid Mech. 694, 100 (2012).

    Article  MATH  Google Scholar 

  33. Schlatter, P. & Örlü, R. Assessment of direct numerical simulation data of turbulent boundary layers. J. Fluid Mech. 659, 116 (2010).

    Article  MATH  Google Scholar 

  34. Atkinson, C., Buchmann, N. A., Amili, O. & Soria, J. On the appropriate filtering of PIV measurements of turbulent shear flows. Exp. Fluids 55, 1 (2014).

    Article  Google Scholar 

  35. Wang, X. et al. ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks. In Proc. ECCV Workshop (2018) (eds Leal-Taixé, L. & Roth, S.) 63–79 (Springer, 2018).

  36. Gross, S. & Wilber, M. Training and investigating residual nets. Facebook AI Res. 6, 3 (2016).

    Google Scholar 

  37. Shi, W. et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proc. IEEE CVPR 1874–1883 (IEEE, 2016).

  38. Abadi, M. et al. TensorFlow: a system for large-scale machine learning. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16) 265–283 (2016).

  39. Li, Y. et al. A public turbulence database cluster and applications to study Lagrangian evolution of velocity increments in turbulence. J. Turbul. 9, N31 (2008).

    Article  Google Scholar 

Download references

Acknowledgements

This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 949085) received by S.D. NOAA High Resolution SST data provided by the NOAA/OAR/ESRL PSL.

Author information

Authors and Affiliations

  1. Aerospace Engineering Research Group, Universidad Carlos III de Madrid, Leganés, Spain

    Alejandro Güemes, Carlos Sanmiguel Vila & Stefano Discetti

  2. Sub-Directorate General of Terrestrial Systems, Spanish National Institute for Aerospace Technology (INTA), San Martín de la Vega, Spain

    Carlos Sanmiguel Vila

Authors
  1. Alejandro Güemes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carlos Sanmiguel Vila
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stefano Discetti
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

A.G.: methodology, software, validation, investigation, data curation, writing—original draft, writing—review and editing, visualization. C.S.V.: conceptualization, methodology, writing—original draft, writing—review and editing. S.D.: methodology, software, resources, data curation, supervision, writing—original draft, writing—review and editing, funding acquisition.

Corresponding authors

Correspondence to Alejandro Güemes, Carlos Sanmiguel Vila or Stefano Discetti.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Machine Intelligence thanks the anonymous reviewers for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Pixel-based root mean square error evaluated in the test data set as a function of the number of training samples for Test Case 1 (left) and Test Case 2 (right) with fu = 4.

The error is scaled with the standard deviation of the corresponding fluctuating quantities. Green and yellow lines indicate respectively the error on the streamwise and crosswise velocity components.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Güemes, A., Sanmiguel Vila, C. & Discetti, S. Super-resolution generative adversarial networks of randomly-seeded fields. Nat Mach Intell 4, 1165–1173 (2022). https://doi.org/10.1038/s42256-022-00572-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s42256-022-00572-7

This article is cited by

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics