Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Deep variational network for rapid 4D flow MRI reconstruction

Nature Machine Intelligence volume 2, pages 228–235 (2020)Cite this article

Subjects

Abstract

Phase-contrast magnetic resonance imaging (MRI) provides time-resolved quantification of blood flow dynamics that can aid clinical diagnosis. Long in vivo scan times due to repeated three-dimensional (3D) volume sampling over cardiac phases and breathing cycles necessitate accelerated imaging techniques that leverage data correlations. Standard compressed sensing reconstruction methods require tuning of hyperparameters and are computationally expensive, which diminishes the potential reduction of examination times. We propose an efficient model-based deep neural reconstruction network and evaluate its performance on clinical aortic flow data. The network is shown to reconstruct undersampled 4D flow MRI data in under a minute on standard consumer hardware. Remarkably, the relatively low amounts of tunable parameters allowed the network to be trained on images from 11 reference scans while generalizing well to retrospective and prospective undersampled data for various acceleration factors and anatomies.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Rent or buy this article

Get just this article for as long as you need it

$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Breathing-resolved 4D flow data acquisition.
Fig. 2: FlowVN architecture and training.
Fig. 3: Reconstruction results on retrospectively undersampled data.
Fig. 4: Retrospective reconstruction of the data from a patient with abnormal flow pattern.
Fig. 5: Quantitative flow evaluation of reconstruction methods on prospectively undersampled data (12.4 ≤ R ≤ 13.8) from seven healthy volunteers.

Data availability

The code for the network training and inference used in this study as well as network weights are available online from CodeOcean together with volunteer data: https://codeocean.com/capsule/0115983/tree48. The code for analysis is available on CodeOcean from https://codeocean.com/capsule/2587940/tree49.

References

  1. Markl, M., Frydrychowicz, A., Kozerke, S., Hope, M. & Wieben, O. 4D flow MRI. J. Magn. Res. Imag. 36, 1015–1036 (2012).

    Article  Google Scholar 

  2. Feinberg, D., Hale, J., Watts, J., Kaufman, L. & Mark, A. Halving MR imaging time by conjugation: demonstration at 3.5 kG. Radiology 161, 527–531 (1986).

    Article  Google Scholar 

  3. Szarf, G. et al. Zero filled partial Fourier phase contrast MR imaging: in vitro and in vivo assessment. J. Magn. Reson. Imag. 23, 42–49 (2006).

    Article  Google Scholar 

  4. Walheim, J., Gotschy, A. & Kozerke, S. On the limitations of partial Fourier acquisition in phase-contrast MRI of turbulent kinetic energy. Magn. Reson. Med. 81, 514–523 (2019).

    Article  Google Scholar 

  5. Pruessmann, K., Weiger, M., Scheidegger, M. & Boesiger, P. SENSE: sensitivity encoding for fast MRI. Magn. Reson. Med. 42, 952–962 (1999).

    Article  Google Scholar 

  6. Wiesinger, F., Boesiger, P. & Pruessmann, K. P. Electrodynamics and ultimate SNR in parallel MR imaging. Magn. Reson. Med. 52, 376–390 (2004).

    Article  Google Scholar 

  7. Lustig, M., Donoho, D. & Pauly, J. M. Sparse MRI: the application of compressed sensing for rapid MR imaging. Magn. Reson. Med. 58, 1182–1195 (2007).

    Article  Google Scholar 

  8. Kim, D. et al. Accelerated phase-contrast cine MRI using k–t SPARSE-SENSE. Magn. Reson. Med. 67, 1054–1064 (2012).

    Article  Google Scholar 

  9. Valvano, G. et al. Accelerating 4D flow MRI by exploiting low-rank matrix structure and hadamard sparsity. Magn. Reson. Med. 78, 1330–1341 (2017).

    Article  Google Scholar 

  10. Bollache, E. et al. k–t accelerated aortic 4D flow MRI in under two minutes: feasibility and impact of resolution, k-space sampling patterns, and respiratory navigator gating on hemodynamic measurements. Magn. Reson. Med. 79, 195–207 (2018).

    Article  Google Scholar 

  11. Walheim, J., Dillinger, H. & Kozerke, S. Multipoint 5D flow cardiovascular magnetic resonance—accelerated cardiac- and respiratory-motion resolved mapping of mean and turbulent velocities. J. Cardiovasc. Magn. Res. 21, 42 (2019).

    Article  Google Scholar 

  12. Ma, L. E. et al. Aortic 4D flow MRI in 2 minutes using compressed sensing, respiratory controlled adaptive k-space reordering, and inline reconstruction. Magn Reson. Med. 81, 3675–3690 (2019).

    Article  Google Scholar 

  13. Rich, A. et al. A Bayesian approach for 4D flow imaging of aortic valve in a single breath-hold. Magn. Reson. Med. 81, 811–824 (2019).

    Article  Google Scholar 

  14. Zhang, T., Pauly, J. M. & Levesque, I. R. Accelerating parameter mapping with a locally low rank constraint. Magn. Reson. Med. 73, 655–661 (2015).

    Article  Google Scholar 

  15. Hammernik, K. et al. Learning a variational network for reconstruction of accelerated MRI data. Magn. Reson. Med. 79, 3055–3071 (2018).

    Article  Google Scholar 

  16. Mardani, M. et al. Deep generative adversarial neural networks for compressive sensing MRI. IEEE Trans. Med. Imag. 38, 167–179 (2018).

    Article  Google Scholar 

  17. Zhu, B., Liu, J. Z., Cauley, S. F., Rosen, B. R. & Rosen, M. S. Image reconstruction by domain-transform manifold learning. Nature 555, 487–492 (2018).

    Article  Google Scholar 

  18. Schlemper, J., Caballero, J., Hajnal, J. V., Price, A. N. & Rueckert, D. A deep cascade of convolutional neural networks for dynamic MR image reconstruction. IEEE Trans. Med. Imag. 37, 491–503 (2017).

    Article  Google Scholar 

  19. Maier, A. K. et al. Learning with known operators reduces maximum error bounds. Nat. Mach. Intell. 1, 373–380 (2019).

    Article  Google Scholar 

  20. Beck, A. & Teboulle, M. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imag. Sci. 2, 183–202 (2009).

    Article  MathSciNet  Google Scholar 

  21. Antun, V., Renna, F., Poon, C., Adcock, B. & Hansen, A. C. On instabilities of deep learning in image reconstruction-does AI come at a cost? Preprint at https://arxiv.org/pdf/1902.05300.pdf (2019).

  22. Yang, G. et al. DAGAN: deep de-aliasing generative adversarial networks for fast compressed sensing MRI reconstruction. IEEE Trans. Med. Imag. 37, 1310–1321 (2017).

    Article  Google Scholar 

  23. Quan, T., Nguyen-Duc, T. & Jeong, W. Compressed sensing MRI reconstruction using a generative adversarial network with a cyclic loss. IEEE Trans. Med. Imag. 37, 1488–1497 (2018).

    Article  Google Scholar 

  24. Narnhofer, D., Hammernik, K., Knoll, F. & Pock, T. Inverse GANs for accelerated MRI reconstruction. Proc. SPIE 11138, 111381A (2019).

    Google Scholar 

  25. Zhang, S., Block, K. & Frahm, J. Magnetic resonance imaging in real time: advances using radial FLASH. J. Mag. Res. Imag. 31, 101–109 (2010).

    Article  Google Scholar 

  26. Landweber, L. An iteration formula for Fredholm integral equations of the first kind. Am. J. Math. 73, 615–624 (1951).

    Article  MathSciNet  Google Scholar 

  27. Liu, Y. & Lew, M. S. Learning relaxed deep supervision for better edge detection. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition 231–240 (CVPR, 2016).

  28. Ravishankar, S. & Bresler, Y. MR image reconstruction from highly undersampled k-space data by dictionary learning. IEEE Trans. Med. Imag. 30, 1028–1041 (2010).

    Article  Google Scholar 

  29. Caballero, J., Price, A. N., Rueckert, D. & Hajnal, J. V. Dictionary learning and time sparsity for dynamic MR data reconstruction. IEEE Trans. Med. Imag 33, 979–994 (2014).

    Article  Google Scholar 

  30. Lee, D., Yoo, J. & Ye, J. C. Deep residual learning for compressed sensing MRI. In Proc. of IEEE Inter. Symp. on Bio. Imag. 15–18 (2017).

  31. LeCun, Y. A theoretical framework for back-propagation. In Proc. Connectionist Models Summer School Vol. 1, 21–28 (Morgan Kaufmann, 1988).

  32. Abadi, M. et al. Tensorflow: a system for large-scale machine learning. In Proceedings of the 12th USENIX conference on Operating Systems Design and Implementation 265–283 (2016).

  33. Domke, J. Generic methods for optimization-based modeling. In Proc. Int. Conf. on Artificial Intelligence and Statistics Vol. 22, 318–326 (ACM, 2012).

  34. Sun, J. et al. Deep ADMM-Net for compressive sensing MRI. In Proc. Int. Conf. on Neural Information Processing Systems 10–18 (Curran Associates, 2016).

  35. Jin, K. H., McCann, M. T., Froustey, E. & Unser, M. Deep convolutional neural network for inverse problems in imaging. IEEE Trans. Imag. Proc. 26, 4509–4522 (2017).

    Article  MathSciNet  Google Scholar 

  36. Vishnevskiy, V., Sanabria, S. J. & Goksel, O. Image reconstruction via variational network for real-time hand-held sound-speed imaging. In First International Workshop for Machine Learning for Medical Image Reconstruction 120–128 (Springer, 2018).

  37. Vishnevskiy, V., Rau, R. & Goksel, O. Deep variational networks with exponential weighting for learning computed tomography. In 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention Part VI, Vol. 11769, 310–318 (LNCS, Springer, 2019).

  38. Cuppen, J. & van Est, A. Reducing MR imaging time by one-sided reconstruction. Magn. Reson. Imag. 5, 526–527 (1987).

    Article  Google Scholar 

  39. Winkelmann, S., Schaeffter, T., Koehler, T., Eggers, H. & Doessel, O. An optimal radial profile order based on the golden ratio for time-resolved MRI. IEEE Trans. Med. Imag. 26, 68–76 (2006).

    Article  Google Scholar 

  40. Zhang, T., Pauly, J. M., Vasanawala, S. S. & Lustig, M. Coil compression for accelerated imaging with cartesian sampling. Magn. Reson. Med. 69, 571–582 (2013).

    Article  Google Scholar 

  41. Uecker, M. et al. ESPIRiT—an eigenvalue approach to autocalibrating parallel MRI: where SENSE meets GRAPPA. Magn. Reson. Med. 71, 990–1001 (2014).

    Article  Google Scholar 

  42. Bernstein, M. A. et al. Concomitant gradient terms in phase contrast MR: analysis and correction. Magn. Reson. Med. 39, 300–308 (1998).

    Article  Google Scholar 

  43. Busch, J., Giese, D. & Kozerke, S. Image-based background phase error correction in 4D flow MRI revisited. J. MRI 46, 1516–1525 (2017).

    Google Scholar 

  44. Walker, P. G. et al. Semiautomated method for noise reduction and background phase error correction in MR phase velocity data. J. Magn. Reson. Imag. 3, 521–530 (1993).

    Article  Google Scholar 

  45. Tamir, J. I., Ong, F., Cheng, J. Y., Uecker, M. & Lustig, M. Generalized magnetic resonance image reconstruction using the Berkeley advanced reconstruction toolbox. In ISMRM Workshop on Data Sampling and Image Reconstruction (ISMRM, 2016).

  46. Wang, Z., Bovik, A. C., Sheikh, H. R. & Simoncelli, E. P. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Imag. Proc. 13, 600–612 (2004).

    Article  Google Scholar 

  47. Altman, D. G. & Bland, J. M. Measurement in medicine: the analysis of method comparison studies. J. R. Stat. Soc. 32, 307–317 (1983).

    Google Scholar 

  48. Vishnevskiy, V., Walheim, J. & Kozerke, S. FlowVN: deep variational network for rapid 4D flow MRI reconstruction. CodeOcean https://doi.org/10.24433/CO.0115983.v1 (2020).

    Article  Google Scholar 

  49. Vishnevskiy, V., Walheim, J. & Kozerke, S. FlowVN: analysis. CodeOcean https://doi.org/10.24433/CO.5994453.v1 (2020).

    Article  Google Scholar 

Download references

Acknowledgements

The authors acknowledge funding from the European Unions Horizon 2020 research and innovation programme under grant agreement no. 668039 and under EuroStars UNIFORM as well as funding of the Platform for Advanced Scientific Computing of the Council of the Federal Institutes of Technology (ETH Board), Switzerland.

Author information

Author notes

  1. These authors contributed equally: Valery Vishnevskiy, Jonas Walheim.

Authors and Affiliations

  1. Institute for Biomedical Engineering, University and ETH Zurich, Zurich, Switzerland

    Valery Vishnevskiy, Jonas Walheim & Sebastian Kozerke

Authors
  1. Valery Vishnevskiy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jonas Walheim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sebastian Kozerke
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

J.W., V.V. and S.K. conceived the study. V.V. implemented the machine-learning reconstruction algorithms. J.W. conducted MR acquisition experiments and data preprocessing. J.W. and V.V. analysed experimental data under the supervision of S.K. All authors discussed the results and contributed to writing the manuscript.

Corresponding author

Correspondence to Valery Vishnevskiy.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Fig. 1, algorithm (1) and tables 1 and 2.

Reporting Summary

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Vishnevskiy, V., Walheim, J. & Kozerke, S. Deep variational network for rapid 4D flow MRI reconstruction. Nat Mach Intell 2, 228–235 (2020). https://doi.org/10.1038/s42256-020-0165-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s42256-020-0165-6

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing