Computed stereo lensless X-ray imaging

Abstract

Recovering the three-dimensional (3D) properties of artificial or biological systems using low X-ray doses is challenging as most techniques are based on computing hundreds of two-dimensional (2D) projections. The requirement for a low X-ray dose also prevents single-shot 3D imaging using ultrafast X-ray sources. Here we show that computed stereo vision concepts can be applied to X-rays. Stereo vision is important in the field of machine vision and robotics. We reconstruct two X-ray stereo views from coherent diffraction patterns and compute a nanoscale 3D representation of the sample from disparity maps. Similarly to brain perception, computed stereo vision algorithms use constraints. We demonstrate that phase-contrast images relax the disparity constraints, allowing occulted features to be revealed. We also show that by using nanoparticles as labels we can extend the applicability of the technique to complex samples. Computed stereo X-ray imaging will find application at X-ray free-electron lasers, synchrotrons and laser-based sources, and in industrial and medical 3D diagnosis methods.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: Experimental setup and sample for single-acquisition 3D stereo imaging.
Fig. 2: Typical dual diffraction pattern and stereo reconstruction of the amplitude of the sample.
Fig. 3: Amplitude and phase computed stereo reconstructions.
Fig. 4: Three-dimensional stereo imaging of an arrangement of nanoparticles.

Data availability

The data that support the plots within this paper and other findings of this study are available from the corresponding author on reasonable request.

References

  1. 1.

    Seibert, M. M. et al. Single mimivirus particles intercepted and imaged with an X-ray laser. Nature 470, 78–81 (2011).

    ADS  Article  Google Scholar 

  2. 2.

    Redecke, L. et al. Natively inhibited Trypanosoma brucei cathepsin B structure determined by using an X-ray laser. Science 339, 227–230 (2013).

    ADS  Article  Google Scholar 

  3. 3.

    Ravasio, A. et al. Single-shot diffractive imaging with a table-top femtosecond soft X-ray laser-harmonics source. Phys. Rev. Lett. 103, 028104 (2009).

    ADS  Article  Google Scholar 

  4. 4.

    Gauthier, D. et al. Single-shot femtosecond X-ray holography using extended references. Phys. Rev. Lett. 105, 093901 (2010).

    ADS  Article  Google Scholar 

  5. 5.

    Barty, A. et al. Ultrafast single-shot diffraction imaging of nanoscale dynamics. Nat. Photon. 2, 415–419 (2008).

    ADS  Article  Google Scholar 

  6. 6.

    Miao, J. et al. Beyond crystallography: diffractive imaging using coherent x-ray light sources. Science. 348, 530–535 (2015).

    ADS  MathSciNet  Article  Google Scholar 

  7. 7.

    Nishino, Y., Takahashi, Y., Imamoto, N., Ishikawa, T. & Maeshima, K. Three-dimensional visualization of a human chromosome using coherent X-ray diffraction. Phys. Rev. Lett. 102, 018101 (2009).

    ADS  Article  Google Scholar 

  8. 8.

    Takahashi, Y. et al. Three-dimensional electron density mapping of shape-controlled nanoparticle by focused hard X-ray diffraction microscopy. Nano Lett. 10, 1922–1926 (2010).

    ADS  Article  Google Scholar 

  9. 9.

    Chapman, H. N. et al. High-resolution ab initio three-dimensional x-ray diffraction microscopy. J. Opt. Soc. Am. A 23, 1179–1200 (2006).

    ADS  Article  Google Scholar 

  10. 10.

    Howells, M. R. et al. An assessment of the resolution limitation due to radiation-damage in x-ray diffraction microscopy. J. Electron Spectrosc. Relat. Phenom. 170, 4–12 (2009).

    Article  Google Scholar 

  11. 11.

    Thibault, P. et al. High-resolution scanning x-ray diffraction microscopy. Science 321, 379–382 (2008).

    ADS  Article  Google Scholar 

  12. 12.

    Dierolf, M. et al. Ptychographic X-ray computed tomography at the nanoscale. Nature 467, 436–439 (2010).

    ADS  Article  Google Scholar 

  13. 13.

    Holler, M. et al. An instrument for 3D x-ray nano-imaging. Rev. Sci. Instrum. 83, 073703 (2012).

    ADS  Article  Google Scholar 

  14. 14.

    Chapman, H. N. et al. Femtosecond diffractive imaging with a soft-X-ray free-electron laser. Nat. Phys. 2, 839–843 (2006).

    Article  Google Scholar 

  15. 15.

    Ekeberg, T. et al. Three-dimensional reconstruction of the giant mimivirus particle with an X-ray free-electron laser. Phys. Rev. Lett. 114, 098102 (2015).

    ADS  Article  Google Scholar 

  16. 16.

    Thomson, E. Stereoscopic Roentgen pictures. Electr. Eng. 21, 256 (1896).

    Google Scholar 

  17. 17.

    Andruleit, H., Geisen, M. & Stäger, S. Stereo-microscopy of coccolithophores-modern applications for imaging and morphological analysis. J. Nannoplankton Res. 28, 1–16 (2006).

    Article  Google Scholar 

  18. 18.

    Hoshino, M. et al. Development of an X-ray real-time stereo imaging technique using synchrotron radiation. J. Synchrot. Radiat. 18, 569–574 (2011).

    Article  Google Scholar 

  19. 19.

    Gleber, S.-C., Thieme, J., Chao, W. & Fischer, P. Stereo soft X-ray microscopy and elemental mapping of haematite and clay suspensions. J. Microsc. 235, 199–208 (2009).

    MathSciNet  Article  Google Scholar 

  20. 20.

    Miao, J. et al. High resolution 3D X-ray diffraction microscopy. Phys. Rev. Lett. 89, 088303 (2002).

    ADS  Article  Google Scholar 

  21. 21.

    Schmidt, K. E. et al. Tomographic femtosecond X-ray diffractive imaging. Phys. Rev. Lett. 101, 115507 (2008).

    ADS  Article  Google Scholar 

  22. 22.

    Gallagher-Jones, M. et al. Macromolecular structures probed by combining single-shot free-electron laser diffraction with synchrotron coherent X-ray imaging. Nat. Commun. 5, 3798 (2014).

    ADS  Article  Google Scholar 

  23. 23.

    Wei, H. Fundamental limits of ‘ankylography’ due to dimensional deficiency. Nature 480, E1 (2011).

    ADS  Article  Google Scholar 

  24. 24.

    Wang, G., Yu, H., Cong, W. & Katsevich, A. Non-uniqueness and instability of ‘ankylography’. Nature 480, E2 (2011).

    ADS  Article  Google Scholar 

  25. 25.

    Thibault, P. Feasibility of 3D reconstructions from a single 2D diffraction measurement. Preprint at http://arXiv.org/abs/0909.1643v2 (2010).

  26. 26.

    Chen, C.-C. et al. Three-dimensional imaging of a phase object from a single sample orientation using an optical laser. Phys. Rev. B 84, 224104 (2011).

    ADS  Article  Google Scholar 

  27. 27.

    Xu, R. et al. Single-shot three-dimensional structure determination of nanocrystals with femtosecond X-ray free-electron laser pulses. Nat. Commun. 5, 4061 (2014).

    ADS  Article  Google Scholar 

  28. 28.

    Yabashi, M. & Tanaka, H. The next ten years of X-ray science. Nat. Photon. 11, 12–14 (2017).

    ADS  Article  Google Scholar 

  29. 29.

    Marr, D. & Poggio, T. Cooperative computation of stereo disparity. Science 194, 283–287 (1976).

    ADS  Article  Google Scholar 

  30. 30.

    Elser, V. Phase retrieval by iterated projections. J. Opt. Soc. Am. A 20, 40–55 (2003).

    ADS  Article  Google Scholar 

  31. 31.

    Furukawa, Y. & Hernández, C. Multi-view stereo: a tutorial. Found. Trends Comput. Graph. Vis. 9, 1–148 (2013).

    Article  Google Scholar 

  32. 32.

    Longo, E. et al. 3D imaging of theranostic nanoparticles in mice organs by means of x-ray phase contrast tomography. Proc. SPIE 10573, 105734I (2018).

    Google Scholar 

  33. 33.

    Longo, E. et al. 3D map of theranostic nanoparticles distribution in mice brain and liver by means of X-ray phase contrast tomography. J. Instrum. 13, C01049 (2018).

    Article  Google Scholar 

  34. 34.

    Menk, R. H. et al. Gold nanoparticle labelling of cells is a sensitive method to investigate cell distribution and migration in animal models of human disease. Nanomed. Nanotechnol. Biol. Med. 7, 647–654 (2011).

    Article  Google Scholar 

  35. 35.

    Ortis, A., Rundo, F., Di Giore, G. & Battiato, S. Adaptive compression of stereoscopic images. In Proc. Int. Conf. Image Analysis and Processing 391–399 (Springer, 2013).

  36. 36.

    Lu, W. et al. Development of a hard X-ray split-and-delay line and performance simulations for two-color pump-probe experiments at the European XFEL. Rev. Sci. Instrum. 89, 063121 (2018).

    ADS  Article  Google Scholar 

  37. 37.

    Osaka, T. et al. Wavelength-tunable split-and-delay optical system for hard X-ray free-electron lasers. Opt. Express 24, 9187–9201 (2016).

    ADS  Article  Google Scholar 

  38. 38.

    Chen, B. et al. Multiple wavelength diffractive imaging. Phys. Rev. A 79, 023809 (2009).

    ADS  Article  Google Scholar 

  39. 39.

    Guizar-Sicairos, M., Thurman, S. T. & Fienup, J. R. Efficient subpixel image registration algorithms. Opt. Lett. 33, 156–158 (2008).

    ADS  Article  Google Scholar 

  40. 40.

    Amidror, I. Scattered data interpolation methods for electronic imaging systems: a survey. J. Electron. Imaging 11, 157–177 (2002).

    ADS  Article  Google Scholar 

  41. 41.

    Hartley, R. I. In defense of the eight-point algorithm. IEEE Trans. Pattern Anal. Mach. Intell. 19, 580–593 (1997).

    Article  Google Scholar 

  42. 42.

    Papadimitriou, D. V. & Dennis, T. J. Epipolar line estimation and rectification for stereo image pairs. IEEE Trans. Image Process. 5, 672–676 (1996).

    ADS  Article  Google Scholar 

  43. 43.

    Sobel, I. in Machine Vision for Three-Dimensional Sciences (ed. Freeman, H.) 376–379 (Academic Press, 1990).

Download references

Acknowledgements

We acknowledge financial support from the European Union through the Future and Emerging Technologies Open H2020: Volumetric medical X-ray imaging at extremely low dose (VOXEL) and the integrated initiative of the European laser research infrastructure (LASERLAB-EUROPE; grant agreement no. 654148). Support from the French ministry of research through the 2013 Agence Nationale de Recherche grant ‘NanoImagine’, 2014 ‘Ultrafast lensless imaging with plasmonic enhanced XUV generation’ and 2016 ‘High repetition rate laser for lensless imaging in the XUV’; from the Centre National de Compétences en Nanosciences research programme through the NanoscopiX grant; from the Laboratoire d’Excelence Physique Atoms Lumière Matière (ANR-10-LABX-0039-PALM), through grant ‘Plasmon-X’ and ‘High repetition rate laser harmonics in crystals’; and from the Action de Soutien à la Technologie et à la Recherche en Essonne programme through the ‘NanoLight’ grant are also acknowledged. We acknowledge financial support from the Deutsche Forschungsgemeinschaft, grant KO 3798/4-11, and from Lower Saxony through ‘Quanten- und Nanometrologie’, project NanoPhotonik. We acknowledge M. Kholodtsova for support on the CDI data treatment and discussions. We acknowledge A. Barty and Y. Nishino for data and discussions.

Author information

Affiliations

Authors

Contributions

J.D., R.C., J.H., W.B., B.I. and H.M. carried out the experiment. The samples were produced by F.F., L.D. and W.B. H.C. provided the 2D images of the pyramid. Simulations were performed by J.D. and R.C. Data analysis was performed by J.D., R.C., J.H., B.I., M.K., M.F., H.C., W.B. and H.M. W.B., M.K. and H.M. proposed the physical concept. All authors discussed the results and contributed to writing the manuscript.

Corresponding author

Correspondence to H. Merdji.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary notes and figures.

Supplementary Video 1

Reconstruction of the cross.

Supplementary Video 2

Reconstruction of the nanopyramid.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Duarte, J., Cassin, R., Huijts, J. et al. Computed stereo lensless X-ray imaging. Nat. Photonics 13, 449–453 (2019). https://doi.org/10.1038/s41566-019-0419-1

Download citation

Further reading

Search

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing