Detection and tracking of moving objects hidden from view

Abstract

The ability to detect motion and track a moving object hidden around a corner or behind a wall provides a crucial advantage when physically going around the obstacle is impossible or dangerous. Previous methods have demonstrated that it is possible to reconstruct the shape of an object hidden from view. However, these methods do not enable the tracking of movement in real time. We demonstrate a compact non-line-of-sight laser ranging technology that relies on the ability to send light around an obstacle using a scattering floor and then detect the return signal from a hidden object within only a few seconds of acquisition time. By detecting this signal with a single-photon avalanche diode (SPAD) camera, we follow the movement of an object located a metre away from the camera with centimetre precision. We discuss the possibility of applying this technology to a variety of real-life situations in the near future.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Figure 1: Looking around a corner.
Figure 2: Retrieving a hidden object's position.
Figure 3: Experimental results for position retrieval of the hidden object.
Figure 4: Non-line-of-sight tracking of a moving target.

References

  1. 1

    Kirmani, A. et al. First-photon imaging. Science 343, 58–61 (2014).

    ADS  Article  Google Scholar 

  2. 2

    Sun, B. et al. 3d computational imaging with single-pixel detectors. Science 340, 844–847 (2013).

    ADS  Article  Google Scholar 

  3. 3

    Wandinger, U. Introduction to Lidar (Springer, 2005).

    Google Scholar 

  4. 4

    Massa, J. S., Wallace, A. M., Buller, G. S., Fancey, S. J. & Walker, A. C. Laser depth measurement based on time-correlated single-photon counting. Opt. Lett. 22, 543–545 (1997).

    ADS  Article  Google Scholar 

  5. 5

    Velten, A. et al. Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nature Commun. 3, 745 (2012).

    ADS  Article  Google Scholar 

  6. 6

    Gao, L., Liang, J., Li, C. & Wang, L. V. Single-shot compressed ultrafast photography at one hundred billion frames per second. Nature 516, 74–77 (2015).

    ADS  Article  Google Scholar 

  7. 7

    Gariepy, G. et al. Single-photon sensitive light-in-flight imaging. Nature Commun. 6, 6021 (2015).

    ADS  Article  Google Scholar 

  8. 8

    Buttafava, M., Zeman, J., Tosi, A., Eliceiri, K. & Velten, A. Non-line-of-sight imaging using a time-gated single photon avalanche diode. Opt. Express 23, 20997 (2015).

    ADS  Article  Google Scholar 

  9. 9

    Sume, A. et al. Radar detection of moving objects around corners. Proc. SPIE 7308, 73080V (2009).

    Article  Google Scholar 

  10. 10

    Chakraborty, B. et al. in 2010 IEEE Int. Conf. Acoust, Speech Signal Process. 3894–3897 (2010).

  11. 11

    Gupta, O., Willwacher, T., Velten, A., Veeraraghavan, A. & Raskar, R. Reconstruction of hidden 3d shapes using diffuse reflections. Opt. Express 20, 19096–19108 (2012).

    ADS  Article  Google Scholar 

  12. 12

    Repasi, E. et al. Advanced short-wavelength infrared range-gated imaging for ground applications in monostatic and bistatic configurations. Appl. Opt. 48, 5956–5969 (2009).

    ADS  Article  Google Scholar 

  13. 13

    Vellekoop, I. M. & Mosk, A. P. Universal optimal transmission of light through disordered materials. Phys. Rev. Lett. 101, 120601 (2008).

    ADS  Article  Google Scholar 

  14. 14

    Mosk, A. P., Lagendijk, A., Lerosey, G. & Fink, M. Controlling waves in space and time for imaging and focusing in complex media. Nature Photon. 6, 283–292 (2012).

    ADS  Article  Google Scholar 

  15. 15

    Bertolotti, J. et al. Non-invasive imaging through opaque scattering layers. Nature 491, 232–234 (2012).

    ADS  Article  Google Scholar 

  16. 16

    Katz, O., Small, E. & Silberberg, Y. Looking around corners and through thin turbid layers in real time with scattered incoherent light. Nature Photon. 6, 459–553 (2012).

    Article  Google Scholar 

  17. 17

    Katz, O., Heidmann, P., Fink, M. & Gigan, S. Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations. Nature Photon. 8, 784–790 (2014).

    ADS  Article  Google Scholar 

  18. 18

    Richardson, J. et al. A 32×32 50ps resolution 10 bit time to digital converter array in 130nm CMOS for time correlated imaging. In Custom Integr. Circ. Conf. 2009 77–80 (IEEE, 2009).

  19. 19

    Richardson, J., Grant, L. & Henderson, R. Low dark count single-photon avalanche diode structure compatible with standard nanometer scale CMOS technology. Photon. Technol. Lett. IEEE 21, 1020–1022 (2009).

    ADS  Article  Google Scholar 

  20. 20

    Niclass, C., Rochas, A., Besse, P.-A. & Charbon, E. Design and characterization of a CMOS 3-d image sensor based on single photon avalanche diodes. IEEE J. Solid-State Circuits 40, 1847–1854 (2005).

    ADS  Article  Google Scholar 

  21. 21

    O'Connor, D. & Phillips, D. Time-Correlated Single Photon Counting (Academic Press, 1984).

    Google Scholar 

  22. 22

    Becker, W. Advanced Time-correlated Single Photon Counting Techniques (Springer, 2005).

    Google Scholar 

  23. 23

    Anderson, B. D. & Moore, J. B. Optimal Filtering (Prentice-Hall, 1979).

    Google Scholar 

  24. 24

    Cucchiara, R., Grana, C., Piccardi, M. & Prati, A. Detecting moving objects, ghosts, and shadows in video streams. IEEE Trans. Pattern Anal. Machine Intel. 25, 1337–1342 (2003).

    Article  Google Scholar 

  25. 25

    Cutler, R. & Davis, L. in Proc. Int. Conf. Pattern Recog. 1, 495 (IEEE, 1998).

    Google Scholar 

  26. 26

    Buller, G. S. & Wallace, A. M. Ranging and three-dimensional imaging using time-correlated single-photon counting and point-by-point acquisition. IEEE J. Sel. Top. Quant. Electron. 13, 1006–1015 (2007).

    ADS  Article  Google Scholar 

  27. 27

    Ho, C. et al. Demonstration of literal three-dimensional imaging. Appl. Opt. 38, 1833–1840 (1999).

    ADS  Article  Google Scholar 

  28. 28

    Albota, M. A. et al. Three-dimensional imaging laser radar with a photon-counting avalanche photodiode array and microchip laser. Appl. Opt. 41, 7671–7678 (2002).

    ADS  Article  Google Scholar 

Download references

Acknowledgements

We acknowledge support from the European Research Council under the European Union's Seventh Framework Programme (FP/2007-2013)/ERC GA 306559, the Engineering and Physical Sciences Research Council (EPSRC, UK, grants EP/M006514/1, EP/M01326X/1, EP/ K03197X/1), and ST Microelectronics, Imaging Division, Edinburgh, for their support in the manufacture of the Megaframe chip. The Megaframe project has been supported by the European Community within the Sixth Framework Programme IST FET Open. G.G. acknowledges the financial support of the Fonds de Recherche Nature et Technologies du Quebec (grant no. 173779).

Author information

Affiliations

Authors

Contributions

D.F. and J.L. conceived the experiment. R.K.H. designed the CMOS SPAD pixel architecture. G.G.performed the experiment. F.T. developed the tracking algorithm. G.G. and F.T. analysed the data and drafted the manuscript. All authors discussed the data and contributed to the manuscript.

Corresponding authors

Correspondence to Genevieve Gariepy or Daniele Faccio.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary information

Supplementary information (PDF 860 kb)

41566_2016_BFnphoton2015234_MOESM6_ESM.mp4

Supplementary Movie 1 (MP4 206 kb)

Supplementary information

Supplementary Movie 1 (MP4 206 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Gariepy, G., Tonolini, F., Henderson, R. et al. Detection and tracking of moving objects hidden from view. Nature Photon 10, 23–26 (2016). https://doi.org/10.1038/nphoton.2015.234

Download citation

Further reading