An aquatic-vision-inspired camera based on a monocentric lens and a silicon nanorod photodiode array

Abstract

Conventional wide-field-of-view cameras consist of multi-lens optics and flat image sensor arrays, which makes them bulky and heavy. As a result, they are poorly suited to advanced mobile applications such as drones and autonomous vehicles. In nature, the eyes of aquatic animals consist of a single spherical lens and a highly sensitive hemispherical retina, an approach that could be beneficial in the development of synthetic wide-field-of-view imaging systems. Here, we report an aquatic-vision-inspired camera that consists of a single monocentric lens and a hemispherical silicon nanorod photodiode array. The imaging system features a wide field of view, miniaturized design, low optical aberration, deep depth of field and simple visual accommodation. Furthermore, under vignetting, the photodiode array enables high-quality panoramic imaging due to the enhanced photodetection properties of the silicon nanorod photodiodes.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: Structural and functional features of aquatic vision in nature.
Fig. 2: Monocentric lens inspired by the protruding monocentric lens of aquatic eyes.
Fig. 3: h-SiNR-PDA inspired by the retina of aquatic eyes.
Fig. 4: Imaging demonstration with the integrated camera module.

Data availability

The data that support the plots within this paper and other findings of this study are available from the corresponding author upon reasonable request.

Code availability

The source codes for Matlab are available from the corresponding authors upon request.

References

  1. 1.

    Floreano, D. & Wood, R. J. Science, technology and the future of small autonomous drones. Nature 521, 460–466 (2015).

    Article  Google Scholar 

  2. 2.

    Jang, H. S. et al. A bezel-less tetrahedral image sensor formed by solvent-assisted plasticization and transformation of an acrylonitrile butadiene styrene framework. Adv. Mater. 30, 1801256 (2018).

    Article  Google Scholar 

  3. 3.

    Lee, G. J., Choi, C., Kim, D.-H. & Song, Y. M. Bioinspired artificial eyes: optic components, digital cameras and visual prostheses. Adv. Funct. Mater. 28, 1705202 (2018).

    Article  Google Scholar 

  4. 4.

    Chung, T. et al. Mining the smartness of insect ultrastructures for advanced imaging and illumination. Adv. Funct. Mater. 28, 1705912 (2018).

    Article  Google Scholar 

  5. 5.

    Lee, G. J., Nam, W. I. & Song, Y. M. Robustness of an artificially tailored fisheye imaging system with a curvilinear image surface. Opt. Laser Technol. 96, 50–57 (2017).

    Article  Google Scholar 

  6. 6.

    Zhou, F. et al. Optoelectronic resistive random access memory for neuromorphic vision sensors. Nat. Nanotechnol. 14, 776–782 (2019).

    Article  Google Scholar 

  7. 7.

    Lee, W. et al. Two-dimensional materials in functional three-dimensional architectures with applications in photodetection and imaging. Nat. Commun. 9, 1417 (2018).

    Article  Google Scholar 

  8. 8.

    Tsai, W.-L. et al. Band tunable microcavity perovskite artificial human photoreceptors. Adv. Mater. 31, 1900231 (2019).

    Article  Google Scholar 

  9. 9.

    Choi, C. et al. Human eye-inspired soft optoelectronic device using high-density MoS2–graphene curved image sensor array. Nat. Commun. 8, 1664 (2017).

    Article  Google Scholar 

  10. 10.

    Zhang, K. et al. Origami silicon optoelectronics for hemispherical electronic eye systems. Nat. Commun. 8, 1782 (2017).

    Article  Google Scholar 

  11. 11.

    Ko, H. C. et al. A hemispherical electronic eye camera based on compressible silicon optoelectronics. Nature 454, 748–753 (2008).

    Article  Google Scholar 

  12. 12.

    Artal, P. Optics of the eye and its impact in vision: a tutorial. Adv. Opt. Photon. 6, 340–367 (2014).

    Article  Google Scholar 

  13. 13.

    Song, Y. M. et al. Digital cameras with designs inspired by the arthropod eye. Nature 497, 95–99 (2013).

    Article  Google Scholar 

  14. 14.

    Floreano, D. et al. Miniature curved artificial compound eyes. Proc. Natl Acad. Sci. USA 110, 9267–9272 (2013).

    Article  Google Scholar 

  15. 15.

    Jeong, K.-H., Kim, J. & Lee, L. P. Biologically inspired artificial compound eyes. Science 312, 557–561 (2006).

    Article  Google Scholar 

  16. 16.

    Huang, C.-C. et al. Large-field-of-view wide-spectrum artificial reflecting superposition compound eyes. Small 10, 3050–3057 (2014).

    Article  Google Scholar 

  17. 17.

    Jagger, W. S. & Sands, P. J. A wide-angle gradient index optical model of the crystalline lens and eye of the rainbow trout. Vis. Res. 36, 2623–2639 (1996).

    Article  Google Scholar 

  18. 18.

    Jagger, W. S. & Sands, P. J. A wide-angle gradient index optical model of the crystalline lens and eye of the octopus. Vis. Res. 39, 2841–2852 (1999).

    Article  Google Scholar 

  19. 19.

    Mass, A. M. & Supin, A. Y. Adaptive features of aquatic mammals’ eye. Anat. Rec. 290, 701–715 (2007).

    Article  Google Scholar 

  20. 20.

    Charman, W. N. & Tucker, J. The optical system of the goldfish eye. Vis. Res. 13, 1–8 (1973).

    Article  Google Scholar 

  21. 21.

    Ott, M. Visual accommodation in vertebrates: mechanisms, physiological response and stimuli. J. Comp. Physiol. A 192, 97–111 (2006).

    Article  Google Scholar 

  22. 22.

    Wagner, H.-J., Frohlich, E., Negishi, K. & Collin, S. P. The eyes of deep-sea fish. II. Functional morphology of the retina. Prog. Retin. Eye Res. 17, 637–685 (1998).

    Article  Google Scholar 

  23. 23.

    Partridge, J. C., Archer, S. N. & Lythgoe, J. N. Visual pigments in the individual rods of deep-sea fishes. J. Comp. Physiol. A 162, 543–550 (1988).

    Article  Google Scholar 

  24. 24.

    Wu, T. et al. Design and fabrication of silicon-tessellated structures for monocentric imagers. Microsyst. Nanoeng. 2, 16019 (2016).

    Article  Google Scholar 

  25. 25.

    Liu, H. W., Huang, Y. & Jiang, H. Artificial eye for scotopic vision with bioinspired all-optical photosensitivity enhancer. Proc. Natl Acad. Sci. USA 113, 3982–3985 (2016).

    Article  Google Scholar 

  26. 26.

    Zukauskas, A. et al. Tuning the refractive index in 3D direct laser writing lithography: towards GRIN microoptics. Laser Photon. Rev. 9, 706–712 (2015).

    Article  Google Scholar 

  27. 27.

    Yu, N. & Capasso, F. Flat optics with designer metasurfaces. Nat. Mater. 13, 139–150 (2014).

    Article  Google Scholar 

  28. 28.

    Steel, W. H. On the choice of glasses for cemented achromatic aplanatic doublets. Aust. J. Phys. 7, 244–253 (1954).

    Article  Google Scholar 

  29. 29.

    Lee, W. et al. High-resolution spin-on-patterning of perovskite thin films for a multiplexed image sensor array. Adv. Mater. 29, 1702902 (2017).

    Article  Google Scholar 

  30. 30.

    Sim, K. et al. Three-dimensional curvy electronics created using conformal additive stamp printing. Nat. Electron. 2, 471–479 (2019).

    Article  Google Scholar 

  31. 31.

    Shin, G. et al. Micromechanics and advances designs for curved photodetector arrays in hemispherical electronic-eye cameras. Small 6, 851–856 (2010).

    Article  Google Scholar 

  32. 32.

    Huang, Z. et al. Three-dimensional integrated stretchable electronics. Nat. Electron. 1, 473–480 (2018).

    Article  Google Scholar 

  33. 33.

    Park, S.-I. et al. Theoretical and experimental studies of bending of inorganic electronic materials on plastic substrates. Adv. Funct. Mater. 18, 2673–2684 (2008).

    Article  Google Scholar 

  34. 34.

    Choi, M. K. et al. Wearable red-green-blue quantum dot light-emitting diode array using high-resolution intaglio transfer printing. Nat. Commun. 6, 7149 (2015).

    Article  Google Scholar 

  35. 35.

    Kim, J. et al. Stretchable silicon nanoribbon electronics for skin prosthesis. Nat. Commun. 5, 5747 (2014).

    Article  Google Scholar 

  36. 36.

    Rim, S.-B., Catrysse, P. B., Dinyari, R., Huang, K. & Peumans, P. The optical advantages of curved focal plane arrays. Opt. Express 16, 4965–4971 (2008).

    Article  Google Scholar 

  37. 37.

    Sheng, X., Johnson, S. G., Michel, J. & Kimerling, L. C. Optimization-based design of surface textures for thin-film Si solar cells. Opt. Express 19, A841–A850 (2011).

    Article  Google Scholar 

  38. 38.

    Sheng, X. et al. Printing-based assembly of quadruple-junction four-terminal microscale solar cells and their use in high-efficiency modules. Nat. Mater. 13, 593–598 (2014).

    Article  Google Scholar 

  39. 39.

    Hoang, N.-V. et al. Giant enhancement of luminescence down-shifting by a doubly resonant rare-earth-doped photonic metastructure. ACS Photonics 4, 1705–1712 (2017).

    Article  Google Scholar 

  40. 40.

    Savin, H. et al. Black silicon solar cells with interdigitated back-contacts achieve 22.1% efficiency. Nat. Nanotechnol. 10, 624–628 (2015).

    Article  Google Scholar 

  41. 41.

    Gao, Y. et al. Photon-trapping microstructures enable high-speed high-efficiency silicon photodiodes. Nat. Photon. 11, 301–308 (2017).

    Article  Google Scholar 

  42. 42.

    Lucovsky, G., Schwarz, R. F. & Emmons, R. B. Transit‐time considerations in p–i–n diodes. J. Appl. Phys. 35, 622–628 (1964).

    Article  Google Scholar 

  43. 43.

    Kyomasu, Mikio Development of an integrated high speed silicon PIN photodiode sensor. IEEE Trans. Electron Devices 42, 1093–1099 (1995).

    Article  Google Scholar 

  44. 44.

    Gao, M., Cho, M., Han, H.-J., Jung, Y. S. & Park, I. Palladium-decorated silicon nanomesh fabricated by nanosphere lithography for high performance, room temperature hydrogen sensing. Small 14, 1703691 (2018).

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by the Institute for Basic Science (IBS-R006-A1). This research was also supported by the National Research Foundation (NRF) of Korea (2017M3D1A1039288/2018R1A4A1025623).

Author information

Affiliations

Authors

Contributions

Min Sung Kim, G.J.L., C.C., Min Seok Kim, K.W.C., Y.M.S. and D.-H.K. designed the experiments, analysed the data and wrote the paper. Min Sung Kim, C.C., M.L., H.C. and M.K.C. fabricated the photodiode array and performed characterization of individual devices. G.J.L., Min Seok Kim and H.M.K. performed theoretical analysis on optics. S.L. and N.L. performed theoretical analysis on mechanics. All authors discussed the results and commented on the manuscript.

Corresponding authors

Correspondence to Young Min Song or Dae-Hyeong Kim.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Notes 1–6, Figs. 1–35 and Tables 1–6.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kim, M.S., Lee, G.J., Choi, C. et al. An aquatic-vision-inspired camera based on a monocentric lens and a silicon nanorod photodiode array. Nat Electron (2020). https://doi.org/10.1038/s41928-020-0429-5

Download citation