Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Bioinspired in-sensor visual adaptation for accurate perception

Nature Electronics volume 5, pages 84–91 (2022)Cite this article

Subjects

Abstract

Machine vision systems that capture images for visual inspection and identification tasks have to be able to perceive a scene under a range of illumination conditions. To achieve this, current systems use circuitry and algorithms that compromise efficiency and increase complexity. Here we report bioinspired vision sensors that are based on molybdenum disulfide phototransistors and exhibit time-varying activation and inhibition characteristics. Charge trap states are intentionally introduced into the surface of molybdenum disulfide, enabling the dynamic modulation of the photosensitivity of the devices under different lighting conditions. The light-intensity-dependent characteristics of the sensors match Weber’s law in which the perceived change in stimuli is proportional to the light stimuli. The approach offers visual adaptation with highly localized and dynamic modulation of photosensitivity under different lighting conditions at the pixel level, creating an effective perception range of up to 199 dB. The phototransistor arrays exhibit image contrast enhancement for both scotopic and photopic adaptation.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Visual adaptation of the retina.
Fig. 2: Light-intensity-dependent characteristics of the MoS2 phototransistor.
Fig. 3: Time-dependent characteristics of the MoS2 phototransistor.
Fig. 4: Scotopic and photopic adaptation of MoS2 phototransistor array.

Similar content being viewed by others

Data availability

Source data are provided with this paper. The data that support the plots within these paper and other findings of this study are available from the corresponding authors upon reasonable request.

Code availability

The codes used for simulation and data plotting are available from the corresponding authors upon reasonable request.

References

  1. Gu, L. et al. A biomimetic eye with a hemispherical perovskite nanowire array retina. Nature 581, 278–282 (2020).

    Article  Google Scholar 

  2. Mennel, L. et al. Ultrafast machine vision with 2D material neural network image sensors. Nature 579, 62–66 (2020).

    Article  Google Scholar 

  3. Chai, Y. In-sensor computing for machine vision. Nature 579, 32–33 (2020).

    Article  Google Scholar 

  4. Zhou, F. & Chai, Y. Near-sensor and in-sensor computing. Nat. Electron. 3, 664–671 (2020).

    Article  Google Scholar 

  5. Liao, F., Zhou, F. & Chai, Y. Neuromorphic vision sensors: principle, progress and perspectives. J. Semicond. 41, 013105 (2020).

    Google Scholar 

  6. Darmont, A. High dynamic range imaging: sensors and architectures. In Society of Photo-Optical Instrumentation Engineers (SPIE, 2013).

  7. Wang, C.-Y. et al. Gate-tunable van der Waals heterostructure for reconfigurable neural network vision sensor. Sci. Adv. 6, eaba6173 (2020).

    Article  Google Scholar 

  8. Sun, L. et al. In-sensor reservoir computing for language learning via two-dimensional memristors. Sci. Adv. 7, eabg1455 (2021).

    Article  Google Scholar 

  9. Ohta, J. Smart CMOS Image Sensors and Applications (CRC Press, 2017).

  10. Liba, O. et al. Handheld mobile photography in very low light. ACM Trans. Graph. 38, 164 (2019).

    Article  Google Scholar 

  11. Rao, Z. et al. Curvy, shape-adaptive imagers based on printed optoelectronic pixels with a kirigami design. Nat. Electron. 4, 513–521 (2021).

    Article  Google Scholar 

  12. Kim, M. S. et al. An aquatic-vision-inspired camera based on a monocentric lens and a silicon nanorod photodiode array. Nat. Electron. 3, 546–553 (2020).

    Article  Google Scholar 

  13. Xie, D. Xie et al. Photoelectric visual adaptation based on 0D‐CsPbBr3‐quantum‐dots/2D‐MoS2 mixed‐dimensional heterojunction transistor. Adv. Funct. Mater. 31, 2010655 (2021).

    Article  Google Scholar 

  14. Hong, S. et al. Sensory adaptation and neuromorphic phototransistors based on CsPb(Br1–xIx)3 perovskite and MoS2 hybrid structure. ACS Nano 14, 9796–9806 (2020).

    Article  Google Scholar 

  15. Kwon, S. M. et al. Environment-adaptable artificial visual perception behaviors using a light-adjustable optoelectronic neuromorphic device array. Adv. Mater. 31, 1906433 (2019).

    Article  Google Scholar 

  16. He, Z. et al. An organic transistor with light intensity-dependent active photoadaptation. Nat. Electron. 4, 522–529 (2021).

    Article  Google Scholar 

  17. Wang, Q. H., Kalantar-Zadeh, K., Kis, A., Coleman, J. N. & Strano, M. S. Electronics and optoelectronics of two-dimensional transition metal dichalcogenides. Nat. Nanotechnol. 7, 699–712 (2012).

    Article  Google Scholar 

  18. Fiori, G. et al. Electronics based on two-dimensional materials. Nat. Nanotechnol. 9, 768–779 (2014).

    Article  Google Scholar 

  19. Liu, Y. et al. Promises and prospects of two-dimensional transistors. Nature 591, 43–53 (2021).

    Article  Google Scholar 

  20. Jang, H. et al. An atomically thin optoelectronic machine vision processor. Adv. Mater. 32, e2002431 (2020).

    Article  Google Scholar 

  21. Choi, C. et al. Human eye-inspired soft optoelectronic device using high-density MoS2-graphene curved image sensor array. Nat. Commun. 8, 1664 (2017).

    Article  Google Scholar 

  22. Choi, C. et al. Curved neuromorphic image sensor array using a MoS2-organic heterostructure inspired by the human visual recognition system. Nat. Commun. 11, 5934 (2020).

    Article  Google Scholar 

  23. Lopez-Sanchez, O., Lembke, D., Kayci, M., Radenovic, A. & Kis, A. Ultrasensitive photodetectors based on monolayer MoS2. Nat. Nanotechnol. 8, 497–501 (2013).

    Article  Google Scholar 

  24. Nur, R. et al. High responsivity in MoS2 phototransistors based on charge trapping HfO2 dielectrics. Commun. Mater. 1, 103 (2020).

    Article  Google Scholar 

  25. Lee, J. et al. Monolayer optical memory cells based on artificial trap-mediated charge storage and release. Nat. Commun. 8, 14734 (2017).

    Article  Google Scholar 

  26. Miller, R. E. & Tredici, T. J. Night Vision Manual for the Flight Surgeon. (ARMSTRONG LAB BROOKS AFB TX, 1992).

  27. Seetzen, H. et al. High dynamic range display systems. In ACM SIGGRAPH 2004 Papers 760–768 (Association for Computing Machinery, 2004).

  28. Kalloniatis, M. & Luu, C. Light and dark adaptation. In Webvision: The Organization of the Retina and Visual System (eds Kolb, H. et al.) (Univ. Utah Health Sciences Center, 1995).

  29. Fechner, G. Elements of Psychophysics Vol. I (Holt, Rinehart and Winston, 1966).

  30. Kandel, E. R. et al. Principles of Neural Science Vol. 4 (McGraw-Hill, 2000).

  31. Meister, M. & Tessier-Lavigne, M. Low-level visual processing: the retina. Prin. Neural Sci. 5, 577–601 (2013).

    Google Scholar 

  32. Jiang, J. et al. Rational design of Al2O3/2D perovskite heterostructure dielectric for high performance MoS2 phototransistors. Nat. Commun. 11, 4266 (2020).

    Article  Google Scholar 

  33. Park, Y., Baac, H. W., Heo, J. & Yoo, G. Thermally activated trap charges responsible for hysteresis in multilayer MoS2 field-effect transistors. Appl. Phys. Lett. 108, 083102 (2016).

    Article  Google Scholar 

  34. Kaushik, N. et al. Reversible hysteresis inversion in MoS2 field effect transistorsr. npj 2D Mater. Appl. 1, 34 (2017).

    Article  Google Scholar 

  35. Fang, H. & Hu, W. Photogating in low dimensional photodetectors. Adv. Sci. 4, 1700323 (2017).

    Article  Google Scholar 

  36. Wu, J. Y. et al. Broadband MoS2 field-effect phototransistors: ultrasensitive visible-light photoresponse and negative infrared photoresponse. Adv. Mater. 30, 1705880 (2018).

    Article  Google Scholar 

  37. Kufer, D. & Konstantatos, G. Highly sensitive, encapsulated MoS2 photodetector with gate controllable gain and speed. Nano Lett. 15, 7307–7313 (2015).

    Article  Google Scholar 

  38. Pierre, A., Gaikwad, A. & Arias, A. C. Charge-integrating organic heterojunction phototransistors for wide-dynamic-range image sensors. Nat. Photon. 11, 193–199 (2017).

    Article  Google Scholar 

  39. Gong, X. et al. High-detectivity polymer photodetectors with spectral response from 300 nm to 1450 nm. Science 325, 1665–1667 (2009).

    Article  Google Scholar 

  40. Peng, B. et al. Achieving ultrafast hole transfer at the monolayer MoS2 and CH3NH3PbI3 perovskite interface by defect engineering. ACS Nano 10, 6383–6391 (2016).

    Article  Google Scholar 

  41. Chen, M. et al. Multibit data storage states formed in plasma-treated MoS2 transistors. ACS Nano 8, 4023–4032 (2014).

    Article  Google Scholar 

  42. Zhou, F. et al. Optoelectronic resistive random access memory for neuromorphic vision sensors. Nat. Nanotechnol. 14, 776–782 (2019).

    Article  Google Scholar 

  43. Choi, M. et al. Full-color active-matrix organic light-emitting diode display on human skin based on a large-area MoS2 backplane. Sci. Adv. 6, eabb5898 (2020).

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by China Postdoctoral Science Foundation (2021M692221); Research Grant Council of Hong Kong (15205619); Science, Technology and Innovation Commission of Shenzhen (JCYJ20180507183424383 and SGDX2020110309540000); and the Hong Kong Polytechnic University (1-ZE1T and 1-ZVGH). J.-H.A. acknowledges support from the National Research Foundation of Korea (NRF-2015R1A3A2066337).

Author information

Authors and Affiliations

  1. The Hong Kong Polytechnic University Shenzhen Research Institute, Shenzhen, China

    Fuyou Liao, Jiewei Chen, Jingli Wang, Cong Wang & Yang Chai

  2. Department of Applied Physics, The Hong Kong Polytechnic University, Hong Kong, China

    Fuyou Liao, Jiewei Chen, Jingli Wang, Tianqing Wan, Yue Zhou, Cong Wang & Yang Chai

  3. Institute of Microelectronics, Peking University, Beijing, China

    Zheng Zhou & Jinfeng Kang

  4. School of Electrical and Electronic Engineering, Yonsei University, Seoul, Republic of Korea

    Beom Jin Kim, Anh Tuan Hoang & Jong-Hyun Ahn

  5. Frontier Institute of Chip and System, Fudan University, Shanghai, China

    Jingli Wang

  6. Research Institute of Intelligent Wearable Systems, The Hong Kong Polytechnic University, Hong Kong, China

    Yang Chai

Authors
  1. Fuyou Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zheng Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Beom Jin Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiewei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jingli Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tianqing Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yue Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Anh Tuan Hoang
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Cong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Jinfeng Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Jong-Hyun Ahn
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Yang Chai
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Y.C. conceived the concept and supervised the project. F.L. designed the test protocol and performed the experiments. B.J.K., A.T.H. and J.-H.A. fabricated the devices. F.L., J.C. and J.W. analysed the experimental data. Z.Z., C.W. and J.K. performed the simulations. F.L., Z.Z., T.W., Y.Z. and Y.C. co-wrote the paper. All the authors discussed the results and commented on the manuscript.

Corresponding authors

Correspondence to Jong-Hyun Ahn or Yang Chai.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Electronics thanks Sunkook Kim, Tse Nga Ng and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Figs. 1–18 and Tables 1 and 2.

Source data

Source Data Fig. 2

Light-intensity-dependent characteristics of the MoS2 phototransistor.

Source Data Fig. 3

Time-dependent characteristics of the MoS2 phototransistor.

Source Data Fig. 4

Scotopic and photopic adaptation of the MoS2 phototransistor array.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liao, F., Zhou, Z., Kim, B.J. et al. Bioinspired in-sensor visual adaptation for accurate perception. Nat Electron 5, 84–91 (2022). https://doi.org/10.1038/s41928-022-00713-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41928-022-00713-1

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing