A low-power biomimetic collision detector based on an in-memory molybdenum disulfide photodetector

Abstract

Accurately detecting a potential collision and triggering a timely escape response is critical in the field of robotics and autonomous vehicle safety. The lobula giant movement detector (LGMD) neuron in locusts can detect an approaching object and prevent collisions within a swarm of millions of locusts. This single neuronal cell performs nonlinear mathematical operations on visual stimuli to elicit an escape response with minimal energy expenditure. Collision avoidance models based on image processing algorithms have been implemented using analogue very-large-scale-integration designs, but none is as efficient as this neuron in terms of energy consumption or size. Here we report a nanoscale collision detector that mimics the escape response of the LGMD neuron. The detector comprises a monolayer molybdenum disulfide photodetector stacked on top of a non-volatile and programmable floating-gate memory architecture. It consumes a small amount of energy (in the range of nanojoules) and has a small device footprint (~1 µm × 5 µm).

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: Biological and artificial collision detectors.
Fig. 2: Monolayer MoS2 photodetector and its response to looming stimuli.
Fig. 3: Inhibitory response of MoS2 FET to programming stimuli.
Fig. 4: Mimicking the escape response of LGMD neurons.
Fig. 5: Compact model for a biomimetic collision detector.

Data availability

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

Code availability

The codes used for plotting the data are available from the corresponding author on reasonable request.

References

  1. 1.

    Wang, Y. & Frost, B. J. Time to collision is signalled by neurons in the nucleus rotundus of pigeons. Nature 356, 236–238 (1992).

    Google Scholar 

  2. 2.

    Preuss, T., Osei-Bonsu, P. E., Weiss, S. A., Wang, C. & Faber, D. S. Neural representation of object approach in a decision-making motor circuit. J. Neurosci. 26, 3454–3464 (2006).

    Google Scholar 

  3. 3.

    Maier, J. X., Neuhoff, J. G., Logothetis, N. K. & Ghazanfar, A. A. Multisensory integration of looming signals by rhesus monkeys. Neuron 43, 177–181 (2004).

    Google Scholar 

  4. 4.

    Rind, F. C. Intracellular characterization of neurons in the locust brain signaling impending collision. J. Neurophysiol. 75, 986–995 (1996).

    Google Scholar 

  5. 5.

    Fotowat, H. & Gabbiani, F. Collision detection as a model for sensory-motor integration. Annu. Rev. Neurosci. 34, 1–19 (2011).

    Google Scholar 

  6. 6.

    Gray, J. R., Blincow, E. & Robertson, R. M. A pair of motion-sensitive neurons in the locust encode approaches of a looming object. J. Comp. Physiol. A 196, 927–938 (2010).

    Google Scholar 

  7. 7.

    Tammero, L. F. & Dickinson, M. H. Collision-avoidance and landing responses are mediated by separate pathways in the fruit fly, Drosophila melanogaster. J. Exp. Biol. 205, 2785–2798 (2002).

    Google Scholar 

  8. 8.

    Glantz, R. M. Defense reflex and motion detector responsiveness to approaching targets: the motion detector trigger to the defense reflex pathway. J. Comp. Physiol. 95, 297–314 (1974).

    Google Scholar 

  9. 9.

    Oliva, D., Medan, V. & Tomsic, D. Escape behavior and neuronal responses to looming stimuli in the crab Chasmagnathus granulatus (Decapoda: Grapsidae). J. Exp. Biol. 210, 865–880 (2007).

    Google Scholar 

  10. 10.

    Poggio, T. & Reichardt, W. Considerations on models of movement detection. Kybernetik 13, 223–227 (1973).

    Google Scholar 

  11. 11.

    Tanner, J. & Mead, C. in VLSI Signal Processing II Ch. 7 (IEEE, 1986).

  12. 12.

    Borst, A. Models of motion detection. Nat. Neurosci. 3, 1168 (2000).

    Google Scholar 

  13. 13.

    Andreou, A. G. & Strohbehn, K. Analog VLSI implementation of the Hassenstein-Reichardt-Poggio models for vision computation. In Proc. 1990 IEEE International Conference on Systems, Man, and Cybernetics 707–710 (IEEE, 1990).

  14. 14.

    Harrison, R. R. & Koch, C. A robust analog VLSI motion sensor based on the visual system of the fly. Auton. Robots 7, 211–224 (1999).

    Google Scholar 

  15. 15.

    Horridge, G. A. The compromise between seeing spatial layout and making visual discriminations. Curr. Sci. 60, 686–693 (1991).

    Google Scholar 

  16. 16.

    Abbott, D. et al. New VLSI smart sensor for collision avoidance inspired by insect vision. In Proc. SPIE 2344 Intelligent Vehicle Highway Systems 105–115 (SPIE, 1995).

  17. 17.

    Moini, A. et al. An insect vision-based motion detection chip. IEEE J. Solid-State Circuits 32, 279–284 (1997).

    Google Scholar 

  18. 18.

    Moini, A., Bouzerdoum, A., Yakovleff, A. & Eshraghian, K. Two-dimensional motion detector based on insect vision. In Proc. SPIE 2950 Advanced Focal Plane Arrays and Electronic Cameras 146–157 (SPIE, 1996).

  19. 19.

    Abbott, D. et al. Status of recent developments in collision avoidance using motion detectors based on insect vision. In Proc. SPIE 2902 Transportation Sensors and Controls: Collision Avoidance, Traffic Management, and ITS 242–247 (SPIE, 1997).

  20. 20.

    Okuno, H. & Yagi, T. A mixed analog–digital vision sensor for detecting objects approaching on a collision course. Rob. Autom. Syst. 57, 508–516 (2009).

    Google Scholar 

  21. 21.

    Takami, R., Shimonomura, K., Kameda, S. & Yagi, T. An image pre-processing system employing neuromorphic 100/spl times/100 pixel silicon retina [robot vision applications]. In Proc. 2005 IEEE International Symposium on Circuits and Systems 2771–2774 (IEEE, 2005).

  22. 22.

    Harrison, R. R. A biologically inspired analog IC for visual collision detection. IEEE Trans. Circuits Syst. I 52, 2308–2318 (2005).

    Google Scholar 

  23. 23.

    Sarkar, M., Bello, D. S. S., van Hoof, C. & Theuwissen, A. J. Biologically inspired CMOS image sensor for fast motion and polarization detection. IEEE Sens. J. 13, 1065–1073 (2012).

    Google Scholar 

  24. 24.

    Zhang, C., Lindner, S., Antolovic, I. M., Wolf, M. & Charbon, E. A CMOS SPAD imager with collision detection and 128 dynamically reallocating TDCs for single-photon counting and 3D time-of-flight imaging. Sensors 18, 4016 (2018).

    Google Scholar 

  25. 25.

    Baxter, J. A., Merced, D. A., Costinett, D. J., Tolbert, L. M. & Ozpineci, B. Review of electrical architectures and power requirements for automated vehicles. In Proc. 2018 IEEE Transportation Electrification Conference and Expo (ITEC) 944–949 (IEEE, 2018).

  26. 26.

    Herberholz, J. & Marquart, G. D. Decision making and behavioral choice during predator avoidance. Front. Neurosci. 6, 125 (2012).

    Google Scholar 

  27. 27.

    Rind, F. C. & Bramwell, D. Neural network based on the input organization of an identified neuron signaling impending collision. J. Neurophysiol. 75, 967–985 (1996).

    Google Scholar 

  28. 28.

    Gabbiani, F., Krapp, H. G., Koch, C. & Laurent, G. Multiplicative computation in a visual neuron sensitive to looming. Nature 420, 320–324 (2002).

    Google Scholar 

  29. 29.

    Gabbiani, F., Krapp, H. G. & Laurent, G. Computation of object approach by a wide-field, motion-sensitive neuron. J. Neurosci. 19, 1122–1141 (1999).

    Google Scholar 

  30. 30.

    Badia, S. B. I., Bernardet, U. & Verschure, P. F. Non-linear neuronal responses as an emergent property of afferent networks: a case study of the locust lobula giant movement detector. PLoS Comput. Biol. 6, e1000701 (2010).

    MathSciNet  Google Scholar 

  31. 31.

    Judge, S. & Rind, F. The locust DCMD, a movement-detecting neurone tightly tuned to collision trajectories. J. Exp. Biol. 200, 2209–2216 (1997).

    Google Scholar 

  32. 32.

    Pearson, K., Heitler, W. & Steeves, J. Triggering of locust jump by multimodal inhibitory interneurons. J. Neurophysiol. 43, 257–278 (1980).

    Google Scholar 

  33. 33.

    Rind, F. C. A chemical synapse between two motion detecting neurones in the locust brain. J. Exp. Biol. 110, 143–167 (1984).

    Google Scholar 

  34. 34.

    Blanchard, M., Rind, F. C. & Verschure, P. F. M. J. Collision avoidance using a model of the locust LGMD neuron. Rob. Autom. Syst. 30, 17–38 (2000).

    Google Scholar 

  35. 35.

    Fu, Q. et al. A visual neural network for robust collision perception in vehicle driving scenarios. In Proc. IFIP International Conference on Artificial Intelligence Applications and Innovations 67–79 (Springer, 2019).

  36. 36.

    Stafford, R., Santer, R. D. & Rind, F. C. A bio-inspired visual collision detection mechanism for cars: combining insect inspired neurons to create a robust system. Biosystems 87, 164–171 (2007).

    Google Scholar 

  37. 37.

    Yue, S., Rind, F. C., Keil, M. S., Cuadri, J. & Stafford, R. A bio-inspired visual collision detection mechanism for cars: optimisation of a model of a locust neuron to a novel environment. Neurocomputing 69, 1591–1598 (2006).

    Google Scholar 

  38. 38.

    Santer, R. D., Stafford, R. & Rind, F. C. Retinally-generated saccadic suppression of a locust looming-detector neuron: investigations using a robot locust. J. R. Soc. Interface 1, 61–77 (2004).

    Google Scholar 

  39. 39.

    Cuadri, J., Linan, G., Stafford, R., Keil, M. & Roca, E. A bioinspired collision detection algorithm for VLSI implementation. In Proc SPIE 5839 Bioengineered and Bioinspired Systems II 238–248 (SPIE, 2005).

  40. 40.

    Lopez-Sanchez, O., Lembke, D., Kayci, M., Radenovic, A. & Kis, A. Ultrasensitive photodetectors based on monolayer MoS2. Nat. Nanotechnol. 8, 497–501 (2013).

    Google Scholar 

  41. 41.

    Cappelletti, P., Golla, C., Olivo, P. & Zanoni, E. Flash Memories (Springer, 2013).

  42. 42.

    Lee, J. et al. Monolayer optical memory cells based on artificial trap-mediated charge storage and release. Nat. Commun. 8, 14734 (2017).

    Google Scholar 

  43. 43.

    Arnold, A. J. et al. Mimicking neurotransmitter release in chemical synapses via hysteresis engineering in MoS2 transistors. ACS Nano 11, 3110–3118 (2017).

    Google Scholar 

  44. 44.

    Sup Choi, M. et al. Controlled charge trapping by molybdenum disulphide and graphene in ultrathin heterostructured memory devices. Nat. Commun. 4, 1624 (2013).

    Google Scholar 

  45. 45.

    Paul, T., Ahmed, T., Tiwari, K. K., Thakur, C. S. & Ghosh, A. A high-performance MoS2 synaptic device with floating gate engineering for neuromorphic computing. 2D Mater. 6, 045008 (2019).

    Google Scholar 

  46. 46.

    Lundstrom, M. S. & Antoniadis, D. A. Compact models and the physics of nanoscale FETs. IEEE Trans. Electron Devices 61, 225–233 (2014).

    Google Scholar 

  47. 47.

    Das, S., Dodda, A. & Das, S. A biomimetic 2D transistor for audiomorphic computing. Nat. Commun. 10, 3450 (2019).

    Google Scholar 

  48. 48.

    Sebastian, A., Pannone, A., Radhakrishnan, S. S. & Das, S. Gaussian synapses for probabilistic neural networks. Nat. Commun. 10, 4199 (2019).

    Google Scholar 

  49. 49.

    Tran, M. D. et al. Two-terminal multibit optical memory via van der Waals heterostructure. Adv. Mater. 31, 1807075 (2019).

    Google Scholar 

  50. 50.

    Xuan, Y. et al. Multi-scale modeling of gas-phase reactions in metal-organic chemical vapor deposition growth of WSe2. J. Cryst. Growth 527, 125247 (2019).

    Google Scholar 

  51. 51.

    Kang, K. et al. High-mobility three-atom-thick semiconducting films with wafer-scale homogeneity. Nature 520, 656–660 (2015).

    Google Scholar 

  52. 52.

    Choudhury, T. H. et al. Chalcogen precursor effect on cold-wall gas-source chemical vapor deposition growth of WS2. Cryst. Growth Des. 18, 4357–4364 (2018).

    Google Scholar 

  53. 53.

    Li, H. et al. From bulk to monolayer MoS2: evolution of Raman scattering. Adv. Funct. Mater. 22, 1385–1390 (2012).

    Google Scholar 

  54. 54.

    Rumble, J. R. CRC Handbook of Chemistry and Physics 98th edn (CRC Press, 2018).

  55. 55.

    Crider, C. A., Poate, J. M., Rowe, J. E. & Sheng, T. T. Platinum silicide formation under ultrahigh-vacuum and controlled impurity ambients. J. Appl. Phys. 52, 2860–2868 (1981).

    Google Scholar 

  56. 56.

    Das, S., Chen, H. Y., Penumatcha, A. V. & Appenzeller, J. High performance multilayer MoS2 transistors with scandium contacts. Nano Lett. 13, 100–105 (2013).

    Google Scholar 

  57. 57.

    Schulman, D. S., Arnold, A. J. & Das, S. Contact engineering for 2D materials and devices. Chem. Soc. Rev. 47, 3037–3058 (2018).

Download references

Acknowledgements

This work was partially supported through grant number FA9550-17-1-0018 from the Air Force Office of Scientific Research (AFOSR) through the Young Investigator Program and Army Research Office (ARO) through contract no. W911NF1920338. We also acknowledge the support from the National Science Foundation (NSF) through the Pennsylvania State University’s 2D Crystal Consortium–Materials Innovation Platform (2DCC-MIP) under NSF cooperative agreement DMR-1539916.

Author information

Affiliations

Authors

Contributions

S.D. conceived the idea, designed the experiments and wrote the manuscript. D.J., A.O. and S.D. performed the experiments, analysed the data, discussed the results and agreed on their implications. A.S. fabricated the devices. T.H.C. synthesized the MoS2 monolayers. All the authors contributed to the preparation of the manuscript.

Corresponding author

Correspondence to Saptarshi Das.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Table 1, Figs. 1–18 and discussions 1–15.

Supplementary Video 1

Scenario depicting a car approaching the collision detector on a direct collision course.

Supplementary Video 2

Visual excitation experienced by the collision detector from an approaching car.

Supplementary Video 3

Scenario depicting a car approaching a wall and the corresponding visual excitation experienced by the collision detector from the reflected light.

Supplementary Video 4

Visual excitation due to increasing light intensity from a static source.

Supplementary Video 5

Simulated escape response from a multipixel collision detector array.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Jayachandran, D., Oberoi, A., Sebastian, A. et al. A low-power biomimetic collision detector based on an in-memory molybdenum disulfide photodetector. Nat Electron 3, 646–655 (2020). https://doi.org/10.1038/s41928-020-00466-9

Download citation

Search

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing