Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

NeuroMechFly, a neuromechanical model of adult Drosophila melanogaster

Nature Methods volume 19, pages 620–627 (2022)Cite this article

Subjects

Abstract

Animal behavior emerges from an interaction between neural network dynamics, musculoskeletal properties and the physical environment. Accessing and understanding the interplay between these elements requires the development of integrative and morphologically realistic neuromechanical simulations. Here we present NeuroMechFly, a data-driven model of the widely studied organism, Drosophila melanogaster. NeuroMechFly combines four independent computational modules: a physics-based simulation environment, a biomechanical exoskeleton, muscle models and neural network controllers. To enable use cases, we first define the minimum degrees of freedom of the leg from real three-dimensional kinematic measurements during walking and grooming. Then, we show how, by replaying these behaviors in the simulator, one can predict otherwise unmeasured torques and contact forces. Finally, we leverage NeuroMechFly’s full neuromechanical capacity to discover neural networks and muscle parameters that drive locomotor gaits optimized for speed and stability. Thus, NeuroMechFly can increase our understanding of how behaviors emerge from interactions between complex neuromechanical systems and their physical surroundings.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Data-driven development, applications, and modularity of NeuroMechFly.
Fig. 2: Constructing a data-driven biomechanical model of adult Drosophila.
Fig. 3: Adding a CTr roll DoF to base DoFs enables the most accurate kinematic replay of real walking and grooming.
Fig. 4: Kinematic replay of forward walking enables the estimation of ground contacts and reaction forces.
Fig. 5: Kinematic replay enables the estimation of self-collisions and reaction forces during foreleg–antennal grooming.
Fig. 6: Evolutionary optimization identifies oscillator network and muscle parameters for fast and stable locomotion.

Similar content being viewed by others

Data availability

Kinematic data and 3D pose estimations for walking behavior are available at https://doi.org/10.7910/DVN/Y3TAEC, and kinematic data and 3D pose estimations for grooming behavior are available at https://dataverse.harvard.edu/dataverse/DeepFly3D

Code availability

The code required to reproduce the experiments described here can be obtained as a Code Ocean capsule (https://codeocean.com/capsule/2418941/tree/v1, ref. 87). Code and documentation for developers are available in GitHub under an Apache 2.0 license (https://github.com/NeLy-EPFL/NeuroMechFly and https://nely-epfl.github.io/NeuroMechFly).

References

  1. Chiel, H. J. & Beer, R. D. The brain has a body: adaptive behavior emerges from interactions of nervous system, body and environment. Trends Neurosci. 20, 553–557 (1997).

    Article  CAS  PubMed  Google Scholar 

  2. Webb, B. A framework for models of biological behaviour. Int. J. Neural Syst. 9, 375–381 (1999).

    Article  CAS  PubMed  Google Scholar 

  3. Pearson, K., Ekeberg, Ö. & Büschges, A. Assessing sensory function in locomotor systems using neuro-mechanical simulations. Trends Neurosci. 29, 625–631 (2006).

    Article  CAS  PubMed  Google Scholar 

  4. Prilutsky, B. I. & Edwards, D. H. (eds) Neuromechanical Modeling of Posture and Locomotion (Springer, 2015).

  5. Seth, A. et al. OpenSim: simulating musculoskeletal dynamics and neuromuscular control to study human and animal movement. PLoS Comput. Biol. 14, e1006223 (2018).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  6. Einevoll, G. T. et al. The scientific case for brain simulations. Neuron 102, 735–744 (2019).

    Article  CAS  PubMed  Google Scholar 

  7. Sigvardt, K. A. & Miller, W. L. Analysis and modeling of the locomotor central pattern generator as a network of coupled oscillators. Ann. NY Acad. Sci. 860, 250–265 (1998).

    Article  CAS  PubMed  Google Scholar 

  8. Lansner, A., Hellgren Kotaleski, J. & Grillner, S. Modeling of the spinal neuronal circuitry underlying locomotion in a lower vertebrate. Ann. NY Acad. Sci. 860, 239–249 (1998).

    Article  CAS  PubMed  Google Scholar 

  9. Ijspeert, A. J. A connectionist central pattern generator for the aquatic and terrestrial gaits of a simulated salamander. Biol. Cybern. 84, 331–348 (2001).

    Article  CAS  PubMed  Google Scholar 

  10. Rybak, I. A., Dougherty, K. J. & Shevtsova, N. A. Organization of the mammalian locomotor CPG: review of computational model and circuit architectures based on genetically identified spinal interneurons (1,2,3). eNeuro 2, ENEURO.0069-15.2015 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  11. Ekeberg, Ö., Blümel, M. & Büschges, A. Dynamic simulation of insect walking. Arthropod Struct. Dev. 33, 287–300 (2004).

    Article  PubMed  Google Scholar 

  12. Toth, T. I., Schmidt, J., Büschges, A. & Daun-Gruhn, S. A neuro-mechanical model of a single leg joint highlighting the basic physiological role of fast and slow muscle fibres of an insect muscle system. PLoS ONE 8, e78247 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Toth, T. I., Grabowska, M., Schmidt, J., Büschges, A. & Daun-Gruhn, S. A neuro-mechanical model explaining the physiological role of fast and slow muscle fibres at stop and start of stepping of an insect leg. PLoS ONE 8, e78246 (2013).

    Article  PubMed  PubMed Central  Google Scholar 

  14. Schilling, M., Hoinville, T., Schmitz, J. & Cruse, H. Walknet, a bio-inspired controller for hexapod walking. Biol. Cybern. 107, 397–419 (2013).

    Article  PubMed  PubMed Central  Google Scholar 

  15. Szczecinski, N. S., Brown, A. E., Bender, J. A., Quinn, R. D. & Ritzmann, R. E. A neuromechanical simulation of insect walking and transition to turning of the cockroach Blaberus discoidalis. Biol. Cybern. 108, 1–21 (2014).

    Article  PubMed  Google Scholar 

  16. Proctor, J., Kukillaya, R. & Holmes, P. A phase-reduced neuro-mechanical model for insect locomotion: feed-forward stability and proprioceptive feedback. Philos. Trans. A Math. Phys. Eng. Sci. 368, 5087–5104 (2010).

    CAS  PubMed  Google Scholar 

  17. Szczecinski, N. S., Martin, J. P., Bertsch, D. J., Ritzmann, R. E. & Quinn, R. D. Neuromechanical model of praying mantis explores the role of descending commands in pre-strike pivots. Bioinspir. Biomim. 10, 065005 (2015).

    Article  PubMed  CAS  Google Scholar 

  18. Guo, S., Lin, J., Wöhrl, T. & Liao, M. A neuro-musculo-skeletal model for insects with data-driven optimization. Sci. Rep. 8, 2129 (2018).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  19. Szigeti, B. et al. Openworm: an open-science approach to modeling Caenorhabditis elegans. Front. Comput. Neurosci. 8, 137 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  20. Izquierdo, E. J. & Beer, R. D. From head to tail: a neuromechanical model of forward locomotion in Caenorhabditis elegans. Philos. Trans. R. Soc. Lond. B Biol. Sci. 373, 1758 (2018).

    Article  Google Scholar 

  21. Loveless, J., Lagogiannis, K. & Webb, B. Modelling the mechanics of exploration in larval Drosophila. PLoS Comput. Biol. 15, e1006635 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Merel, J. et al. Deep neuroethology of a virtual rodent. In Proceedings of the International Conference on Learning Representations https://openreview.net/forum?id=SyxrxR4KPS (2020).

  23. Seeds, A. M. et al. A suppression hierarchy among competing motor programs drives sequential grooming in Drosophila. eLife 3, e02951 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  24. Pavlou, H. J. & Goodwin, S. F. Courtship behavior in Drosophila melanogaster: towards a ‘courtship connectome’. Curr. Opin. Neurobiol. 23, 76–83 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Fry, S. N., Sayaman, R. & Dickinson, M. H. The aerodynamics of free-flight maneuvers in Drosophila. Science 300, 495–498 (2003).

    Article  CAS  PubMed  Google Scholar 

  26. Mendes, C. S. et al. Quantification of gait parameters in freely walking wild type and sensory deprived Drosophila melanogaster. eLife 2, e00231 (2013).

    Article  PubMed  PubMed Central  Google Scholar 

  27. Wosnitza, A. et al. Inter-leg coordination in the control of walking speed in Drosophila. J. Exp. Biol. 216, 480–491 (2013).

    PubMed  Google Scholar 

  28. Pick, S. & Strauss, R. Goal-driven behavioral adaptations in gap-climbing Drosophila. Curr. Biol. 15, 1473–1478 (2005).

    Article  CAS  PubMed  Google Scholar 

  29. Pereira, T. D. et al. Fast animal pose estimation using deep neural networks. Nat. Methods 16, 117–125 (2019).

    Article  CAS  PubMed  Google Scholar 

  30. Mathis, A. et al. DeepLabCut: markerless pose estimation of user-defined body parts with deep learning. Nat. Neurosci. 21, 1281–1289 (2018).

    Article  CAS  PubMed  Google Scholar 

  31. Günel, S. et al. DeepFly3D, a deep learning-based approach for 3D limb and appendage tracking in tethered, adult Drosophila. eLife 8, e48571 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  32. Gosztolai, A. et al. LiftPose3D, a deep learning-based approach for transforming two-dimensional to three-dimensional poses in laboratory animals. Nat. Methods 18, 975–981 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  33. Jenett, A. et al. A GAL4-driver line resource for Drosophila neurobiology. Cell Rep. 2, 991–1001 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. Seelig, J. D. et al. Two-photon calcium imaging from head-fixed Drosophila during optomotor walking behavior. Nat. Methods 7, 535–540 (2010).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Maimon, G., Straw, A. D. & Dickinson, M. H. Active flight increases the gain of visual motion processing in Drosophila. Nat. Neurosci. 13, 393–399 (2010).

    Article  CAS  PubMed  Google Scholar 

  36. Chen, C. et al. Imaging neural activity in the ventral nerve cord of behaving adult Drosophila. Nat. Commun. 9, 4390 (2018).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  37. Hermans, L. et al. Long-term imaging of the ventral nerve cord in behaving adult Drosophila. Preprint at https://doi.org/10.1101/2021.10.15.463778v1 (2021).

  38. Phelps, J. S. et al. Reconstruction of motor control circuits in adult Drosophila using automated transmission electron microscopy. Cell 184, 759–774 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  39. Scheffer, L. K. et al. A connectome and analysis of the adult Drosophila central brain. eLife 9, e57443 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  40. Isakov, A. et al. Recovery of locomotion after injury in Drosophila melanogaster depends on proprioception. J. Exp. Biol. 219, 1760–1771 (2016).

    PubMed  Google Scholar 

  41. Ramdya, P. et al. Climbing favours the tripod gait over alternative faster insect gaits. Nat. Commun. 8, 14494 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  42. Coumans, E. Bullet physics simulation. In ACM SIGGRAPH 2015 Courses https://doi.org/10.1145/2776880.2792704 (2015).

  43. Soler, C., Daczewska, M., Da Ponte, J. P., Dastugue, B. & Jagla, K. Coordinated development of muscles and tendons of the Drosophila leg. Development 131, 6041–6051 (2004).

    Article  CAS  PubMed  Google Scholar 

  44. Sink, H. Muscle Development in Drosophila (Springer, 2006).

  45. Cruse, H., Dürr, V. & Schmitz, J. Insect walking is based on a decentralized architecture revealing a simple and robust controller. Philos. Trans. A Math. Phys. Eng. Sci. 365, 221–250 (2007).

    PubMed  Google Scholar 

  46. Loper, M., Mahmood, N., Romero, J., Pons-Moll, G. & Black, M. J. SMPL: a skinned multi-person linear model. ACM Trans. Graphics 34, 248 (2015).

    Article  Google Scholar 

  47. Zuffi, S., Kanazawa, A., Jacobs, D. W. & Black, M. J. 3D menagerie: modeling the 3d shape and pose of animals. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 6365–6373 (2017).

  48. Li, S. et al. Deformation-aware unpaired image translation for pose estimation on laboratory animals. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 13158–13168 (2020).

  49. Mu, J., Qiu, W., Hager, G. D. & Yuille, A. L. Learning from synthetic animals. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 12386–12395 (2020).

  50. Bolaños, L. A. et al. A three-dimensional virtual mouse generates synthetic training data for behavioral analysis. Nat. Methods 18, 378–381 (2021).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  51. Watson, J. T., Ritzmann, R. E., Zill, S. N. & Pollack, A. J. Control of obstacle climbing in the cockroach, Blaberus discoidalis. I. Kinematics. J. Comp. Physiol. A Neuroethol. Sens. Neural Behav. Physiol. 188, 39–53 (2002).

    Article  PubMed  Google Scholar 

  52. Frantsevich, L. & Wang, W. Gimbals in the insect leg. Arthropod Struct. Dev. 38, 16–30 (2009).

    Article  PubMed  Google Scholar 

  53. Bender, J. A., Simpson, E. M. & Ritzmann, R. E. Computer-assisted 3D kinematic analysis of all leg joints in walking insects. PLoS ONE 5, e13617 (2010).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  54. Zill, S. N. et al. Effects of force detecting sense organs on muscle synergies are correlated with their response properties. Arthropod Struct. Dev. 46, 564–578 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  55. Cofer, D., Cymbalyuk, G., Heitler, W. J. & Edwards, D. H. Neuromechanical simulation of the locust jump. J. Exp. Biol. 213, 1060–1068 (2010).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  56. Moore, R. J. D. et al. Fictrac: a visual method for tracking spherical motion and generating fictive animal paths. J. Neurosci. Methods 225, 106–119 (2014).

    Article  PubMed  Google Scholar 

  57. Azevedo, A. W. et al. A size principle for recruitment of Drosophila leg motor neurons. eLife 9, e56754 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  58. Fuchs, E., Holmes, P., Kiemel, T. & Ayali, A. Intersegmental coordination of cockroach locomotion: adaptive control of centrally coupled pattern generator circuits. Front. Neural Circuits 4, 125 (2011).

    PubMed  PubMed Central  Google Scholar 

  59. Mantziaris, C. et al. Intra-and intersegmental influences among central pattern generating networks in the walking system of the stick insect. J. Neurophysiol. 118, 2296–2310 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  60. Schilling, M. & Cruse, H. Decentralized control of insect walking: a simple neural network explains a wide range of behavioral and neurophysiological results. PLoS Comput. Biol. 16, e1007804 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  61. Ijspeert, A. J., Crespi, A., Ryczko, D. & Cabelguen, J. From swimming to walking with a salamander robot driven by a spinal cord model. Science 315, 1416–1420 (2007).

    Article  CAS  PubMed  Google Scholar 

  62. Ekeberg, Ö. A combined neuronal and mechanical model of fish swimming. Biol. Cybern. 69, 363–374 (1993).

    Article  Google Scholar 

  63. Daun-Gruhn, S. A mathematical modeling study of inter-segmental coordination during stick insect walking. J. Comput. Neurosci. 30, 255–278 (2011).

    Article  PubMed  Google Scholar 

  64. DeAngelis, B. D., Zavatone-Veth, J. A. & Clark, D. A. The manifold structure of limb coordination in walking Drosophila. eLife 8, e46409 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  65. Oliveira, M. et al. Multi-objective parameter CPG optimization for gait generation of a quadruped robot considering behavioral diversity. In 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems 2286–2291 (2011).

  66. Deb, K., Pratap, A., Agarwal, S. & Meyarivan, T. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evolutionary Computation 6, 182–197 (2002).

    Article  Google Scholar 

  67. Strauss, R. & Heisenberg, M. Coordination of legs during straight walking and turning in Drosophila melanogaster. J. Comp. Physiol. A 167, 403–412 (1990).

    Article  CAS  PubMed  Google Scholar 

  68. Szczecinski, N. S., Bockemühl, T., Chockley, A. S. & Büschges, A. Static stability predicts the continuum of interleg coordination patterns in Drosophila. J. Exp. Biol. 221, jeb189142 (2018).

    Article  PubMed  Google Scholar 

  69. Vincent, J. F. & Wegst, U. G. Design and mechanical properties of insect cuticle. Arthropod Struct. Dev. 33, 187–199 (2004).

    Article  PubMed  Google Scholar 

  70. Flynn, P. C. & Kaufman, W. R. Mechanical properties of the cuticle of the tick Amblyomma hebraeum (Acari: Ixodidae). J. Exp. Biol. 218, 2806–2814 (2015).

    PubMed  Google Scholar 

  71. Kimura, K., Minami, R., Yamahama, Y., Hariyama, T. & Hosoda, N. Framework with cytoskeletal actin filaments forming insect footpad hairs inspires biomimetic adhesive device design. Commun. Biol. 3, 272 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  72. Takahashi, H. et al. Maximum force capacity of legs of a fruit fly during landing motion. In 19th International Conference on Solid-State Sensors, Actuators and Microsystems 1061–1064 (2017).

  73. Elliott, C. J. & Sparrow, J. C. In vivo measurement of muscle output in intact Drosophila. Methods 56, 78–86 (2012).

    Article  CAS  PubMed  Google Scholar 

  74. Mamiya, A., Gurung, P. & Tuthill, J. C. Neural coding of leg proprioception in Drosophila. Neuron 100, 636–650 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  75. Kuan, A. T. et al. Dense neuronal reconstruction through X-ray holographic nano-tomography. Nat. Neurosci. 23, 1637–1643 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  76. Hayat, M.A. Principles and Techniques of Electron Microscopy: Biological Applications (Van Nostrand Reinhold, 1976).

  77. Schindelin, J. et al. Fiji: an open-source platform for biological-image analysis. Nat. Methods 9, 676–682 (2012).

    Article  CAS  PubMed  Google Scholar 

  78. Lewiner, T., Lopes, H., Vieira, A. W. & Tavares, G. Efficient implementation of marching cubes’ cases with topological guarantees. J. Graphics Tools 8, 1–15 (2003).

    Article  Google Scholar 

  79. van der Walt, S. et al. Scikit-image: image processing in Python. PeerJ 2, e453 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  80. Blender Foundation. Blender: a 3D Modelling and Rendering Package http://www.blender.org (2012).

  81. Ferris, G. External morphology of the adult. In Biology of Drosophila (ed. Demerec M) 368–419 (Wiley & Sons, 1950).

  82. Dickson, W. B., Straw, A. D. & Dickinson, M. H. Integrative model of Drosophila flight. AIAA J. 46, 2150–2164 (2008).

    Article  Google Scholar 

  83. Geurten, B. R. H., Jähde, P., Corthals, K. & Göpfert, M. C. Saccadic body turns in walking Drosophila. Front. Behav. Neurosci. 8, 365 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  84. Mantziaris, C., Bockemühl, T. & Büschges, A. Central pattern generating networks in insect locomotion. Dev. Neurobiol. 80, 16–30 (2020).

    Article  PubMed  Google Scholar 

  85. Cohen, A. H., Holmes, P. J. & Rand, R. H. The nature of the coupling between segmental oscillators of the lamprey spinal generator for locomotion: a mathematical model. J. Math. Biol. 13, 345–369 (1982).

    Article  CAS  PubMed  Google Scholar 

  86. Benitez-Hidalgo, A., Nebro, A., Garcia-Nieto, J., Oregi, I. & Del Ser, J. jMetalPy: a Python framework for multi-objective optimization with metaheuristics. Swarm Evol. Comput. 51, 100598 (2019).

    Article  Google Scholar 

  87. Lobato-Rios, V. et al. NeuroMechFly, a neuromechanical model of adult Drosophila melanogaster: Code Ocean https://codeocean.com/capsule/2418941/tree/v1 (2022).

Download references

Acknowledgements

We thank S. Clerc Rosset and G. Knott (Biological Electron Microscopy Facility, EPFL, Lausanne, Switzerland) for preparing Drosophila melanogaster samples for X-ray microtomography. We thank H. Sigurthorsdottir for early work on limb degrees of freedom. Furthermore, P.R. acknowledges support from an SNSF Project Grant (175667) and an SNSF Eccellenza Grant (181239). V.L.-R. acknowledges support from the Mexican National Council for Science and Technology, CONACYT, under the grant number 709993. S.T.R. acknowledges support from the European Union’s Horizon 2020 research and innovation program under grant agreement nos. 720270 (SGA1) and 785907 (SGA2). P.G.O. acknowledges support from the Swiss Government Excellence Scholarship for Doctoral Studies. J.A. acknowledges support from the Human Frontier Science Program (HFSP-RGP0027/2017).

Author information

Author notes

  1. These authors contributed equally: Shravan Tata Ramalingasetty, Pembe Gizem Özdil.

Authors and Affiliations

  1. Neuroengineering Laboratory, Brain Mind Institute and Institute of Bioengineering, EPFL, Lausanne, Switzerland

    Victor Lobato-Rios, Pembe Gizem Özdil & Pavan Ramdya

  2. Biorobotics Laboratory, EPFL, Lausanne, Switzerland

    Shravan Tata Ramalingasetty, Pembe Gizem Özdil, Jonathan Arreguit & Auke Jan Ijspeert

Authors
  1. Victor Lobato-Rios
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shravan Tata Ramalingasetty
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pembe Gizem Özdil
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jonathan Arreguit
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Auke Jan Ijspeert
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Pavan Ramdya
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

V.L-R., conceptualization, methodology, software, validation, formal analysis, investigation, data curation, validation, writing (original draft preparation, and review and editing), visualization. S.T.R., conceptualization, methodology, software, validation, writing (review and editing), visualization. P.G.O., conceptualization, methodology, software, validation, formal analysis, investigation, data curation, writing (review and editing), visualization. J.A., conceptualization, methodology, software, validation, writing (review and editing). A.J.I., conceptualization, methodology, resources, writing (review and editing), supervision, project administration, funding acquisition. P.R., conceptualization, methodology, resources, writing (original draft preparation, and review and editing), supervision, project administration, funding acquisition.

Corresponding author

Correspondence to Pavan Ramdya.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Methods thanks Benjamin de Bivort and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Nina Vogt was the primary editor on this article and managed its editorial process and peer review in collaboration with the rest of the editorial team.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Leg segment lengths for real female Drosophila melanogaster and NeuroMechFly.

(A) Legs were dissected, straightened, and fixed onto a glass slide for measurements. Scale bar is 0.5mm. (B) The lengths of leg segments from 1-3 dpe animals (pink) and NeuroMechFly (red) are shown. Violin plots indicate median, upper, and lower quartiles.

Extended Data Fig. 2 The ’zero pose’ of NeuroMechFly and its leg joint degrees-of-freedom.

Zero pose of NeuroMechFly from (A) front and (B) side views. Each leg is composed of 11 hinge joints. Joints with more than one DoF were modeled as a union of multiple hinge joints. The left foreleg observed from the (C) side and (D) front views. Rotational axes of joints are shown in light green. The global coordinate system’s x, y, and z axes are red, green, and blue, respectively.

Extended Data Fig. 3 The position error for every joint in the distal leg during walking and grooming as a function of kinematic chain configuration.

Body-length normalized mean absolute errors (MAE) comparing measured 3D poses and angle-derived joint positions during walking. Errors are compared among different DoF configurations for (A) Coxa–Trochanter joints, (B) Femur–Tibia joints, (C) Tibia–Tarsus joints, and (D) Claw positions during walking and (E-H) grooming. For each condition, n = 2400 samples were computed across all six legs from 4s of 100 Hz video data. Data for each leg are color-coded. ’R’ and ’L’ indicate right and left legs, respectively. ’F’, ’M’, and ’H’ indicate front, middle, and hind legs, respectively. Violin plots indicate median, upper, and lower quartiles (dashed lines). Results from adding a coxa–trochanter roll DoF to based DoFs are highlighted in light gray.

Extended Data Fig. 4 Sensitivity to proportional and derivative gains of kinematic replay during walking.

MSE of (A) joint angles and (B) joint velocities as a function of derivative (Kd) and positional gains (Kp). Selected Kp and Kd values are indicated (blue). Kp and Kd pairs rendering the simulation nonfunctional during kinematic replay are also indicated (white). (C) Estimated ThC pitch torques and (D) contact force measurements of the right legs during forward walking as a function of proportional gain (Kp) when the derivative gain (Kd) is fixed at 0.9. Measurements for the contralateral legs were nearly symmetrically identical and are not shown. Results from the selected Kp and Kd values are indicated (red). (E) Estimated ThC pitch torques and (F) Contact force measurements of the right legs during forward walking as a function of derivative gain (Kd) while holding proportional gain (Kp) fixed at 0.4. Measurements for the contralateral legs were nearly symmetrically identical and are not shown. Results from the selected Kp and Kd values are indicated (red).

Extended Data Fig. 5 Sensitivity of simulated spherical treadmill rotation prediction accuracy during tethered walking to ERP and CFM constraint parameters.

Spherical treadmill rotational velocities resulting from Kinematic Replay of walking depend on simulation constraint parameters. Shown are Spearman correlation coefficients computed between measured and estimated treadmill rotational velocities for (A) forward, (B) lateral, and (C) yaw axes when varying the simulation’s error reduction parameter (ERP), and the constraint force mixing (CFM). (D) The best combination of ERP and CFM—0.1 and 3, respectively (black outline)—was selected through a normalized weighted sum (NWS) of the correlation coefficients for each axis.

Extended Data Fig. 6 Comparing real to simulated spherical treadmill rotational velocities during tethered walking.

Spherical treadmill rotations depend on a tethered fly’s (A) inclination (Φ, green), (B) lateral, and (C) longitudinal positions with respect to the ball (green outlines). These positions (orange dots) were automatically detected and recreated in the simulation. Rotational velocities of the spherical treadmill generated by three real flies (blue) were compared with those generated by NeuroMechFly (orange) for (D) forward, (E) lateral, and (F) yaw axes. Spearman correlation coefficients (ρ) comparing blue and orange traces are indicated.

Extended Data Fig. 7 Comparing real and simulation predictions for gait diagrams during tethered walking.

Gait diagrams showing manually annotated stance phases for three real flies (A-C, gold) as well as those obtained from estimated ground reaction forces in NeuroMechFly (blue). Percentage of overlap in real and simulated stance phases (green) is quantified. ’R’ and ’L’ indicate right and left legs, respectively. ’F’, ’M’, and ’H’ indicate front, middle, and hind legs, respectively.

Extended Data Fig. 8 Comparison of walking paths and velocities for real tethered walking versus kinematic replay in a tethered or untethered model.

Leg kinematics from a tethered walking experiment (blue) were used for kinematic replay in NeuroMechFly either tethered on a simulated spherical treadmill (orange) or freely walking on flat ground (green). Shown are resulting (A) integrated walking paths, as well as associated (B) forward, (C) lateral, and (D) yaw velocities.

Extended Data Fig. 9 The impact of the morphological realism on estimates of leg–leg and leg-antenna contact during grooming.

Collision diagrams from kinematic replay of foreleg–antennal grooming when using either (A) NeuroMechFly’s morphologically detailed legs and antennae, or after replacing its (B) forelegs, or (C) forelegs and antennae with simple cylinders, as in a conventional stick skeletal model.

Extended Data Fig. 10 Joints controlled and comparison over generations when optimizing for fast and statically stable tethered walking.

Joint angles for the (A) left and (B) right legs measured from a real fly during forward walking. Only the three DoFs with the highest amplitudes (solid lines) were controlled during optimization. The remaining four DoFs per leg (dashed lines) were fixed during optimization because they did not exhibit pronounced angular changes. (C) Pareto front approximations for six optimization generations. The more negative values indicate the more optimal objective functions. Four individual solutions dominated by the pareto optimal solutions were selected for more in-depth analysis (10th (purple), 20th (blue), 30th (green), and 50th (dark red); all are outlined in black). (D) Gait diagrams from selected solutions. Stance (black) and swing (white) phases were calculated by reading-out tarsal ground contacts for each leg. Indicated are the velocities of each solution as calculated by averaging the spherical treadmill forward velocity. (E) Progression of weighted objective values (shown without sign inversion) and penalties over the course of 60 generations. Objectives (distance and stability coefficients) increase across generations, while penalties decrease or converge to, or near, zero. The objective distance (mm) is the distance traveled in 2 s. The penalty duty factor is the number of legs violating the duty factor constraint. The remaining penalties are shown in Arbitrary Units.

Supplementary information

Supplementary Information

Supplementary Tables 1–6

Reporting Summary

Supplementary Video 1

Constructing a data-driven biomechanical model of adult Drosophila.

Supplementary Video 2

Visualization of possible additional leg degrees of freedom.

Supplementary Video 3

The effect of additional degrees of freedom on the accuracy of replaying forward walking.

Supplementary Video 4

The effect of additional degrees of freedom on the accuracy of replaying foreleg–antennal grooming.

Supplementary Video 5

Kinematic replay of Drosophila forward walking using NeuroMechFly..

Supplementary Video 6

Kinematic replay of Drosophila foreleg–antennal grooming using NeuroMechFly.

Supplementary Video 7

The influence of leg and antenna morphological detail on collision predictions.

Supplementary Video 8

Kinematic replay of tethered Drosophila forward walking using NeuroMechFly on flat terrain without body support.

Supplementary Video 9

Replaying real tethered walking kinematics on flat terrain and applying external perturbations.

Supplementary Video 10

Forward walking across optimization generations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lobato-Rios, V., Ramalingasetty, S.T., Özdil, P.G. et al. NeuroMechFly, a neuromechanical model of adult Drosophila melanogaster. Nat Methods 19, 620–627 (2022). https://doi.org/10.1038/s41592-022-01466-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41592-022-01466-7

This article is cited by

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics