NeuroMechFly: an integrative simulation testbed for studying Drosophila behavioral control

Neuromechanical simulations enable the study of how interactions between organisms and their physical surroundings give rise to behavior. NeuroMechFly is an open-source neuromechanical model of adult Drosophila, with data-driven morphological biorealism that enables a synergistic cross-talk between computational and experimental neuroscience.

The problem

Disentangling how complex neural dynamics, musculoskeletal biomechanics and interactions with the physical environment contribute to animal behavior is a long-standing goal of neuroscience research. With recent computational advances, it has become possible to use morphologically realistic neuromechanical simulations to pursue this goal. New experimental and computational approaches for measuring neural connectivity, neural activity and body-part kinematics have also enabled a synergy between experimental and computational neuroscience, with experimental data informing computational models and modelling motivating further experiments.

Drosophila melanogaster, a widely studied fly, is an ideal model organism for creating morphologically realistic neuromechanical simulations and furthering this synergy between experimental and computational neuroscience. Drosophila have neurons that are few in number and identifiable (making it possible to revisit the same neurons repeatedly across animals), they can be genetically manipulated (making it possible to perturb neural dynamics and musculoskeletal biomechanics), and they generate a large repertoire of complex behaviors (providing a rich reverse-engineering challenge). When we began this project, toy models of Drosophila were available¹, but a morphologically realistic neuromechanical simulation of this fly had not been developed.

The solution

We set out to develop what we now call ‘NeuroMechFly’, a data-driven, morphologically realistic neuromechanical computational model of adult Drosophila for studying fly behavior.

To achieve this morphological realism, we first embedded an adult female fly (Fig. 1a) and used X-ray microtomography (Fig. 1b) to digitize a biorealistic 3D Drosophila exoskeleton (Fig. 1c). Then we dissected this Drosophila exoskeleton into 65 articulating segments, including the antennae, proboscis, head, wings, halteres, abdominal segments and leg segments (Fig. 1d). This process resulted in a morphologically realistic model of Drosophila that we could use to perform bio- and neuromechanical simulation studies.

Fig. 1: Construction and applications of NeuroMechFly.

a, Drosophila encased in resin for X-ray microtomography. b, A cross-section from the X-ray scan. c, The exoskeleton and wings are digitally extracted. d, Articulated body parts are separated from one another and then used to create the final, textured NeuroMechFly model. e, Pose estimation of a tethered Drosophila performing real foreleg rubbing behavior. f, In silico kinematic replay of foreleg rubbing poses using NeuroMechFly. Leg segments in collision are indicated in green. g, Estimated collision force vectors during kinematic replay of foreleg rubbing are overlaid on real Drosophila data. Arrow lengths represent force magnitudes. © 2022, Lobato-Rios, V. et al.

However, when we first used this Drosophila model to reconstruct 3D poses measured during experiments with real, behaving flies² (Fig. 1e), we were surprised to find that we needed an additional, unaccounted for degree of freedom in the limbs of our model. We addressed this by incorporating a new joint, enabling us to accurately replay walking and grooming limb kinematics from real flies in our model (Fig. 1f). Inferred but unmeasured forces were validated by comparing walking gaits and spherical treadmill rotations between real Drosophila data and our simulations. Furthermore, our model could also infer unmeasured and experimentally inaccessible physical quantities like joint torques, collisions and reaction forces (Fig. 1g).

Having validated NeuroMechFly, we next illustrated how it could be applied in neuromechanical simulation studies by exploring how muscles and neural controllers generate complex behaviors. Specifically, we optimized the parameters of a central pattern generator–based neural network³ to drive fast and stable tethered locomotion. Remarkably, some emergent gaits resembled the tripod gait used by real insects¹.

In summary, we created and validated NeuroMechFly, a morphologically realistic bio- and neuromechanical computational model of adult Drosophila. We then showed how its application can disentangle complex neural dynamics, musculoskeletal biomechanics and interactions with the physical environment contributing to fly behavior.

Future directions

Our results provide a first glimpse of how a morphologically accurate neuromechanical simulation can complement and inform real neurobiological experiments to accelerate advances in neuroscience.

Like any new tool, NeuroMechFly has limitations. For example, at the moment, our simulation uses abstract neuromuscular controllers that are difficult to map onto the real Drosophila nervous system. Future work may improve NeuroMechFly by increasing the amount of biomechanical compliance as well as the biorealism of the neural network controllers and the muscle models it uses. NeuroMechFly is open source, so refinement by the scientific community is expected.

We are excited about the prospect of using large-scale connectomics⁴ and functional neural imaging⁵ data along with precise behavioral kinematic data² to further refine and constrain NeuroMechFly. Ultimately, we aim to routinely perform whole-fly simulations to uncover the relative contributions of neural, muscular and biomechanical properties to real, complex animal behaviors.

Victor L. Ríos and Pavan Ramdya

Brain Mind Institute, EPFL, Lausanne, Switzerland.

Expert opinion

“This model has the potential to further neuromechanical studies of behavior in flies, and may be particularly valuable as connectome-inspired models come online in the coming years. I also think the synergistic links the authors draw between this work and machine learning and robotics are plausible. The work seems valuable to the field, timely, and well conducted.” Benjamin de Bivort, Harvard University, Cambridge, MA, USA.

Behind the paper

In late 2018, we were focused on extending the capabilities of our previous, abstract Drosophila model¹. One day, we received an email from the university advertising a new micro-CT scan system on campus. This advertisement included an image of an X-ray-scanned ant. This made us realize that it might now be possible to create a biorealistic exoskeleton for our fly model. We dropped what we were doing and embarked on our first attempts to scan and digitize a real adult female fly. After a few hiccups, and with the help of experts in electron microscopy sample preparation, we were able to overcome early problems in fixation and embedding. The next scans were simply amazing. This inspired us, along with collaborators, to push the development of this more advanced model, despite intervening challenges including the COVID-19 lockdown. We had to overcome many other difficulties, including appropriately scaling down the model to the size of a real fly. Finally, we converged on the NeuroMechFly model that you can now use today. V.L.R. & P.R.

From the editor

“This work stood out to me because it is exciting how well the authors can model an adult fruit fly and what they can learn from this modeling approach. In addition to the types of analyses shown in the paper, the model could also be useful for training deep-learning-based algorithms for behavioral studies.” Nina Vogt, Senior Editor, Nature Methods.