Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Machine learning reveals the control mechanics of an insect wing hinge

Nature volume 628, pages 795–803 (2024)Cite this article

Subjects

Abstract

Insects constitute the most species-rich radiation of metazoa, a success that is due to the evolution of active flight. Unlike pterosaurs, birds and bats, the wings of insects did not evolve from legs1, but are novel structures that are attached to the body via a biomechanically complex hinge that transforms tiny, high-frequency oscillations of specialized power muscles into the sweeping back-and-forth motion of the wings2. The hinge consists of a system of tiny, hardened structures called sclerites that are interconnected to one another via flexible joints and regulated by the activity of specialized control muscles. Here we imaged the activity of these muscles in a fly using a genetically encoded calcium indicator, while simultaneously tracking the three-dimensional motion of the wings with high-speed cameras. Using machine learning, we created a convolutional neural network3 that accurately predicts wing motion from the activity of the steering muscles, and an encoder–decoder4 that predicts the role of the individual sclerites on wing motion. By replaying patterns of wing motion on a dynamically scaled robotic fly, we quantified the effects of steering muscle activity on aerodynamic forces. A physics-based simulation incorporating our hinge model generates flight manoeuvres that are remarkably similar to those of free-flying flies. This integrative, multi-disciplinary approach reveals the mechanical control logic of the insect wing hinge, arguably among the most sophisticated and evolutionarily important skeletal structures in the natural world.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: The wing hinge of a fly is actuated by large power muscles and controlled by small steering muscles.
Fig. 2: Simultaneous imaging of muscle activity and wing motion.
Fig. 3: A trained CNN predicts wing motion from steering muscle activity and wingbeat frequency.
Fig. 4: Modulation of wing motion by steering muscle activity revealed by virtual experiments.
Fig. 5: MPC simulation of a rightward saccade.
Fig. 6: Proposed role of sclerite function in the control of wing motion.

Similar content being viewed by others

Data availability

The data required to perform the analyses in this paper and reconstruct all the data figure are available in the following files: main_muscle_and_wing_data.h5, flynet_data.zip, robofly_data.zip, which are available from the Caltech Data website: https://doi.org/10.22002/aypcy-ck464. main_muscle_and_wing_data.h5 contains the time series of muscle activity and wing kinematics used to train the muscle-to-wing motion CNN and the encoder–decoder used in the latent variable analysis. flynet_data.zip contains a series of data files for training and running Flynet: (1) camera/calibration/cam_calib.txt (example camera calibration data); (2) movies/session_01_12_2020_10_22 (folder containing example movies); (3) labels.h5 and valid_labels.h5 (data for training); and (4) weights_24_03_2022_09_43_14.h5 (example weights). robofly_data.zip contains the MATLAB data files with force and torque data acquired using the dynamically scaled robotic fly.

Code availability

The code required to perform the analyses in this paper and reconstruct all the data figures are available at https://github.com/FlyRanch/mscode-melis-siwanowicz-dickinson. The software is organized into seven submodules: flynet, flynet-kalman, flynet-optimizer, latent-analysis, mpc-simulations, robofly and wing-hinge-cnn. The installation instructions, system requirements and dependency information are given separately in their respective folders. flynet is a neural network and GUI application that requires the dataset flynet_data.zip, and may be used to create Extended Data Fig. 2. An example demonstrating how to train the network can be found in the examples sub-directory and is called train_flynet.py. flynet-kalman is a Kalman filter Python extension used by Flynet. flynet-optimizer is a particle swarm optimization extension module used by Flynet. latent-analysis is a Python library and Jupyter notebook for performing latent variable analysis that requires the dataset main_muscle_and_wing_data.h5, and may be used to create Fig. 6 and Extended Data Fig. 8. mpc-simulations is a Python library and Jupyter notebook for MPC simulations, and may be used to create Fig. 5 and Extended Data Fig. 7. robofly is a Python library and Jupyter notebook for extracting force and torque data from the robotic fly experiments and plotting forces superimposed on 3D wing kinematics. It requires dataset robofly_data.zip, and may be used to create Extended Data Figs. 5 and  6. wing-hinge-cnn is a Python library and Jupyter notebook for creating the muscle-to-wing motion CNN. It requires main_muscle_and_wing_data.h5, and may be used to create Figs. 3 and 4 and Extended Data Fig. 3. An example demonstrating how to train the network can be found in the examples sub-directory as is called train_wing_hinge_cnn.py. The files containing the raw videos of the muscle Ca2+ images and high-speed videos of wing motion are too large to be hosted on a publicly accessible website. Example high-speed videos are provided in the folder movies/session_01_12_2020_10_22 mentioned in Data availability. Additional sequences are available upon request by contacting the corresponding author.

References

  1. Grimaldi, D. & Engel, M. S. Evolution of the Insects (Cambridge Univ. Press, 2005).

  2. Deora, T., Gundiah, N. & Sane, S. P. Mechanics of the thorax in flies. J. Exp. Biol. 220, 1382–1395 (2017).

    Article  PubMed  Google Scholar 

  3. Gu, J. et al. Recent advances in convolutional neural networks. Pattern Recognit. 77, 354–377 (2018).

    Article  ADS  Google Scholar 

  4. Kramer, M. A. Nonlinear principal component analysis using autoassociative neural networks. AlChE J. 37, 233–243 (1991).

    Article  ADS  CAS  Google Scholar 

  5. Pringle, J. W. S. The excitation and contraction of the flight muscles of insects. J. Physiol. 108, 226–232 (1949).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. Josephson, R. K., Malamud, J. G. & Stokes, D. R. Asynchronous muscle: a primer. J. Exp. Biol. 203, 2713–2722 (2000).

    Article  CAS  PubMed  Google Scholar 

  7. Gau, J. et al. Bridging two insect flight modes in evolution, physiology and robophysics. Nature 622, 767–774 (2023).

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  8. Boettiger, E. G. & Furshpan, E. The mechanics of flight movements in diptera. Biol. Bull. 102, 200–211 (1952).

    Article  Google Scholar 

  9. Pringle, J. W. S. Insect Flight (Cambridge Univ. Press, 1957).

  10. Miyan, J. A. & Ewing, A. W. How Diptera move their wings: a re-examination of the wing base articulation and muscle systems concerned with flight. Phil. Trans. R. Soc. B 311, 271–302 (1985).

    ADS  Google Scholar 

  11. Wisser, A. Wing beat of Calliphora erythrocephala: turning axis and gearbox of the wing base (Insecta, Diptera). Zoomorph. 107, 359–369 (1988).

    Article  Google Scholar 

  12. Ennos, R. A. A comparative study of the flight mechanism of diptera. J. Exp. Biol. 127, 355–372 (1987).

    Article  Google Scholar 

  13. Dickinson, M. H. & Tu, M. S. The function of dipteran flight muscle. Comp. Biochem. Physiol. A 116, 223–238 (1997).

    Article  Google Scholar 

  14. Nalbach, G. The gear change mechanism of the blowfly (Calliphora erythrocephala) in tethered flight. J. Comp. Physiol. A 165, 321–331 (1989).

    Article  Google Scholar 

  15. Walker, S. M., Thomas, A. L. R. & Taylor, G. K. Operation of the alula as an indicator of gear change in hoverflies. J. R. Soc. Inter. 9, 1194–1207 (2011).

    Article  Google Scholar 

  16. Walker, S. M. et al. In vivo time-resolved microtomography reveals the mechanics of the blowfly flight motor. PLoS Biol. 12, e1001823 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  17. Wisser, A. & Nachtigall, W. Functional-morphological investigations on the flight muscles and their insertion points in the blowfly Calliphora erythrocephala (Insecta, Diptera). Zoomorph. 104, 188–195 (1984).

    Article  Google Scholar 

  18. Heide, G. Funktion der nicht-fibrillaren Flugmuskeln von Calliphora. I. Lage Insertionsstellen und Innervierungsmuster der Muskeln. Zool. Jahrb., Abt. allg. Zool. Physiol. Tiere 76, 87–98 (1971).

    Google Scholar 

  19. Fabian, B., Schneeberg, K. & Beutel, R. G. Comparative thoracic anatomy of the wild type and wingless (wg1cn1) mutant of Drosophila melanogaster (Diptera). Arth. Struct. Dev. 45, 611–636 (2016).

    Article  Google Scholar 

  20. Tu, M. & Dickinson, M. Modulation of negative work output from a steering muscle of the blowfly Calliphora vicina. J. Exp. Biol. 192, 207–224 (1994).

    Article  CAS  PubMed  Google Scholar 

  21. Tu, M. S. & Dickinson, M. H. The control of wing kinematics by two steering muscles of the blowfly (Calliphora vicina). J. Comp. Physiol. A 178, 813–830 (1996).

    Article  CAS  PubMed  Google Scholar 

  22. Muijres, F. T., Iwasaki, N. A., Elzinga, M. J., Melis, J. M. & Dickinson, M. H. Flies compensate for unilateral wing damage through modular adjustments of wing and body kinematics. Interface Focus 7, 20160103 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  23. O’Sullivan, A. et al. Multifunctional wing motor control of song and flight. Curr. Biol. 28, 2705–2717.e4 (2018).

    Article  PubMed  Google Scholar 

  24. Azevedo, A. et al. Tools for comprehensive reconstruction and analysis of Drosophila motor circuits. Preprint at BioRxiv https://doi.org/10.1101/2022.12.15.520299 (2022).

  25. Donovan, E. R. et al. Muscle activation patterns and motoranatomy of Anna’s hummingbirds Calypte anna and zebra finches Taeniopygia guttata. Physiol. Biochem. Zool. 86, 27–46 (2013).

    Article  PubMed  Google Scholar 

  26. Bashivan, P., Kar, K. & DiCarlo, J. J. Neural population control via deep image synthesis. Science 364, eaav9436 (2019).

    Article  CAS  PubMed  Google Scholar 

  27. Lindsay, T., Sustar, A. & Dickinson, M. The function and organization of the motor system controlling flight maneuvers in flies. Curr. Biol. 27, 345–358 (2017).

    Article  CAS  PubMed  Google Scholar 

  28. Reiser, M. B. & Dickinson, M. H. A modular display system for insect behavioral neuroscience. J. Neurosci. Meth. 167, 127–139 (2008).

    Article  Google Scholar 

  29. Albawi, S., Mohammed, T. A. & Al-Zawi, S. Understanding of a convolutional neural network. In 2017 International Conference on Engineering and Technology (ICET) 1–6 https://doi.org/10.1109/ICEngTechnol.2017.8308186 (2017).

  30. Kennedy, J. & Eberhart, R. Particle swarm optimization. In Proc. ICNN’95—International Conference on Neural Networks Vol. 4, 1942–1948 (1995).

  31. Dana, H. et al. High-performance calcium sensors for imaging activity in neuronal populations and microcompartments. Nat. Methods 16, 649–657 (2019).

    Article  CAS  PubMed  Google Scholar 

  32. Muijres, F. T., Elzinga, M. J., Melis, J. M. & Dickinson, M. H. Flies evade looming targets by executing rapid visually directed banked turns. Science 344, 172–177 (2014).

    Article  ADS  CAS  PubMed  Google Scholar 

  33. Gordon, S. & Dickinson, M. H. Role of calcium in the regulation of mechanical power in insect flight. Proc. Natl Acad. Sci. USA 103, 4311–4315 (2006).

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  34. Nachtigall, W. & Wilson, D. M. Neuro-muscular control of dipteran flight. J. Exp. Biol. 47, 77–97 (1967).

    Article  CAS  PubMed  Google Scholar 

  35. Heide, G. & Götz, K. G. Optomotor control of course and altitude in Drosophila melanogaster is correlated with distinct activities of at least three pairs of flight steering muscles. J. Exp. Biol. 199, 1711–1726 (1996).

    Article  CAS  PubMed  Google Scholar 

  36. Balint, C. N. & Dickinson, M. H. The correlation between wing kinematics and steering muscle activity in the blowfly Calliphora vicina. J. Exp. Biol. 204, 4213–4226 (2001).

    Article  CAS  PubMed  Google Scholar 

  37. Elzinga, M. J., Dickson, W. B. & Dickinson, M. H. The influence of sensory delay on the yaw dynamics of a flapping insect. J. R. Soc. Interface 9, 1685–1696 (2012).

    Article  PubMed  Google Scholar 

  38. Dickinson, M. H., Lehmann, F.-O. & Sane, S. P. Wing rotation and the aerodynamic basis of insect flight. Science 284, 1954–1960 (1999).

    Article  CAS  PubMed  Google Scholar 

  39. Lehmann, F. O. & Dickinson, M. H. The changes in power requirements and muscle efficiency during elevated force production in the fruit fly Drosophila melanogaster. J. Exp. Biol. 200, 1133–1143 (1997).

    Article  CAS  PubMed  Google Scholar 

  40. Lucia, S., Tătulea-Codrean, A., Schoppmeyer, C. & Engell, S. Rapid development of modular and sustainable nonlinear model predictive control solutions. Control Eng. Pract. 60, 51–62 (2017).

    Article  Google Scholar 

  41. Cheng, B., Fry, S. N., Huang, Q. & Deng, X. Aerodynamic damping during rapid flight maneuvers in the fruit fly Drosophila. J. Exp. Biol. 213, 602–612 (2010).

    Article  CAS  PubMed  Google Scholar 

  42. Collett, T. S. & Land, M. F. Visual control of flight behaviour in the hoverfly, Syritta pipiens L. J. Comp. Physiol. 99, 1–66 (1975).

    Article  Google Scholar 

  43. Muijres, F. T., Elzinga, M. J., Iwasaki, N. A. & Dickinson, M. H. Body saccades of Drosophila consist of stereotyped banked turns. J. Exp. Biol. 218, 864–875 (2015).

    Article  PubMed  Google Scholar 

  44. Syme, D. A. & Josephson, R. K. How to build fast muscles: synchronous and asynchronous designs. Integr. Comp. Biol. 42, 762–770 (2002).

    Article  PubMed  Google Scholar 

  45. Snodgrass, R. E. Principles of Insect Morphology (Cornell Univ. Press, 2018).

  46. Williams, C. M. & Williams, M. V. The flight muscles of Drosophila repleta. J. Morphol. 72, 589–599 (1943).

    Article  Google Scholar 

  47. Wootton, R. The geometry and mechanics of insect wing deformations in flight: a modelling approach. Insects 11, 446 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  48. Lerch, S. et al. Resilin matrix distribution, variability and function in Drosophila. BMC Biol. 18, 195 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  49. Weis-Fogh, T. A rubber-like protein in insect cuticle. J. Exp. Biol. 37, 889–907 (1960).

    Article  CAS  Google Scholar 

  50. Weis-Fogh, T. Energetics of hovering flight in hummingbirds and in Drosophila. J. Exp. Biol. 56, 79–104 (1972).

    Article  Google Scholar 

  51. Ellington, C. P. The aerodynamics of hovering insect flight. VI. Lift and power requirements. Phil. Trans. R. Soc. B 305, 145–181 (1984).

    ADS  Google Scholar 

  52. Alexander, R. M. & Bennet-Clark, H. C. Storage of elastic strain energy in muscle and other tissues. Nature 265, 114–117 (1977).

    Article  ADS  CAS  PubMed  Google Scholar 

  53. Mronz, M. & Lehmann, F.-O. The free-flight response of Drosophila to motion of the visual environment. J. Exp. Biol. 211, 2026–2045 (2008).

    Article  PubMed  Google Scholar 

  54. Ristroph, L., Bergou, A. J., Guckenheimer, J., Wang, Z. J. & Cohen, I. Paddling mode of forward flight in insects. Phys. Rev. Lett. 106, 178103 (2011).

    Article  ADS  PubMed  Google Scholar 

  55. Takemura, S. et al. A connectome of the male Drosophila ventral nerve cord. Preprint at bioRxiv https://doi.org/10.1101/2023.06.05.543757 (2023).

  56. Cheong, H. S. J. et al. Transforming descending input into behavior: The organization of premotor circuits in the Drosophila male adult nerve cord connectome. Preprint at BioRxiv https://doi.org/10.1101/2023.06.07.543976 (2023).

  57. Martynov, A. B. Über zwei Grundtypen der Flügel bei den Insecten und ihre Evolution. Z. Morph. Ökol. Tiere 4, 465–501 (1925).

    Article  Google Scholar 

  58. Wipfler, B. et al. Evolutionary history of Polyneoptera and its implications for our understanding of early winged insects. Proc. Natl Acad. Sci. USA 116, 3024–3029 (2019).

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  59. Hasenfuss, I. The evolutionary pathway to insect flight—a tentative reconstruction. Arthr. System. Phylog. 66, 19–35 (2008).

    Article  Google Scholar 

  60. Willkommen, J. & Hörnschemeyer, T. The homology of wing base sclerites and flight muscles in Ephemeroptera and Neoptera and the morphology of the pterothorax of Habroleptoides confusa (Insecta: Ephemeroptera: Leptophlebiidae). Arthro. Struc. Develop. 36, 253–269 (2007).

    Article  Google Scholar 

  61. Willmann, R. in Arthropod Relationships (eds Fortey, R. A. & Thomas, R. H.) 269–279 (Springer, 1998); https://doi.org/10.1007/978-94-011-4904-4_20.

  62. Shao, L. et al. A neural circuit encoding the experience of copulation in female Drosophila. Neuron 102, 1025–1036.e6 (2019).

    Article  CAS  PubMed  Google Scholar 

  63. Suver, M. P., Huda, A., Iwasaki, N., Safarik, S. & Dickinson, M. H. An array of descending visual interneurons encoding self-motion in Drosophila. J. Neurosci. 36, 11768–11780 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  64. Götz, K. G. Course-control, metabolism and wing interference during ultralong tethered flight in Drosophila melanogaster. J. Exp. Biol. 128, 35–46 (1987).

    Article  Google Scholar 

  65. Klambauer, G., Unterthiner, T., Mayr, A. & Hochreiter, S. in Advances in Neural Information Processing Systems Vol. 30 (Curran Associates, 2017).

  66. Grewal, M. S. & Andrews, A. P. Kalman Filtering: Theory and Practice with MATLAB (John Wiley & Sons, 2014).

  67. Fischler, M. A. & Bolles, R. C. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24, 381–395 (1981).

    Article  MathSciNet  Google Scholar 

  68. Birch, J. M. & Dickinson, M. H. The influence of wing–wake interactions on the production of aerodynamic forces in flapping flight. J. Exp. Biol. 206, 2257–2272 (2003).

    Article  PubMed  Google Scholar 

  69. Kouvaritakis, B. & Cannon, M. Model Predictive Control: Classical, Robust and Stochastic (Springer, 2016).

Download references

Acknowledgements

The authors thank W. Dickson for extensive expertise in instrumentation, programming, data analysis, formatting all the data and code for public repositories, and creating the animations of free flight data in Supplementary Videos 3–8; T. Lindsay for assistance in the design of the epifluorescence microscope and data acquisition software used for muscle imaging; A. Erickson for helpful comments on the manuscript and Supplementary Information; A. Huda for assistance in the construction of genetic lines; J. Omoto for collecting confocal images of wings to visualize resilin using autofluorescence; J. Tuthill and T. Azevedo for a tomographic dataset of the Drosophila wing hinge that was collected at the European Synchrotron Radiation Facility in Grenoble, France; S. Whitehead for analysis of this tomography data to provide a preliminary reconstruction of the hinge sclerites, and for critical feedback on the manuscript text and data presentation; and B. Fabian and R. G. Beutel for providing μ-CT data from their publication on the morphology of the adult fly body. The research reported in this publication was supported by the National Institute of Neurological Disorders and Stroke of the NIH (U19NS104655). I.S. was supported through the AniBody Project Team at HHMI’s Janelia Research Campus for this work.

Author information

Authors and Affiliations

  1. Division of Biology and Bioengineering, California Institute of Technology, Pasadena, CA, USA

    Johan M. Melis & Michael H. Dickinson

  2. Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA, USA

    Igor Siwanowicz

Authors
  1. Johan M. Melis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Igor Siwanowicz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael H. Dickinson
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

J.M.M. collected all the data presented in the manuscript and developed the software for data analysis. J.M.M. and M.H.D. collaborated on planning the experiments, preparing figures, and writing the manuscript. I.S. collected the high-resolution morphological images of the Drosophila thorax and created Supplementary Video 1.

Corresponding author

Correspondence to Michael H. Dickinson.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature thanks the anonymous reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data figures and tables

Extended Data Fig. 1 Automated setup for simultaneous recording of muscle fluorescence and wing motion.

a, Illustration of experimental apparatus, created using Solidworks (www.solidworks.com). High-speed cameras, equipped with 0.5X telecentric lenses and collimated IR back-lighting capture synchronized frames of the fly from three orthogonal angles at a rate of 15,000 frames per second. An epi-fluorescence microscope with a muscle imaging camera records GCaMP7f fluorescence in the left steering muscles at approximately 100 frames per second, utilizing a strobing mechanism triggered every other wingbeat. A blue LED provides a brief, 1 ms illumination of the fly’s thorax during dorsal stroke reversal. A camera operating at 30 fps captures a top view of the fly for the kinefly wing tracker. b, Image of the flight arena featuring the components of the setup: LED panorama, IR diode and wingbeat analyzer for triggering the muscle camera and blue LED, prism for splitting the top view between the high-speed camera and kinefly camera, IR backlight, 4X lens of the epi-fluorescence microscope, and a tethered fly illuminated by the blue LED.

Extended Data Fig. 2 Flynet workflow and definitions of wing kinematic angles.

a, The Flynet algorithm takes three synchronized frames as input. Each frame undergoes CNN processing, resulting in a 256-element feature vector extracted from the image. These three feature vectors are concatenated and analyzed by a fully connected (dense) layer with Scaled Exponential Linear Unit (SELU) activation, consisting of 1024 neurons. The output of the neural network is the predicted state (37 elements) of the five model components represented by a quaternion (q), translation vector (p), and wing deformation angle (ξ). Subsequently, the state vector is refined using 3D model fitting and particle swarm optimization (PSO). Normally distributed noise is added to the predicted state, forming the initial state for 16 particles. During the 3D model fitting, the particles traverse the state-space, maximizing the overlap between binary body and wing masks of the segmented frames (Ib) and the binary masks of the 3D model projected onto the camera views (Ip). The cost function (Ib ∆Ip)/(Ib∪Ip) is evaluated iteratively for a randomly selected 3D model component. The PSO algorithm tracks the personal best cost encountered by each particle and the overall lowest cost (global best). After 300 iterations, the refined state is determined by selecting the global best for each 3D model component. See Supplementary Information for more details. b, Training and validation error of the Flynet CNN as a function of training epoch.

Extended Data Fig. 3 CNN-predicted wing motion for example flight sequences.

a, The top five traces show activity of the steering muscles in the four sclerite groups as well as wingbeat frequency during a full, 1.1 second recording. The bottom four traces indicate comparison between the tracked (black) and CNN-predicted (red) wing kinematic angles throughout the sequence. Expanded plots of a 100-ms sequence (0.5 to 0.6 seconds) are plotted on the right. b, c, d. Same as but for a different flight sequences.

Extended Data Fig. 4 Correlation analysis of steering muscle fluorescence and wingbeat frequency.

Linear models (colored lines) fitted to wingbeats in the entire dataset of 72,219 wingbeats from 82 flies. Gray dots represent the normalized baseline muscle activity level, while colored dots represent the normalized maximum muscle activity level. The correlation coefficients associated with these plots are provided in Extended Data Table 1. For more detail on regression methods, see Supplementary Information.

Extended Data Fig. 5 Aerodynamic force measurements and inertial force calculations.

a, Dynamically scaled flapping fly wing model immersed in mineral oil. b, Non-dimensional forces and torques in the strokeplane reference frame (SRF) for the baseline wingbeat. The four traces in each panel correspond to the total (black: Ftotal, Ttotal), aerodynamic (blue: Faero, Taero), inertial components due to acceleration (green: Facc, Tacc), and inertial components due to angular velocity (red: Fangvel; Tangvel). See Supplementary Information for more details. c, Representation of total forces during the baseline wingbeat, viewed from the front, left, and top. Gray trace represents the wing trajectory; cyan arrows represent instantaneous total force on the wing. At the wing joint, three arrows depict the total mean force, half the body weight, and half the estimated body drag.

Extended Data Fig. 6 Aerodynamic and inertial forces for maximum muscle activity wingbeats.

Figures depict the CNN-predicted wing motion for maximum muscle activity patterns, viewed from the front, left, and top. Instantaneous vectors depicting the sum of aerodynamic and inertial forces are shown in cyan. The wingbeat-averaged force vector is indicated by the color corresponding to the specific steering muscle set to maximum activity. Note that the scaling for the wingbeat-averaged forces differs from that for the instantaneous forces. The black gravitational force and blue body drag force are plotted as in Extended Data Fig. 5c.

Extended Data Fig. 7 Simulation of free flight maneuvers using the state-space system and Model Predictive Control.

a, Schematic of the state-space system and MPC loop, including system matrix (A), control matrix (B), the state vector (x), temporal derivative (\(\dot{x}\)) left and right steering muscle activity (uL, uR), initial state (xinit) and goal state (xgoal). b, Forward flight simulation with wingtip traces in red and blue. c, Wing motion during forward flight simulation plotted in stationary body frame. d, Backward flight simulation. e, Wing motion during backward flight simulation plotted in stationary body frame. f, Left and right steering muscle activity during the forward flight manoeuvre. g, State vector during forward flight manoeuvre. h, Steering muscle activity for the backward flight manoeuvre. i, State vector for the backward flight maneuver. j, CNN-predicted left (red) and right (blue) wing kinematics for the forward flight manoeuvre. Note that because this is a bilaterally symmetric flight manoeuvre, the model generates left and right wing kinematics that are identical. The left wing kinematics are displayed underneath the right kinematics,and thus cannot be seen. A baseline wingbeat is shown to emphasize the relative changes in wing motion. k, CNN-predicted wing motion for the backward flight manoeuvre.

Extended Data Fig. 8 Latent variable analysis reveals sclerite function using an encoder–decoder.

a, The network architecture consists of an encoder (red), muscle activity decoder (green), and wing kinematics decoder (blue). The encoder splits the input data into five streams corresponding to different muscle groups and frequency. Feature extraction is performed using convolutional and fully connected layers with SELU activation. Each stream is projected onto a single latent variable. In the muscle activity decoder, the latent variables are transformed back into the input data. A backpropagation stop prevents weight adjustments in the encoder based on the muscle activity reconstruction. The wing kinematics decoder predicts the Legendre coefficients of wing motion using the latent variables. See Supplementary Information for more details. b, Predicted muscle activity (replotted from Fig. 6) and normalized wingbeat frequency as a function of each latent parameter varied within the range of −3σ to +3σ. Color bar indicates the latent variable value in panels (c) and (e). c, Predicted wing motion by the wing kinematics decoder for the five latent parameters. d, Absolute angle-of-attack (|α|), wingtip velocity (utip) in mm s−1, non-dimensional lift (L mg−1), and non-dimensional drag (D mg−1). The non-dimensional lift and drag were computed using a quasi-steady model as described in Supplementary Information.

Extended Data Fig. 9 Flexible wing root facilitates elastic storage during wingbeat and allows wing to passively respond to changes in lift and drag throughout stroke.

a, Top view of ventral stroke reversal in free flight. Red circles mark the estimated position of the wing hinge, dotted lines indicate the expected position of the wing if a chord-wise flexure line was not present. Images are reproduced from previously publish data32 from Drosophila hydei. b, Composite confocal image of the wing base on Drosophila melanogaster, indicating a bright blue band of auto-fluorescence consistent with the presence of resilin and existence of a chord-wise flexure line (dashed arrows). The image shown is characteristic of the 4 wings (from 4 individual female flies) that we processed for confocal microscopy.

Extended Data Table 1 Correlation slopes and baseline muscle activity pattern
Full size table
Extended Data Table 2 Number of high-speed sequences recorded per trigger muscle
Full size table

Supplementary information

Supplementary Information

This file contains Supplementary Information, including Supplementary Figs. 1–4.

Reporting Summary

Supplementary Video 1

The left side of a Drosophila thorax, annotated to illustrate the arrangement of wing sclerites and associated musculature. The colour scheme used for the sclerites and muscles are consistent with Fig. 1.

Supplementary Video 2

Animations of seven simulated flight maneuvers shown in world and body frames (forward acceleration, backward acceleration, upward acceleration, downward acceleration, left saccade, right saccade, and sideways flight) generated using the CNN model of the wing hinge and state-space model operating with a MPC loop (see Fig. 5 and Extended Data Fig. 7).

Supplementary Video 3

A previously published free flight maneuver of D. hydei (Mujires et al., 2014), animated in the same format as that used to depict the simulated flight maneuvers in Supplementary Video 2. The sequence provides examples of slow and fast saccades and forward acceleration.

Supplementary Video 4

A previously published free flight maneuver of D. hydei (Mujires et al., 2014), animated in the same format as that used to depict the simulated flight maneuvers in Supplementary Video 2. The sequence provides examples of slow ascent, slow descent, and fast ascent.

Supplementary Video 5

A previously published free flight maneuver of D. hydei (Mujires et al., 2014), animated in the same format as that used to depict the simulated flight maneuvers in Supplementary Video 2. The sequence provides examples slow ascent with sideslip and a saccade.

Supplementary Video 6

A previously published free flight maneuver of D. hydei (Mujires et al., 2014), animated in the same format as that used to depict the simulated flight maneuvers in Supplementary Video 2. The sequence provides an example of a very fast saccade.

Supplementary Video 7

A previously published free flight maneuver of D. hydei (Mujires et al., 2014), animated in the same format as that used to depict the simulated flight maneuvers in Supplementary Video 2. The sequence provides an example of backwards flight.

Supplementary Video 8

A previously published free flight maneuver of D. hydei (Mujires et al., 2014), animated in the same format as that used to depict the simulated flight maneuvers in Supplementary Video 2. The sequence provides examples of forward flight with ascent and descent.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Melis, J.M., Siwanowicz, I. & Dickinson, M.H. Machine learning reveals the control mechanics of an insect wing hinge. Nature 628, 795–803 (2024). https://doi.org/10.1038/s41586-024-07293-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41586-024-07293-4

This article is cited by

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing