Dynamic sensory cues shape song structure in Drosophila

Journal name:
Nature
Volume:
507,
Pages:
233–237
Date published:
DOI:
doi:10.1038/nature13131
Received
Accepted
Published online

The generation of acoustic communication signals is widespread across the animal kingdom1, 2, and males of many species, including Drosophilidae, produce patterned courtship songs to increase their chance of success with a female. For some animals, song structure can vary considerably from one rendition to the next3; neural noise within pattern generating circuits is widely assumed to be the primary source of such variability, and statistical models that incorporate neural noise are successful at reproducing the full variation present in natural songs4. In direct contrast, here we demonstrate that much of the pattern variability in Drosophila courtship song can be explained by taking into account the dynamic sensory experience of the male. In particular, using a quantitative behavioural assay combined with computational modelling, we find that males use fast modulations in visual and self-motion signals to pattern their songs, a relationship that we show is evolutionarily conserved. Using neural circuit manipulations, we also identify the pathways involved in song patterning choices and show that females are sensitive to song features. Our data not only demonstrate that Drosophila song production is not a fixed action pattern5, 6, but establish Drosophila as a valuable new model for studies of rapid decision-making under both social and naturalistic conditions.

At a glance

Figures

  1. A novel assay to study Drosophila song behaviour.
    Figure 1: A novel assay to study Drosophila song behaviour.

    a, Behavioural chamber with tracked fly movements (see Methods). Fly movements are divided into: male/female forward velocity (mFV/fFV), male/female lateral and rotational speeds (mLS/fLS and mRS/fRS), the distance between fly centres (Dis), the absolute angle from female/male heading to male/female centre (Ang1/Ang2). b, Segmentation of song bouts into pulse (red) and sine (blue) elements (top). Corresponding traces for mFV and fFV (bottom). c, Song is important for mating. Time to copulation increases (black, *P<0.001) and fraction of copulated pairs decreases (red, *P<0.01) when females are deaf or males are mute. Individual points, mean, and s.d. are given for each genotype (n = 35–48 pairs). AC, arista cut; WC, wing cut. d, Song is variable. The number of repeated bouts (containing pulse and sine) per fly (see Methods). n = 60 wild-type males.

  2. Song bout patterning is predictable and based on few features.
    Figure 2: Song bout patterning is predictable and based on few features.

    a, Schematic of the GLM (see Methods). Inputs—stimulus histories (features; f (t)) for each movement parameter—are used to predict binary event probabilities. Significant features are convolved with a linear filter h(τ), and the result, g(t), is transformed into a probability P(t), via a logistic function. Performance plots show the predicted and actual event probability relationships. Confusion matrices, from which we derive PCor values, quantify model performance. b, Filters for pulse and sine song initiation GLMs. Unlike male lateral speed (mLS) or male forward velocity (mFV), the Dis filter indicates a time lag between distance estimation and sine song initiation. c, GLM performance for identifying pulse song starts (PS) using male forward velocity and male lateral speed filters (n = 11,020 test events from 315 males) and sine song starts (SS) using the Dis filter (n = 2,476 test events from 315 males). N = no song start. d, Male forward velocity pulse song start filters and Dis sine song start filters are similar for data from pheromone-insensitive or arista-cut males or males paired with arista-cut or sex-peptide-injected females; filters from wild-type males are also plotted. e, GLM performance for classifying current song mode (PM, pulse mode; SM, sine mode) using mean male forward velocity and male lateral speed (n = 55,464 test events from 315 males). f, Filters for sine to pulse (S–P) transitions (top) and the pulse to sine (P–S) transitions (bottom). g, GLM performance for identifying S–P transitions (versus continued sine song (S–S)) using male forward velocity and male lateral speed filters (n = 17,118 test events from 315 males) and P–S transitions (versus continued pulse song (P–P)) using male forward velocity and female lateral speed filters (n = 11,748 test events from 315 males). Error bars (most too small to visualize) indicate 95% confidence intervals (c, e, g).

  3. Neural pathways that modulate song patterning.
    Figure 3: Neural pathways that modulate song patterning.

    a, Percentage of song in pulse mode (mean and s.d.) increases with inter-fly distance (r2 = 0.95; see Methods). b, Pulse song percentage increases in blind males (n = 11–48 flies, *P<0.001). c, Individual WT2 males (paired with PIBL females) produce more pulse song in dark versus light (n = 5 flies, *P<0.0001). d, Normalized event frequency for pulse or sine song (red or blue, n = 57,0225 or n = 95,2541, from 315 males) at each male forward velocity, across all wild-type strains. e, PCor values from classifying current song mode (using mean male forward velocity and male lateral speed) for wild-type strains (white bars) and various sensory manipulations (n = 924–16,256 test events from 11–48 flies for each model). f, Two potential neural circuit mechanisms underlying the correlation between male motion and song. Female cue(s) directly modulate both song patterning and locomotor circuits (left) or circuits carrying information about male motion (right) modulate song patterning circuits. g, Wild-type data were split into songs produced when females were not moving or moving (left, black or magenta, n = 9,454 or n = 46,204, test events from 315 males; see Methods). Inset shows corresponding male speeds. Corresponding PCor values from classifying current song mode using mean male forward velocity and male lateral speed (right). h, Song variability with TrpA1-activated flies (n = 14 males from 3 genotypes) is similar to wild type (Fig. 1d). i, PCor values from classifying current song mode using mean mFV and mLS for Fru-A, Fru-B, and P1-activated males (n = 1,987 and 200 and 100 test events from 7 and 10 and 8 males, respectively). j, For each genotype, the percentage of pulse song increases (*P<0.01) when flies are fixed. k, Correlation between female forward velocity and the percentage of pulse song (n = 16,092 from 315 wild-type males paired with PIBL females, binned by percentile) at a 60ms lag (r2 = 0.91). Error bars indicate 95% confidence intervals (e, g, i). b and j show individual data points, mean and s.d. for each group.

  4. Song patterning decisions and female responses.
    Figure 4: Song patterning decisions and female responses.

    a, Filter from GLM for song ends. b, GLM performance for identifying song ends (SE) using female lateral speed (n = 10,708 events from 315 males). N, no song end. c, Summary of the influence of sensory inputs on song patterning, as revealed by GLM analysis. d, Normalized changes in female motion before copulation (n = 233 flies). e, GLM coefficient values between pulse or sine song density and female speed (*P<0.01, n = 40 to n = 1,429 samples from 7–38 pairs; see Methods). f, GLM performance for classifying current song mode using mean male forward velocity and male lateral speed for WT2 or sim (black or green, n = 1424 or 9854 test events from 31 or 40 flies) males paired with oe- (oenocyte-less and PIBL) females. Error bars are 95% confidence intervals (b, e and f).

  5. Courtship behaviour with PIBL females.
    Extended Data Fig. 1: Courtship behaviour with PIBL females.

    a, Recently, genes involved in photoreceptor development have been implicated in JO neuron function35, so we confirmed that the GMR-hid mutation (which induces photoreceptor apoptosis) did not affect JO neuron development. Here we show a single z plane image of the antenna of a wild-type (left) or PIBL (right) female fly, labelled with anti-elav (blue) to mark the nuclei of JO neurons. b, Time to copulation (black) and fraction of copulating pairs (red) are similar for all 8 wild-type strains. n = 34–48 males for each strain. c, The percentage of time males spent singing for all 8 wild-type strains. n = 34–48 males for each strain. d, As in b, but for arista cut, pheromone-insensitive, blind and PIBL males compared with wild-type strains of matched genetic background (WT2 for pheromone-insensitive, blind, and PIBL and WT1 for arista cut). *P<0.05, n = 11–48 males for each genotype. e, As in c, but for arista-cut, pheromone-insensitive, blind, and PIBL males compared with wild-type strains of matched genetic background. *P<0.01, n = 11–48 males for each genotype. b–e, Individual points, mean and s.d. are given for each strain/genotype.

  6. Song bout statistics for wild-type strains courting PIBL females.
    Extended Data Fig. 2: Song bout statistics for wild-type strains courting PIBL females.

    For all panels, data come from the 116 males singing more than 100 song bouts. a, Relative frequency of the pulse/sine ratio for mixed bouts (song bouts containing both sine and pulse elements). n = 15,489 bouts. b, The empirical joint probability density function (PDF) of pulse/sine ratios for consecutive pairs of mixed bouts (see Methods). n = 10,805 bouts. c, As in b, but the independent, rather than empirical, joint PDF. The independent joint PDF is given by multiplying the individual 1D distributions of current and previous bouts for each bin within the 2D space. The distributions in b and c are not significantly different (P = 0.99, two-sample Kolmogorov–Smirnov test). d, Relative frequency of bout durations for mixed bouts. n = 15,489 bouts. e, The empirical joint PDF of bout durations for consecutive pairs of mixed bouts lasting less than 2s. n = 3,535 bouts. f, As in e, but the independent, rather than empirical, joint PDF. The distributions in e and f are not significantly different (P = 0.19, two-sample Kolmogorov–Smirnov test). g, The fraction of mixed bouts starting and ending in pulse mode. n = 15,489 bouts. h, The empirical joint PDF of the ending and starting modes for consecutive pairs of mixed bouts. n = 10,805 bouts. i, As in h, but the independent, rather than empirical, joint PDF. The distributions in h and i are not significantly different (P = 0.99, two-sample Kolmogorov–Smirnov test). j, Relative frequency of the number of mode (sine or pulse) transitions within mixed bouts. n = 15,489 bouts. k, Relative frequency of the durations of each song mode (sine or pulse) within mixed bouts. n = 76,979 song modes. l, Relative frequency of durations of non-mixed song bouts, comprising a single song mode. n = 8,624 or n = 772 for pulse only or sine only, respectively.

  7. Bout triggered averages (BTAs) for all nine movement parameters for song starts.
    Extended Data Fig. 3: Bout triggered averages (BTAs) for all nine movement parameters for song starts.

    BTAs are formed similar to spike-triggered averages (STAs) for neural data. Movement parameters for each of the 8 wild-type strains were aligned to the start of song (n = 2,427–7,586 bouts from 34–48 males). All males were paired with PIBL females. Female and male parameters are coloured magenta and grey, respectively. For each trial, movement parameters were mean-subtracted before averaging (see Methods).

  8. Model selection criteria examples and comparison of model performance statistics.
    Extended Data Fig. 4: Model selection criteria examples and comparison of model performance statistics.

    a, Top, first, we train nine separate GLMs, each based on a single feature, followed by cross-validation on two-thirds of the data, with 1,000 repetitions. The single feature which gives the greatest reduction in deviance is chosen—here male forward velocity for the detection of song bouts that start in pulse mode. Bottom, a second feature is included in the model if the additional reduction in deviance improves the model by a minimum of 10%—here male lateral speed. b, Top, as in a but for song bouts that start in sine mode. Dis is selected as the most predictive feature. Bottom, as in a, but no second feature results in a significant model improvement, so only the one feature model is used. c, Receiver operating characteristic curves for GLM models designed to identify pulse (red) and sine (blue) song starts. Integrating the area under the curve (AUC) shows that both models perform significantly better than chance, for which AUC would be 0.5. AUC = 0.72 (for the pulse starts model) and 0.62 (for the sine starts model). d, Comparison between the PCor and AUC values for every model presented in this study, showing a high correlation between the two measures: r2 = 0.98. For every model tested, the PCor value is a more conservative measure of performance. Error bars indicate 95% confidence intervals, although some are too small to visualize (a–c).

  9. Female forward velocity changes predict male forward velocity changes in wild-type and pheromone-insensitive males, but not blind or PIBL males.
    Extended Data Fig. 5: Female forward velocity changes predict male forward velocity changes in wild-type and pheromone-insensitive males, but not blind or PIBL males.

    a, Relative deviance reduction for GLMs, one for each movement feature, to predict male forward velocity at time points during song. Female forward velocity is the optimal predictor. Error bars indicate 95% confidence intervals, although some are too small to visualize. b, The female forward velocity linear filter is most predictive of male forward velocity values at a lag of ~60ms. c, GLM performance for predicting male forward velocity based only on female forward velocity (n = 58,1814 test events from 315 pairs, r2 = 0.39). Values of male forward velocity and female forward velocity were normalized such that μ = 0 and s.d. = 1 (see Methods). A total of 1% of the data (randomly selected) is plotted here for illustrative purposes. d, As an estimate of the time males spent following females, we measured the maximum cross-covariance (normalized by the auto-covariance) between male and female forward velocities, n = 11–48 males for each strain. Perfect following behaviour, over the entire trial, would produce a value of 1. We tested all following delays between 0 and 300ms. BL and PIBL, but not PI, males show significantly reduced following compared with all other WT strains, *P<0.05. Individual points, mean, and s.d. are given for each strain/genotype. e, The two blind male genotypes (blind and PIBL, red) sing a higher percentage of pulse song at all male speeds (binned to nearest mms−1) compared with wild-type males or males with other sensory manipulations (WT1, WT2, pheromone-insensitive, and arista-cut, black). In all cases, females were PIBL. Speeds >15mms−1 were excluded owing to insufficient data. For each point, n = 1,208–15,736 samples.

  10. Relationships between male-female distance, male velocity and song bout starts.
    Extended Data Fig. 6: Relationships between male–female distance, male velocity and song bout starts.

    a, A two-dimensional normalized kernel density estimate of the male centre relative to the female centre (0,0) at the time of song bout initiation using combined data from all wild-type males. Males are positioned further from the female when they start a song bout in pulse mode (top, n = 27,820 bouts from 315 males) versus sine mode (bottom, n = 5,749 bouts from 315 males). b, Linear filters for male forward velocity and Dis, the most predictive features for the song start mode classification GLM (predicting sine song starts (SS) versus pulse song starts (PS)). c, GLM performance for classifying song start mode with male forward velocity and Dis filters (PCor = 0.73, n = 3,904 test events from 315 males). Error bars indicate 95% confidence intervals. d, Relative frequency distribution of Dis for periods 150ms before the start of song bouts (solid) and during song (dashed), n20,1414 time points from 315 males. The variance in Dis is larger, by 229%, for time points before song. e, As in d, but for male forward velocity. The variance increase in male forward velocity for time points before song is 58%, much smaller than the increase observed with Dis.

  11. Failed copulations do not result from differences in song patterning decisions.
    Extended Data Fig. 7: Failed copulations do not result from differences in song patterning decisions.

    a, Time to copulation (black) and fraction of copulating pairs (red) for sex-peptide-injected (SP) or control-peptide-injected (In-C) females paired with WT1 males (n = 28 or 30 males). Individual data points, mean and s.d. are given for In-C females (no SP females copulated). b, Male forward velocity (solid) and Dis (dashed) linear filters for song start classification (predicting songs that start in pulse mode versus sine mode). The GLM is based on data from wild-type flies that copulated (top, black) or did not copulate (bottom, green). c, GLM performance for classifying song starts with male forward velocity and Dis filters for wild type flies that copulated (black, PCor = 0.72, n = 2,490 test events from 278 males) or did not copulate (green, PCor = 0.71, n = 1,458 test events from 37 males). d, GLM performance for classifying current song mode (based on mean male forward velocity and male lateral speed) for wild-type flies that copulated (black, PCor = 0.78, n = 36,094 test events from 278 males) or did not copulate (green, PCor = 0.81, n = 17,666 test events from 37 males). Error bars indicate 95% confidence intervals, although some are too small to visualize (c, d).

  12. Male velocity consistently predicts the current mode of song.
    Extended Data Fig. 8: Male velocity consistently predicts the current mode of song.

    a, Male-centric features used to examine model performance. Dis and Ang2 are the same as described in Fig. 1a. Siz represents a projection of the female’s body axis onto a plane perpendicular to a line joining the two fly centres (that is, the absolute value of the sine of the angle between female body axis and Dis). Thus, when Siz = 0 or 1, the female occupies a minimal or maximal region of the male’s visual space respectively. dDis, dAng2 and dSiz represent the rate of change of Dis, Ang2 and Siz. b, Comparison of models to classify current song mode based on only male forward velocity and male lateral speed versus all 6 male-centric features combined (*P<0.001). Models were tested using the same data set (n = 55,464 test events from 315 males). c, Deviance reduction statistics for models predicting song bouts starting in sine mode, using only male-centric features as inputs. Dis remains the most important feature (compare with Fig. 2). d, The distribution of Dis during song for wild-type males (n = 338,238 time points from 315 males). e, Left, the distribution of Dis during song for blind males (n = 10,802 time points from 33 males) is broader than for wild type. However, model performance (right) for GLMs using male forward velocity and male lateral speed to classify current song mode remains high for song samples produced at <5mm (black, n = 2,074 test events) or >5mm (green, n = 650 test events) from 33 males. f, As in d, but for Ang2 rather than Dis. g, As in e, but for Ang2 rather than Dis and splitting the data for Ang2<60° (black, n = 2,258 test events) or >60° (green, n = 534 test events). Error bars indicate 95% confidence intervals, although some are too small to visualize (b, c, e, g).

  13. Models to predict current song mode during times when the female is stationary.
    Extended Data Fig. 9: Models to predict current song mode during times when the female is stationary.

    a, GLM performance for classifying current song mode (based on mean male forward velocity and male lateral speed) for wild-type flies. The data set was divided into samples for which the female was effectively stationary during the song sample (black, PCor = 0.85, n = 9,454 test events from 315 males), and those where she was moving (magenta, PCor = 0.76, n = 46,204 test events from 315 males). Model performance remains high even without any motion cues from the female. b, As in a, but the data set is divided according to a shifted estimate of female speed, 240ms before the song sample. This matches the most predictive region of the female BTA (Extended Data Fig. 3). Model performance remains high whether the female is stationary (black, PCor = 0.77, n = 3,110 test events from 315 males) or moving (magenta, PCor = 0.75, n = 44,314 test events from 315 males) 240ms before the song sample. c, As in a, but using data from pheromone-insensitive males. Male velocity remains a successful predictor even when males cannot detect pheromones and when the female is stationary (black, PCor = 0.89, n = 1,788 test events from 25 males) or moving (magenta, PCor = 0.78, n = 9,018 test events from 25 males) during the song sample. Error bars in all plots indicate 95% confidence intervals, although some are too small to visualize.

Tables

  1. Descriptions and acronyms for all fly strains/genotypes
    Extended Data Table 1: Descriptions and acronyms for all fly strains/genotypes

Videos

  1. Tracking of flies in the behavioural chamber
    Video 1: Tracking of flies in the behavioural chamber
    The video shows two flies (WT1 male (grey) and PIBL female (magenta)) interacting in the behavioural chamber, and tracked with our software. Male and female centres are indicated by the larger circles. Dots mark 3 positions along the body axis, with head direction indicated by the larger circles. Lines indicate 3 seconds of tracking history for each fly.

References

  1. Doupe, A. J. & Kuhl, P. K. Birdsong and human speech: common themes and mechanisms. Annu. Rev. Neurosci. 22, 567631 (1999)
  2. Bentley, D. & Hoy, R. R. The neurobiology of cricket song. Sci. Am. 231, 3450 (1974)
  3. Sakata, J. T. & Brainard, M. S. Online contributions of auditory feedback to neural activity in avian song control circuitry. J. Neurosci. 28, 1137811390 (2008)
  4. Jin, D. Z. & Kozhevnikov, A. A. A compact statistical model of the song syntax in Bengalese finch. PLoS Comput. Biol. 7, e1001108 (2011)
  5. Hall, J. C. The mating of a fly. Science 264, 17021714 (1994)
  6. Demir, E. & Dickson, B. J. fruitless splicing specifies male courtship behavior in Drosophila. Cell 121, 785794 (2005)
  7. Ziegler, A. B., Berthelot-Grosjean, M. & Grosjean, Y. The smell of love in Drosophila. Front. Physiol. 4, 72 (2013)
  8. von Schilcher, F. The role of auditory stimuli in the courtship of Drosophila melanogaster. Anim. Behav. 24, 1826 (1976)
  9. Bennet-Clark, H. C. & Ewing, A. W. Pulse interval as a critical parameter in the courtship song of Drosophila melanogaster. Anim. Behav. 17, 755759 (1969)
  10. Baker, B. S., Taylor, B. J. & Hall, J. C. Are complex behaviors specified by dedicated regulatory genes? Reasoning from Drosophila. Cell 105, 1324 (2001)
  11. Mineault, P. J., Barthelme, S. & Pack, C. C. Improved classification images with sparse priors in a smooth basis. J. Vis. 9, http://dx.doi.org/10.1167/9.10.17 (2009)
  12. Calabrese, A., Schumacher, J. W., Schneider, D. M., Paninski, L. & Woolley, S. M. A generalized linear model for estimating spectrotemporal receptive fields from responses to natural sounds. PLoS ONE 6, e16104 (2011)
  13. Pillow, J. W. et al. Spatio-temporal correlations and visual signalling in a complete neuronal population. Nature 454, 995999 (2008)
  14. Sharpee, T., Rust, N. C. & Bialek, W. Analyzing neural responses to natural signals: maximally informative dimensions. Neural Comput. 16, 223250 (2004)
  15. Trott, A. R., Donelson, N. C., Griffith, L. C. & Ejima, A. Song choice is modulated by female movement in Drosophila males. PLoS ONE 7, e46025 (2012)
  16. Soon, C. S., Brass, M., Heinze, H. J. & Haynes, J. D. Unconscious determinants of free decisions in the human brain. Nature Neurosci. 11, 543545 (2008)
  17. Perisse, E. et al. Different Kenyon cell populations drive learned approach and avoidance in Drosophila. Neuron 79, 945956 (2013)
  18. Yapici, N., Kim, Y. J., Ribeiro, C. & Dickson, B. J. A receptor that mediates the post-mating switch in Drosophila reproductive behaviour. Nature 451, 3337 (2008)
  19. McFarland, D. J. Decision making in animals. Nature 269, 1521 (1977)
  20. Sakata, J. T. & Brainard, M. S. Real-time contributions of auditory feedback to avian vocal motor control. J. Neurosci. 26, 96199628 (2006)
  21. Fortune, E. S., Rodriguez, C., Li, D., Ball, G. F. & Coleman, M. J. Neural mechanisms for the coordination of duet singing in wrens. Science 334, 666670 (2011)
  22. Ewing, A. W. The neuromuscular basis of courtship song in Drosophila: the role of the direct and axillary wing muscles. J. Comp. Physiol. 130, 8793 (1979)
  23. Shirangi, T. R., Stern, D. L. & Truman, J. W. Motor control of Drosophila courtship song. Cell Rep. 5, 678686 (2013)
  24. Aronov, D., Veit, L., Goldberg, J. H. & Fee, M. S. Two distinct modes of forebrain circuit dynamics underlie temporal patterning in the vocalizations of young songbirds. J. Neurosci. 31, 1635316368 (2011)
  25. Dickson, B. J. Wired for sex: the neurobiology of Drosophila mating decisions. Science 322, 904909 (2008)
  26. Olveczky, B. P., Andalman, A. S. & Fee, M. S. Vocal experimentation in the juvenile songbird requires a basal ganglia circuit. PLoS Biol. 3, e153 (2005)
  27. Tinbergen, N. The Study of Instinct (Clarendon Press, 1951)
  28. Censi, A., Straw, A. D., Sayaman, R. W., Murray, R. M. & Dickinson, M. H. Discriminating external and internal causes for heading changes in freely flying Drosophila. PLoS Comput. Biol. 9, e1002891 (2013)
  29. Song, C., Qu, Z., Blumm, N. & Barabasi, A. L. Limits of predictability in human mobility. Science 327, 10181021 (2010)
  30. Arthur, B. J., Sunayama-Morita, T., Coen, P., Murthy, M. & Stern, D. L. Multi-channel acoustic recording and automated analysis of Drosophila courtship songs. BMC Biol. 11, 11 (2013)
  31. Yapici, N., Kim, Y. J., Ribeiro, C. & Dickson, B. J. A receptor that mediates the post-mating switch in Drosophila reproductive behaviour. Nature 451, 3337 (2008)
  32. Verleyen, P. et al. SIFamide is a highly conserved neuropeptide: a comparative study in different insect species. Biochem Biophys Res Commun. 320, 334341 (2004)
  33. Simon, J. C. & Dickinson, M. H. A new chamber for studying the behavior of Drosophila. PLoS ONE 5, e8793 (2010)
  34. Deng, Y., Coen, P., Sun, M. & Shaevitz, J. W. Efficient multiple object tracking using mutually repulsive active membranes. PLoS ONE 8, e65769 (2013)
  35. Senthilan, P. R. et al. Drosophila auditory organ genes and genetic hearing defects. Cell 150, 10421054 (2012)

Download references

Author information

Affiliations

  1. Princeton Neuroscience Institute, Princeton University, Princeton, New Jersey 08544, USA

    • Philip Coen,
    • Jan Clemens,
    • Andrew J. Weinstein,
    • Diego A. Pacheco &
    • Mala Murthy
  2. Department of Molecular Biology, Princeton University, Princeton, New Jersey 08544, USA

    • Philip Coen,
    • Jan Clemens,
    • Andrew J. Weinstein,
    • Diego A. Pacheco &
    • Mala Murthy
  3. Department of Physics, Princeton University, Princeton, New Jersey 08544, USA

    • Yi Deng
  4. Present address: Department of Biophysics, University of Washington School of Medicine, Seattle, Washington 98195, USA.

    • Yi Deng

Contributions

P.C. and M.M. designed the study. P.C., A.J.W. and D.A.P. collected and processed the data. Y.D. developed the fly tracking algorithm. P.C. and J.C. analysed the data. P.C. and M.M. wrote the paper.

Competing financial interests

The authors declare no competing financial interests.

Corresponding author

Correspondence to:

Author details

Extended data figures and tables

Extended Data Figures

  1. Extended Data Figure 1: Courtship behaviour with PIBL females. (158 KB)

    a, Recently, genes involved in photoreceptor development have been implicated in JO neuron function35, so we confirmed that the GMR-hid mutation (which induces photoreceptor apoptosis) did not affect JO neuron development. Here we show a single z plane image of the antenna of a wild-type (left) or PIBL (right) female fly, labelled with anti-elav (blue) to mark the nuclei of JO neurons. b, Time to copulation (black) and fraction of copulating pairs (red) are similar for all 8 wild-type strains. n = 34–48 males for each strain. c, The percentage of time males spent singing for all 8 wild-type strains. n = 34–48 males for each strain. d, As in b, but for arista cut, pheromone-insensitive, blind and PIBL males compared with wild-type strains of matched genetic background (WT2 for pheromone-insensitive, blind, and PIBL and WT1 for arista cut). *P<0.05, n = 11–48 males for each genotype. e, As in c, but for arista-cut, pheromone-insensitive, blind, and PIBL males compared with wild-type strains of matched genetic background. *P<0.01, n = 11–48 males for each genotype. b–e, Individual points, mean and s.d. are given for each strain/genotype.

  2. Extended Data Figure 2: Song bout statistics for wild-type strains courting PIBL females. (118 KB)

    For all panels, data come from the 116 males singing more than 100 song bouts. a, Relative frequency of the pulse/sine ratio for mixed bouts (song bouts containing both sine and pulse elements). n = 15,489 bouts. b, The empirical joint probability density function (PDF) of pulse/sine ratios for consecutive pairs of mixed bouts (see Methods). n = 10,805 bouts. c, As in b, but the independent, rather than empirical, joint PDF. The independent joint PDF is given by multiplying the individual 1D distributions of current and previous bouts for each bin within the 2D space. The distributions in b and c are not significantly different (P = 0.99, two-sample Kolmogorov–Smirnov test). d, Relative frequency of bout durations for mixed bouts. n = 15,489 bouts. e, The empirical joint PDF of bout durations for consecutive pairs of mixed bouts lasting less than 2s. n = 3,535 bouts. f, As in e, but the independent, rather than empirical, joint PDF. The distributions in e and f are not significantly different (P = 0.19, two-sample Kolmogorov–Smirnov test). g, The fraction of mixed bouts starting and ending in pulse mode. n = 15,489 bouts. h, The empirical joint PDF of the ending and starting modes for consecutive pairs of mixed bouts. n = 10,805 bouts. i, As in h, but the independent, rather than empirical, joint PDF. The distributions in h and i are not significantly different (P = 0.99, two-sample Kolmogorov–Smirnov test). j, Relative frequency of the number of mode (sine or pulse) transitions within mixed bouts. n = 15,489 bouts. k, Relative frequency of the durations of each song mode (sine or pulse) within mixed bouts. n = 76,979 song modes. l, Relative frequency of durations of non-mixed song bouts, comprising a single song mode. n = 8,624 or n = 772 for pulse only or sine only, respectively.

  3. Extended Data Figure 3: Bout triggered averages (BTAs) for all nine movement parameters for song starts. (173 KB)

    BTAs are formed similar to spike-triggered averages (STAs) for neural data. Movement parameters for each of the 8 wild-type strains were aligned to the start of song (n = 2,427–7,586 bouts from 34–48 males). All males were paired with PIBL females. Female and male parameters are coloured magenta and grey, respectively. For each trial, movement parameters were mean-subtracted before averaging (see Methods).

  4. Extended Data Figure 4: Model selection criteria examples and comparison of model performance statistics. (92 KB)

    a, Top, first, we train nine separate GLMs, each based on a single feature, followed by cross-validation on two-thirds of the data, with 1,000 repetitions. The single feature which gives the greatest reduction in deviance is chosen—here male forward velocity for the detection of song bouts that start in pulse mode. Bottom, a second feature is included in the model if the additional reduction in deviance improves the model by a minimum of 10%—here male lateral speed. b, Top, as in a but for song bouts that start in sine mode. Dis is selected as the most predictive feature. Bottom, as in a, but no second feature results in a significant model improvement, so only the one feature model is used. c, Receiver operating characteristic curves for GLM models designed to identify pulse (red) and sine (blue) song starts. Integrating the area under the curve (AUC) shows that both models perform significantly better than chance, for which AUC would be 0.5. AUC = 0.72 (for the pulse starts model) and 0.62 (for the sine starts model). d, Comparison between the PCor and AUC values for every model presented in this study, showing a high correlation between the two measures: r2 = 0.98. For every model tested, the PCor value is a more conservative measure of performance. Error bars indicate 95% confidence intervals, although some are too small to visualize (a–c).

  5. Extended Data Figure 5: Female forward velocity changes predict male forward velocity changes in wild-type and pheromone-insensitive males, but not blind or PIBL males. (136 KB)

    a, Relative deviance reduction for GLMs, one for each movement feature, to predict male forward velocity at time points during song. Female forward velocity is the optimal predictor. Error bars indicate 95% confidence intervals, although some are too small to visualize. b, The female forward velocity linear filter is most predictive of male forward velocity values at a lag of ~60ms. c, GLM performance for predicting male forward velocity based only on female forward velocity (n = 58,1814 test events from 315 pairs, r2 = 0.39). Values of male forward velocity and female forward velocity were normalized such that μ = 0 and s.d. = 1 (see Methods). A total of 1% of the data (randomly selected) is plotted here for illustrative purposes. d, As an estimate of the time males spent following females, we measured the maximum cross-covariance (normalized by the auto-covariance) between male and female forward velocities, n = 11–48 males for each strain. Perfect following behaviour, over the entire trial, would produce a value of 1. We tested all following delays between 0 and 300ms. BL and PIBL, but not PI, males show significantly reduced following compared with all other WT strains, *P<0.05. Individual points, mean, and s.d. are given for each strain/genotype. e, The two blind male genotypes (blind and PIBL, red) sing a higher percentage of pulse song at all male speeds (binned to nearest mms−1) compared with wild-type males or males with other sensory manipulations (WT1, WT2, pheromone-insensitive, and arista-cut, black). In all cases, females were PIBL. Speeds >15mms−1 were excluded owing to insufficient data. For each point, n = 1,208–15,736 samples.

  6. Extended Data Figure 6: Relationships between male–female distance, male velocity and song bout starts. (90 KB)

    a, A two-dimensional normalized kernel density estimate of the male centre relative to the female centre (0,0) at the time of song bout initiation using combined data from all wild-type males. Males are positioned further from the female when they start a song bout in pulse mode (top, n = 27,820 bouts from 315 males) versus sine mode (bottom, n = 5,749 bouts from 315 males). b, Linear filters for male forward velocity and Dis, the most predictive features for the song start mode classification GLM (predicting sine song starts (SS) versus pulse song starts (PS)). c, GLM performance for classifying song start mode with male forward velocity and Dis filters (PCor = 0.73, n = 3,904 test events from 315 males). Error bars indicate 95% confidence intervals. d, Relative frequency distribution of Dis for periods 150ms before the start of song bouts (solid) and during song (dashed), n20,1414 time points from 315 males. The variance in Dis is larger, by 229%, for time points before song. e, As in d, but for male forward velocity. The variance increase in male forward velocity for time points before song is 58%, much smaller than the increase observed with Dis.

  7. Extended Data Figure 7: Failed copulations do not result from differences in song patterning decisions. (124 KB)

    a, Time to copulation (black) and fraction of copulating pairs (red) for sex-peptide-injected (SP) or control-peptide-injected (In-C) females paired with WT1 males (n = 28 or 30 males). Individual data points, mean and s.d. are given for In-C females (no SP females copulated). b, Male forward velocity (solid) and Dis (dashed) linear filters for song start classification (predicting songs that start in pulse mode versus sine mode). The GLM is based on data from wild-type flies that copulated (top, black) or did not copulate (bottom, green). c, GLM performance for classifying song starts with male forward velocity and Dis filters for wild type flies that copulated (black, PCor = 0.72, n = 2,490 test events from 278 males) or did not copulate (green, PCor = 0.71, n = 1,458 test events from 37 males). d, GLM performance for classifying current song mode (based on mean male forward velocity and male lateral speed) for wild-type flies that copulated (black, PCor = 0.78, n = 36,094 test events from 278 males) or did not copulate (green, PCor = 0.81, n = 17,666 test events from 37 males). Error bars indicate 95% confidence intervals, although some are too small to visualize (c, d).

  8. Extended Data Figure 8: Male velocity consistently predicts the current mode of song. (90 KB)

    a, Male-centric features used to examine model performance. Dis and Ang2 are the same as described in Fig. 1a. Siz represents a projection of the female’s body axis onto a plane perpendicular to a line joining the two fly centres (that is, the absolute value of the sine of the angle between female body axis and Dis). Thus, when Siz = 0 or 1, the female occupies a minimal or maximal region of the male’s visual space respectively. dDis, dAng2 and dSiz represent the rate of change of Dis, Ang2 and Siz. b, Comparison of models to classify current song mode based on only male forward velocity and male lateral speed versus all 6 male-centric features combined (*P<0.001). Models were tested using the same data set (n = 55,464 test events from 315 males). c, Deviance reduction statistics for models predicting song bouts starting in sine mode, using only male-centric features as inputs. Dis remains the most important feature (compare with Fig. 2). d, The distribution of Dis during song for wild-type males (n = 338,238 time points from 315 males). e, Left, the distribution of Dis during song for blind males (n = 10,802 time points from 33 males) is broader than for wild type. However, model performance (right) for GLMs using male forward velocity and male lateral speed to classify current song mode remains high for song samples produced at <5mm (black, n = 2,074 test events) or >5mm (green, n = 650 test events) from 33 males. f, As in d, but for Ang2 rather than Dis. g, As in e, but for Ang2 rather than Dis and splitting the data for Ang2<60° (black, n = 2,258 test events) or >60° (green, n = 534 test events). Error bars indicate 95% confidence intervals, although some are too small to visualize (b, c, e, g).

  9. Extended Data Figure 9: Models to predict current song mode during times when the female is stationary. (114 KB)

    a, GLM performance for classifying current song mode (based on mean male forward velocity and male lateral speed) for wild-type flies. The data set was divided into samples for which the female was effectively stationary during the song sample (black, PCor = 0.85, n = 9,454 test events from 315 males), and those where she was moving (magenta, PCor = 0.76, n = 46,204 test events from 315 males). Model performance remains high even without any motion cues from the female. b, As in a, but the data set is divided according to a shifted estimate of female speed, 240ms before the song sample. This matches the most predictive region of the female BTA (Extended Data Fig. 3). Model performance remains high whether the female is stationary (black, PCor = 0.77, n = 3,110 test events from 315 males) or moving (magenta, PCor = 0.75, n = 44,314 test events from 315 males) 240ms before the song sample. c, As in a, but using data from pheromone-insensitive males. Male velocity remains a successful predictor even when males cannot detect pheromones and when the female is stationary (black, PCor = 0.89, n = 1,788 test events from 25 males) or moving (magenta, PCor = 0.78, n = 9,018 test events from 25 males) during the song sample. Error bars in all plots indicate 95% confidence intervals, although some are too small to visualize.

Extended Data Tables

  1. Extended Data Table 1: Descriptions and acronyms for all fly strains/genotypes (188 KB)

Supplementary information

Video

  1. Video 1: Tracking of flies in the behavioural chamber (3.14 MB, Download)
    The video shows two flies (WT1 male (grey) and PIBL female (magenta)) interacting in the behavioural chamber, and tracked with our software. Male and female centres are indicated by the larger circles. Dots mark 3 positions along the body axis, with head direction indicated by the larger circles. Lines indicate 3 seconds of tracking history for each fly.

Additional data