Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

A database and deep learning toolbox for noise-optimized, generalized spike inference from calcium imaging

Nature Neuroscience volume 24, pages 1324–1337 (2021)Cite this article

Subjects

Abstract

Inference of action potentials (‘spikes’) from neuronal calcium signals is complicated by the scarcity of simultaneous measurements of action potentials and calcium signals (‘ground truth’). In this study, we compiled a large, diverse ground truth database from publicly available and newly performed recordings in zebrafish and mice covering a broad range of calcium indicators, cell types and signal-to-noise ratios, comprising a total of more than 35 recording hours from 298 neurons. We developed an algorithm for spike inference (termed CASCADE) that is based on supervised deep networks, takes advantage of the ground truth database, infers absolute spike rates and outperforms existing model-based algorithms. To optimize performance for unseen imaging data, CASCADE retrains itself by resampling ground truth data to match the respective sampling rate and noise level; therefore, no parameters need to be adjusted by the user. In addition, we developed systematic performance assessments for unseen data, openly released a resource toolbox and provide a user-friendly cloud-based implementation.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Rent or buy this article

Get just this article for as long as you need it

$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Ground truth datasets.
Fig. 2: Training a deep network with noise-matched ground truth improves spike inference.
Fig. 3: Generalization across datasets.
Fig. 4: Comparison with model-based algorithms.
Fig. 5: Inference of spiking activity with CASCADE from population calcium imaging across more than 1,100 neurons in adult zebrafish.
Fig. 6: Inference of spiking activity with CASCADE for the Allen Brain Observatory dataset in mice.

Data availability

Ground truth data, including extracted spike times and calcium traces, are deposited in the GitHub repository together with demo scripts (https://github.com/HelmchenLabSoftware/Cascade). We provide a cloud-based Colaboratory Notebook that allows for interactive browsing through all datasets (https://colab.research.google.com/github/HelmchenLabSoftware/Cascade/blob/master/Demo%20scripts/Explore_ground_truth_datasets.ipynb). Raw data were recorded in different formats, and all newly recorded raw datasets are also available upon reasonable request in their original formats. Publicly available datasets are described in detail in the Methods (‘Extraction of ground truth from publicly available datasets’).

Additional information on experimental design and reagents is available in the Research Life Sciences Reporting Summary linked to this paper.

Code availability

A cloud-based version of CASCADE is available as a Colaboratory Notebook (https://colab.research.google.com/github/HelmchenLabSoftware/Cascade/blob/master/Demo%20scripts/Calibrated_spike_inference_with_Cascade.ipynb). The code is also available as a GitHub repository together with demo scripts, installation instructions and FAQs (https://github.com/HelmchenLabSoftware/Cascade). Pre-trained models for CASCADE are archived in an online server (https://www.switch.ch/drive/) and retrieved automatically by the CASCADE code.

References

  1. Göbel, W. & Helmchen, F. In vivo calcium imaging of neural network function. Physiology 22, 358–365 (2007).

    Article  PubMed  Google Scholar 

  2. Harris, K. D., Quiroga, R. Q., Freeman, J. & Smith, S. L. Improving data quality in neuronal population recordings. Nat. Neurosci. 19, 1165–1174 (2016).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  3. Rose, T., Goltstein, P. M., Portugues, R. & Griesbeck, O. Putting a finishing touch on GECIs. Front. Mol. Neurosci. 7, 88 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  4. Sabatini, B. L. The impact of reporter kinetics on the interpretation of data gathered with fluorescent reporters. Preprint at https://www.biorxiv.org/content/10.1101/834895v1 (2019).

  5. Wei, Z. et al. A comparison of neuronal population dynamics measured with calcium imaging and electrophysiology. PLoS Comput. Biol. 16, e1008198 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. Ali, F. & Kwan, A. C. Interpreting in vivo calcium signals from neuronal cell bodies, axons, and dendrites: a review. Neurophotonics 7, 011402 (2020).

    CAS  PubMed  Google Scholar 

  7. Yaksi, E. & Friedrich, R. W. Reconstruction of firing rate changes across neuronal populations by temporally deconvolved Ca2+ imaging. Nat. Methods 3, 377–383 (2006).

    Article  CAS  PubMed  Google Scholar 

  8. Greenberg, D. S., Houweling, A. R. & Kerr, J. N. D. Population imaging of ongoing neuronal activity in the visual cortex of awake rats. Nat. Neurosci. 11, 749–751 (2008).

    Article  CAS  PubMed  Google Scholar 

  9. Vogelstein, J. T. et al. Spike inference from calcium imaging using sequential Monte Carlo methods. Biophys. J. 97, 636–655 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Vogelstein, J. T. et al. Fast nonnegative deconvolution for spike train inference from population calcium imaging. J. Neurophysiol. 104, 3691–3704 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  11. Lütcke, H., Gerhard, F., Zenke, F., Gerstner, W. & Helmchen, F. Inference of neuronal network spike dynamics and topology from calcium imaging data. Front. Neural Circuits 7, 201 (2013).

    Article  PubMed  PubMed Central  Google Scholar 

  12. Deneux, T. et al. Accurate spike estimation from noisy calcium signals for ultrafast three-dimensional imaging of large neuronal populations in vivo. Nat. Commun. 7, 1–17 (2016).

    Article  Google Scholar 

  13. Greenberg, D. S. et al. Accurate action potential inference from a calcium sensor protein through biophysical modeling. Preprint at https://www.biorxiv.org/content/10.1101/479055v1 (2018).

  14. Pachitariu, M., Stringer, C. & Harris, K. D. Robustness of spike deconvolution for neuronal calcium imaging. J. Neurosci. 38, 7976–7985 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Friedrich, J., Zhou, P. & Paninski, L. Fast online deconvolution of calcium imaging data. PLoS Comput. Biol. 13, e1005423 (2017).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  16. Berens, P. et al. Community-based benchmarking improves spike rate inference from two-photon calcium imaging data. PLoS Comput. Biol. 14, e1006157 (2018).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  17. Jewell, S. & Witten, D. Exact spike inference via l0 optimization. Ann. Appl. Stat. 12, 2457–2482 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  18. Sasaki, T., Takahashi, N., Matsuki, N. & Ikegaya, Y. Fast and accurate detection of action potentials from somatic calcium fluctuations. J. Neurophysiol. 100, 1668–1676 (2008).

    Article  CAS  PubMed  Google Scholar 

  19. Theis, L. et al. Benchmarking spike rate inference in population calcium imaging. Neuron 90, 471–482 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  20. Sebastian, J., Sur, M., Murthy, H. A. & Magimai-Doss, M. Signal-to-signal neural networks for improved spike estimation from calcium imaging data. PLoS Comput. Biol. 17, e1007921 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  21. Hoang, H. et al. Improved hyperacuity estimation of spike timing from calcium imaging. Sci. Rep. 10, 17844 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Éltes, T., Szoboszlay, M., Kerti-Szigeti, K. & Nusser, Z. Improved spike inference accuracy by estimating the peak amplitude of unitary [Ca2+] transients in weakly GCaMP6f-expressing hippocampal pyramidal cells. J. Physiol. 597, 2925–2947 (2019).

    Article  PubMed  CAS  Google Scholar 

  23. Evans, M. H., Petersen, R. S. & Humphries, M. D. On the use of calcium deconvolution algorithms in practical contexts. Preprint at https://www.biorxiv.org/content/10.1101/871137v1 (2019).

  24. Zhu, P., Fajardo, O., Shum, J., Zhang Schärer, Y.-P. & Friedrich, R. W. High-resolution optical control of spatiotemporal neuronal activity patterns in zebrafish using a digital micromirror device. Nat. Protoc. 7, 1410–1425 (2012).

    Article  CAS  PubMed  Google Scholar 

  25. Schoenfeld, G., Carta, S., Rupprecht, P., Ayaz, A. & Helmchen, F. In vivo calcium imaging of CA3 pyramidal neuron populations in adult mouse hippocampus. Preprint at https://www.biorxiv.org/content/10.1101/2021.01.21.427642v1 (2021).

  26. Bethge, P. et al. An R-CaMP1.07 reporter mouse for cell-type-specific expression of a sensitive red fluorescent calcium indicator. PLoS ONE 12, e0179460 (2017).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  27. Tada, M., Takeuchi, A., Hashizume, M., Kitamura, K. & Kano, M. A highly sensitive fluorescent indicator dye for calcium imaging of neural activity in vitro and in vivo. Eur. J. Neurosci. 39, 1720–1728 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  28. Khan, A. G. et al. Distinct learning-induced changes in stimulus selectivity and interactions of GABAergic interneuron classes in visual cortex. Nat. Neurosci. 21, 851–859 (2018).

    Article  CAS  PubMed  Google Scholar 

  29. Kwan, A. C. & Dan, Y. Dissection of cortical microcircuits by single-neuron stimulation in vivo. Curr. Biol. CB 22, 1459–1467 (2012).

    Article  CAS  PubMed  Google Scholar 

  30. Huang, L. et al. Relationship between simultaneously recorded spiking activity and fluorescence signal in GCaMP6 transgenic mice. Elife 10, e51675 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  31. Ledochowitsch, P. et al. On the correspondence of electrical and optical physiology in in vivo population-scale two-photon calcium imaging. Preprint at https://www.biorxiv.org/content/10.1101/800102v1 (2019).

  32. Dana, H. et al. Sensitive red protein calcium indicators for imaging neural activity. eLife 5, e12727 (2016).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  33. Akerboom, J. et al. Optimization of a GCaMP calcium indicator for neural activity imaging. J. Neurosci. 32, 13819–13840 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. Chen, T.-W. et al. Ultrasensitive fluorescent proteins for imaging neuronal activity. Nature 499, 295–300 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Russakovsky, O. et al. ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115, 211–252 (2015).

    Article  Google Scholar 

  36. Mathis, A. et al. DeepLabCut: markerless pose estimation of user-defined body parts with deep learning. Nat. Neurosci. 21, 1281–1289 (2018).

    Article  CAS  PubMed  Google Scholar 

  37. Deng, J. et al. ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition https://ieeexplore.ieee.org/document/5206848 (2009).

  38. Giovannucci, A. et al. CaImAn an open source tool for scalable calcium imaging data analysis. eLife 8, e38173 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  39. Keemink, S. W. et al. FISSA: a neuropil decontamination toolbox for calcium imaging signals. Sci. Rep. 8, 3493 (2018).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  40. Charles, A. S., Song, A., Gauthier, J. L., Pillow, J. W. & Tank, D. W. Neural anatomy and optical microscopy (NAOMi) simulation for evaluating calcium imaging methods. J. Neurosci. Methods 358, 109173 (2019).

  41. Pachitariu, M. et al. Suite2p: beyond 10,000 neurons with standard two-photon microscopy. Preprint at https://www.biorxiv.org/content/10.1101/061507v2 (2017).

  42. Jewell, S., Hocking, T. D., Fearnhead, P. & Witten, D. Fast nonconvex deconvolution of calcium imaging data. Biostatistics 21, 709–726 (2019).

  43. Rupprecht, P., Prendergast, A., Wyart, C. & Friedrich, R. W. Remote z-scanning with a macroscopic voice coil motor for fast 3D multiphoton laser scanning microscopy. Biomed. Opt. Express 7, 1656–1671 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  44. Blumhagen, F. et al. Neuronal filtering of multiplexed odour representations. Nature 479, 493–498 (2011).

    Article  CAS  PubMed  Google Scholar 

  45. Rupprecht, P. & Friedrich, R. W. Precise synaptic balance in the zebrafish homolog of olfactory cortex. Neuron 100, 669–683.e5 (2018).

    Article  CAS  PubMed  Google Scholar 

  46. Mackevicius, E. L. et al. Unsupervised discovery of temporal sequences in high-dimensional datasets, with applications to neuroscience. eLife 8, e38471 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  47. de Vries, S. E. J. et al. A large-scale standardized physiological survey reveals functional organization of the mouse visual cortex. Nat. Neurosci. 23, 138–151 (2020).

    Article  PubMed  CAS  Google Scholar 

  48. Lin, I.-C., Okun, M., Carandini, M. & Harris, K. D. The nature of shared cortical variability. Neuron 87, 644–656 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  49. Kaifosh, P., Zaremba, J. D., Danielson, N. B. & Losonczy, A. SIMA: Python software for analysis of dynamic fluorescence imaging data. Front. Neuroinformatics 8, 80 (2014).

    Article  Google Scholar 

  50. Siegle, J. H. et al. Reconciling functional differences in populations of neurons recorded with two-photon imaging and electrophysiology. Preprint at https://www.biorxiv.org/content/10.1101/2020.08.10.244723v1.full (2020).

  51. Vanwalleghem, G., Constantin, L. & Scott, E. K. Calcium imaging and the curse of negativity. Front. Neural Circuits 14, 607391 (2021).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  52. Kay, K. et al. Constant sub-second cycling between representations of possible futures in the hippocampus. Cell 180, 552–567 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  53. van der, Bourg,A. et al. Temporal refinement of sensory-evoked activity across layers in developing mouse barrel cortex. Eur. J. Neurosci. 50, 2955–2969 (2019).

    Article  Google Scholar 

  54. Pégard, N. C. et al. Three-dimensional scanless holographic optogenetics with temporal focusing (3D-SHOT). Nat. Commun. 8, 1228 (2017).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  55. Packer, A. M., Russell, L. E., Dalgleish, H. W. P. & Häusser, M. Simultaneous all-optical manipulation and recording of neural circuit activity with cellular resolution in vivo. Nat. Methods 12, 140–146 (2015).

    Article  CAS  PubMed  Google Scholar 

  56. Griffiths, V. A. et al. Real-time 3D movement correction for two-photon imaging in behaving animals. Nat. Methods 17, 741–748 (2020).

  57. Inoue, M. et al. Rational engineering of XCaMPs, a multicolor GECI suite for in vivo imaging of complex brain circuit dynamics. Cell 177, 1346–1360 (2019).

    Article  CAS  PubMed  Google Scholar 

  58. Frank, T., Mönig, N. R., Satou, C., Higashijima, S. & Friedrich, R. W. Associative conditioning remaps odor representations and modifies inhibition in a higher olfactory brain area. Nat. Neurosci. 22, 1844–1856 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  59. Kitamura, K., Judkewitz, B., Kano, M., Denk, W. & Häusser, M. Targeted patch-clamp recordings and single-cell electroporation of unlabeled neurons in vivo. Nat. Methods 5, 61–67 (2008).

    Article  CAS  PubMed  Google Scholar 

  60. Perkins, K. L. Cell-attached voltage-clamp and current-clamp recording and stimulation techniques in brain slices. J. Neurosci. Methods 154, 1–18 (2006).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  61. Pologruto, T. A., Sabatini, B. L. & Svoboda, K. ScanImage: flexible software for operating laser scanning microscopes. Biomed. Eng. Online 2, 13 (2003).

    Article  PubMed  PubMed Central  Google Scholar 

  62. Suter, B. A. et al. Ephus: multipurpose data acquisition software for neuroscience experiments. Front. Neural Circuits 4, 100 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  63. Huang, K.-H. et al. A virtual reality system to analyze neural activity and behavior in adult zebrafish. Nat. Methods 17, 343–351 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  64. Langer, D. et al. HelioScan: a software framework for controlling in vivo microscopy setups with high hardware flexibility, functional diversity and extendibility. J. Neurosci. Methods 215, 38–52 (2013).

    Article  PubMed  Google Scholar 

  65. Pecka, M., Han, Y., Sader, E. & Mrsic-Flogel, T. D. Experience-dependent specialization of receptive field surround for selective coding of natural scenes. Neuron 84, 457–469 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  66. Pernía-Andrade, A. J. et al. A deconvolution-based method with high sensitivity and temporal resolution for detection of spontaneous synaptic currents in vitro and in vivo. Biophys. J. 103, 1429–1439 (2012).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  67. Guzman, S. J., Schlögl, A. & Schmidt-Hieber, C. Stimfit: quantifying electrophysiological data with Python. Front. Neuroinformatics 8, 16 (2014).

    Article  Google Scholar 

  68. GENIE project, Janelia Farm Campus, HHMI & Svoboda, K. Simultaneous imaging and loose-seal cell-attached electrical recordings from neurons expressing a variety of genetically encoded calcium indicators. https://crcns.org/data-sets/methods/cai-1/about-cai-1 (2015).

  69. Boaz, M., Dana, H., Kim, D. S., Svoboda, K. & GENIE project, Janelia Farm Campus, HHMI. jRGECO1a and jRCaMP1a characterization in the intact mouse visual cortex, using AAV-based gene transfer, 2-photon imaging and loose-seal cell attached recordings. https://crcns.org/data-sets/methods/cai-2/about-cai-2 (2016).

  70. Reynolds, S., Abrahamsson, T., Sjöström, P. J., Schultz, S. R. & Dragotti, P. L. CosMIC: a consistent metric for spike inference from calcium imaging. Neural Comput. 30, 2726–2756 (2018).

    Article  PubMed  Google Scholar 

  71. Ioffe, S. & Szegedy, C. Batch normalization: accelerating deep network training by reducing internal covariate shift. In International conference on machine learning. 448-456 (PMLR, 2015).

  72. Hochreiter, S. & Schmidhuber, J. Long short-term memory. Neural Comput. 9, 1735–1780 (1997).

    Article  CAS  PubMed  Google Scholar 

  73. Gers, F. A., Schmidhuber, J. & Cummins, F. Learning to forget: continual prediction with LSTM. Neural Comput. 12, 2451–2471 (1999).

  74. Schuster, M. & Paliwal, K. Bidirectional recurrent neural networks. Signal Process. IEEE Trans. 45, 2673–2681 (1997).

    Article  Google Scholar 

  75. Graves A., Fernández S., Schmidhuber J. Bidirectional LSTM Networks for Improved Phoneme Classification and Recognition. Duch W., Kacprzyk J., Oja E., Zadrożny S. (eds) Artificial Neural Networks: Formal Models and Their Applications – ICANN 2005. Lecture Notes in Computer Science, vol 3697. (Springer, Berlin, Heidelberg, 2005).

  76. Eden, U. T. & Kramer, M. A. Drawing inferences from Fano factor calculations. J. Neurosci. Methods 190, 149–152 (2010).

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We thank the members of the GENIE project, the Allen Institute and the Spikefinder project for publicly providing existing ground truth datasets together with excellent documentation. We thank P. Berens and E. Froudarakis for providing additional information on the Spikefinder datasets. We thank G. Schoenfeld for helpful discussions on DS #18 and H. Heiser, N. Temiz, C. Satou, G. Schoenfeld and H. Luetcke for testing earlier versions of the toolbox. This work was supported by grants to F.H. from the Swiss National Science Foundation (project grant no. 310030-127091 and Sinergia grant no. CRSII5-18O316) and the European Research Council (ERC Advanced Grant BRAINCOMPATH, grant agreement no. 670757); by grants to K.K. from MEXT, Japan (Scientific Research for Innovative Areas, no. 17H06313); by grants to R.W.F. from the Swiss National Science Foundation (project grant no. 310030B-152833/1) and the European Research Council (ERC Advanced Grant MCircuits, grant agreement no. 742576); by the Novartis Research Foundation; by a UZH Forschungskredit and a fellowship from the Boehringer Ingelheim Fonds to P.R.

Author information

Author notes

  1. These authors contributed equally: Fritjof Helmchen, Rainer W. Friedrich.

Authors and Affiliations

  1. Brain Research Institute, University of Zürich, Zurich, Switzerland

    Peter Rupprecht, Stefano Carta, Adrian Hoffmann & Fritjof Helmchen

  2. Friedrich Miescher Institute for Biomedical Research, Basel, Switzerland

    Peter Rupprecht & Rainer W. Friedrich

  3. Department of Neurophysiology, University of Tokyo, Tokyo, Japan

    Mayumi Echizen & Kazuo Kitamura

  4. Department of Anesthesiology, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University, Tokyo, Japan

    Mayumi Echizen

  5. Sainsbury Wellcome Centre for Neural Circuits and Behaviour, University College London, London, United Kingdom

    Antonin Blot & Sonja B. Hofer

  6. Biozentrum, University of Basel, Basel, Switzerland

    Antonin Blot & Sonja B. Hofer

  7. Department of Psychiatry, Yale University School of Medicine, New Haven, CT, USA

    Alex C. Kwan

  8. Division of Neurobiology, Department of Molecular and Cell Biology, Helen Wills Neuroscience Institute, Howard Hughes Medical Institute, University of California, Berkeley, Berkeley CA, USA

    Yang Dan

  9. Department of Neurophysiology, University of Yamanashi, Yamanashi, Japan

    Kazuo Kitamura

  10. University of Basel, Basel, Switzerland

    Rainer W. Friedrich

Authors
  1. Peter Rupprecht
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stefano Carta
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Adrian Hoffmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mayumi Echizen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Antonin Blot
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Alex C. Kwan
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yang Dan
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Sonja B. Hofer
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Kazuo Kitamura
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Fritjof Helmchen
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Rainer W. Friedrich
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

P.R. conceived the project, developed the algorithm, performed ground truth recordings (DSs #4–8), performed all analyses, developed the toolbox and wrote the paper. S.C. performed ground truth recordings (DS #18 and DS #19). A.H. developed the toolbox. M.E. and K.K. (DS #3), A.K. and Y.D. (DS #2, DS #22 and DS #23) and A.B. and S.H. (DSs #24–27) performed and pre-processed ground truth recordings. F.H. supervised ground truth recordings (DS #18 and DS #19) and the development of the toolbox and wrote the paper. R.W.F. supervised ground truth recordings (DSs #4–8) and the development of the algorithm and wrote the paper.

Corresponding authors

Correspondence to Peter Rupprecht, Fritjof Helmchen or Rainer W. Friedrich.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Additional information

Peer review information Nature Neuroscience thanks the anonymous reviewers for their contribution to the peer review of this work.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Linear kernels extracted from all ground truth datasets.

The kernels are optimized such that when the ground truth spike times are linearly convolved with the kernel, the experimentally recorded ΔF/F trace is ideally approximated. In practice, this is achieved using regularized linear deconvolution of calcium traces based on spike times (Methods). Kernels vary both in amplitude and shape across datasets and within datasets. For single neurons, the kernel area (right panels) is only shown if the kernel could be reliably determined, as tested with the variability of the kernel across the recording (Methods). The red arrow in panel (r) indicates an outlier case that is discussed in Extended Data Fig. 4a. m: Mouse, zf: Zebrafish.

Extended Data Fig. 2 Illustration of different baseline noise levels.

ΔF/F ground truth traces were resampled with added noise to reach the target noise level ν. a-d, Noise level illustration from ν = 15 (very high noise level) to ν = 1 (very low noise level). Standardized noise ν is given in units of %·Hz−1/2.

Extended Data Fig. 3 Matching standardized noise level ν of training and test data.

Same as Fig. 2e–g, but with each column (testing level) normalized in order to highlight that the optimal training level for each testing noise level lies close to the diagonal. The correlation (a) was normalized by the maximum of each column, while error and bias metrics have been normalized by the minimum of each column. ν in units of standardized noise, %·Hz−1/2.

Extended Data Fig. 4 Generalization across neurons within a dataset.

The deep network was trained on all neurons of a specific dataset except one, and then tested with the remaining neuron. This analysis shows how the network is able to generalize to new neurons recorded under the same conditions, as a function of the standardized noise level ν in %·Hz−1/2. a-d, Performance of the predictions for 4 selected ground truth datasets in terms of correlation, error and bias as a function of the standardized noise level. Error values were cropped at a value of 5 for display purposes. Single neurons in grey, median across neurons in blue. Grey lines highlighted by arrows indicate outlier neurons with particularly low spike rates (black and green arrows) and particularly distinct calcium response kernel (red arrow, see main text for discussion). e, Correlation, error and biases as a distribution across neurons within each dataset (number of neurons for each dataset as indicated in Table 1). For box plots, the median is indicated by the central line, 25th and 75th percentiles by the box, and maximum/minimum values excluding outliers (points) by the whiskers. All datasets were re-sampled at a frame rate of 7.5 Hz.

Extended Data Fig. 5 Typical artifacts in ground truth recordings.

Calcium trace (ΔF/F), true action potentials (APs), inferred spiking activity (SR) and true ground truth spiking activity (GT). a, The baseline of this recording is unstable, exhibiting irregular bumps (arrowheads). The supervised deep network can learn to ignore these movement artifacts if their dynamics is dissimilar from the sharp onset of calcium transients. Predictions of the deep network are shown in black, ground truth in grey. Green arrowheads indicate movement artifacts that are not associated with high spiking acitivity (correct rejections of artifacts), while black arrowheads indicate movement artifacts that are not recognized as artifacts by the network (false positives). The zoom-in on the right shows an example where a movement artifact is associated with a negligeable spike rate (correct rejection). b, Fluorescence transients without corresponding action potentials are clearly visible (red arrowheads). These are induced by contamination through bright neuropil. The deep network is unable to distinguish this artifact from true calcium transients. c, Negative transients (arrowheads) are generated by standard neuropil decontamination (subtraction of the neuropil surround). The deep network can learn to partially ignore these events (correct rejections). d, Trace showing periodic movement artifacts that do not correspond to action potentials. e, A power spectral density of the recording in (d) exhibits a peak at ca. 1.5 Hz, suggesting breathing of the anaesthetized animal underlying the movement artifact.

Extended Data Fig. 6 Improvement of performance with ground truth dataset size.

The global EXC model (see Fig. 3) was trained as before, but using only a subset of the ground truth data points (x-axis). The performance (correlation) across each dataset was normalized to the performance with 5 million data points (horizontal dashed line). The performance approaches an asymptote at approximately 100,000 data points. A typical single ground truth dataset contains ca. 400,000 data points (median across all datasets; vertical dashed line). This result also indicates that a diverse but smaller training dataset sampled from all ground truth datasets results in better generalization than a larger training dataset from a single ground truth dataset.

Extended Data Fig. 7 Comparison with model-based algorithms, extension of Fig. 4a.

Example predictions from the deep-learning based method (CASCADE) and five model-based algorithms (MLSpike, CaImAn, Peeling, Suite2p, Jewell&Witten) of a ΔF/F recording. Inferred spike rates are in black, ground truth spike rates in orange. r indicates correlation of predictions with ground truth. Events that are not detected across all algorithms (false negatives) are labeled with red arrowheads. Compared to the example in Fig. 4a, the calcium recording here is rather noisy due to the insensitivity of GCaMP to single action potentials in this neuron.

Extended Data Fig. 8 Comparison of CASCADE with model-based algorithms, extension of Fig. 4b.

Comparison of the six algorithms when optimized for a single dataset, showing relative error and relative bias for all neurons, grouped by ground truth dataset.

Extended Data Fig. 9 Performance dependence on temporal precision of predictions.

All algorithms were optimized via the mean squared error to infer spike rates at a specific temporal precision defined by the smoothing of the ground truth (default: Gaussian smoothing with kernel of σ = 200 ms). For all model-based algorithms, the inferred spike traces were shifted in time to optimize the mean squared error. a, Predictions from an example ΔF/F trace (top; dataset #09). Ground truth spike rates are shown in orange, inferred spike rates as black overlay. Correlation values are indicated at the right. The scale bars for ΔF/F and time are the same as in Fig. 4a. b, Highlighted excerpt from (a). Due to the high temporal precisions of the inferred spike rates, small time shifts lead to low performance (clearly visible for the Peeling algorithm in this example). The CaImAn and Suite2p algorithms deconvolve less aggressively, therefore making less dramatic errors. CASCADE and MLSpike perform best for this example neuron, with CASCADE detecting more events than MLSpike. c, Overall performance (correlation) change with temporal precision of predictions (smoothing kernels shown below) on a subset of datasets (datasets #4, #6, #9, #11-14 and #18). As expected, correlation with ground truth decreased with higher temporal resolution of the desired temporal resolution. This decrease was especially prominent for algorithms that, by design, aim at the inference of precise (discrete) spike rates (Peeling, Jewell&Witten). The decrease was less pronounced for CASCADE compared to for example MLSpike. Shaded corridors indicate SEM across n = 8 datasets. All recordings resampled at a noise level of 2 with a frame rate of 7.5 Hz.

Extended Data Fig. 10 Predictions of spiking probabilities and discrete spikes from the Allen Brain Institute Visual Coding dataset.

Predictions were produced with the global EXC model trained at 30 Hz. From dataset ID ‘552195520’, plotting a total of 40 neurons out of 74, approximately 1 minute out of 63.2 minutes of recording for this dataset. Discrete spikes are the most likely fit, generated with an algorithm using Metropolis-Monte Carlo sampling as starting point (see Methods).

Supplementary information

Supplementary Information

Supplementary Notes 1–3, Supplementary Figs. S1–S13 and Supplementary Table 1

Reporting Summary

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rupprecht, P., Carta, S., Hoffmann, A. et al. A database and deep learning toolbox for noise-optimized, generalized spike inference from calcium imaging. Nat Neurosci 24, 1324–1337 (2021). https://doi.org/10.1038/s41593-021-00895-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41593-021-00895-5

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing