Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

FIOLA: an accelerated pipeline for fluorescence imaging online analysis

Nature Methods volume 20, pages 1417–1425 (2023)Cite this article

Subjects

Abstract

Optical microscopy methods such as calcium and voltage imaging enable fast activity readout of large neuronal populations using light. However, the lack of corresponding advances in online algorithms has slowed progress in retrieving information about neural activity during or shortly after an experiment. This gap not only prevents the execution of real-time closed-loop experiments, but also hampers fast experiment–analysis–theory turnover for high-throughput imaging modalities. Reliable extraction of neural activity from fluorescence imaging frames at speeds compatible with indicator dynamics and imaging modalities poses a challenge. We therefore developed FIOLA, a framework for fluorescence imaging online analysis that extracts neuronal activity from calcium and voltage imaging movies at speeds one order of magnitude faster than state-of-the-art methods. FIOLA exploits algorithms optimized for parallel processing on GPUs and CPUs. We demonstrate reliable and scalable performance of FIOLA on both simulated and real calcium and voltage imaging datasets. Finally, we present an online experimental scenario to provide guidance in setting FIOLA parameters and to highlight the trade-offs of our approach.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Online analysis pipeline for fluorescence imaging data.
Fig. 2: Comparison of motion correction shifts generated by FIOLA, CaImAn (NoRMCorre) and Suite2p.
Fig. 3: FIOLA source separation performance.
Fig. 4: FIOLA performance on simulated and real voltage imaging data.
Fig. 5: FIOLA speed performance.
Fig. 6: FIOLA performance and stability.

Similar content being viewed by others

Data availability

Voltage data with simultaneous electrophysiology can be found at https://janelia.figshare.com/collections/Simultaneous_Voltron_1_0_imaging_and_whole-cell_patch-clamp_recordings_of_somatosensory_cortex_layer_1_interneurons_in_vivo/5325254/1 and https://zenodo.org/record/4515768/export/hx#.ZEK_sXaZO5d. Calcium imaging datasets can be found at https://zenodo.org/record/1659149#.ZELAEHaZO5c and https://zenodo.org/record/7779164#.ZELAJ3aZO5c. The HPR dataset can be found at https://dandiarchive.org/dandiset/000054/draft. Source data are provided with this paper.

Code availability

Code for FIOLA can be found in the FIOLA GitHub repository: https://github.com/nel-lab/FIOLA. FIOLA is under GNU General Public License v2.0. A google colab demo that allows users to quickly try the FIOLA pipeline can be found at https://colab.research.google.com/drive/1y98SHqjAqalJ0LXvVF2drjtVdH8tzMa2?usp=sharing. Voltage imaging simulation code can be found at https://github.com/KasparP/PositronSimulations. License information: Creative Commons Attribution 4.0 International.

References

  1. Grienberger, C., Giovannucci, A., Zeiger, W. & Portera-Cailliau, C. Two-photon calcium imaging of neuronal activity. Nature Reviews Methods Primers 2, 67 (2022).

    Article  CAS  Google Scholar 

  2. Peterka, D. S., Takahashi, H. & Yuste, R. Imaging voltage in neurons. Neuron 69, 9–21 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. Sofroniew, N. J., Flickinger, D., King, J. & Svoboda, K. A large field of view two-photon mesoscope with subcellular resolution for in vivo imaging. Elife 5, e14472 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  4. Voleti, V. et al. Real-time volumetric microscopy of in vivo dynamics and large-scale samples with SCAPE 2.0. Nature Methods 16, 1054–1062 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Demas, J. et al. High-speed, cortex-wide volumetric recording of neuroactivity at cellular resolution using light beads microscopy. Nature Methods 18, 1103–1111 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. Zhang, Y. et al. Fast and sensitive GCaMP calcium indicators for imaging neural populations. Nature 615, 884–891 (2023).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Abdelfattah, A. S. et al. Bright and photostable chemigenetic indicators for extended in vivo voltage imaging. Science 365, 699–704 (2019).

    Article  CAS  PubMed  Google Scholar 

  8. Villette, V. et al. Ultrafast two-photon imaging of a high-gain voltage indicator in awake behaving mice. Cell 179, 1590–1608 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  9. Adam, Y. et al. Voltage imaging and optogenetics reveal behaviour-dependent changes in hippocampal dynamics. Nature 569, 413–417 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Carrillo-Reid, L., Han, S., Yang, W., Akrouh, A. & Yuste, R. Controlling visually guided behavior by holographic recalling of cortical ensembles. Cell 178, 447–457 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Dalgleish, H. W. et al. How many neurons are sufficient for perception of cortical activity? Elife 9, e58889 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Robinson, N. T. et al. Targeted activation of hippocampal place cells drives memory-guided spatial behavior. Cell 183, 1586–1599 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Shemesh, O. A. et al. Temporally precise single-cell-resolution optogenetics. Nat. Neurosci. 20, 1796–1806 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Packer, A. M., Russell, L. E., Dalgleish, H. W. & Häusser, M. Simultaneous all-optical manipulation and recording of neural circuit activity with cellular resolution in vivo. Nat. Methods 12, 140–146 (2015).

    Article  CAS  PubMed  Google Scholar 

  15. Dal Maschio, M., Donovan, J. C., Helmbrecht, T. O. & Baier, H. Linking neurons to network function and behavior by two-photon holographic optogenetics and volumetric imaging. Neuron 94, 774–789 (2017).

    Article  CAS  PubMed  Google Scholar 

  16. Athalye, V. R., Carmena, J. M. & Costa, R. M. Neural reinforcement: re-entering and refining neural dynamics leading to desirable outcomes. Curr. Opin. Neurobiol. 60, 145–154 (2020).

    Article  CAS  PubMed  Google Scholar 

  17. Marshel, J. H. et al. Cortical layer-specific critical dynamics triggering perception. Science 365, eaaw5202 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Grosenick, L., Marshel, J. H. & Deisseroth, K. Closed-loop and activity-guided optogenetic control. Neuron 86, 106–139 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. Zhang, Z., Russell, L. E., Packer, A. M., Gauld, O. M. & Häusser, M. Closed-loop all-optical interrogation of neural circuits in vivo. Nat. Methods 15, 1037–1040 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  20. Pnevmatikakis, E. A. Analysis pipelines for calcium imaging data. Curr. Opin. Neurobiol. 55, 15–21 (2019).

    Article  CAS  PubMed  Google Scholar 

  21. Cai, C. et al. VolPy: automated and scalable analysis pipelines for voltage imaging datasets. PLoS Comput. Biol. 17, e1008806 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Pnevmatikakis, E. A. & Giovannucci, A. NoRMCorre: an online algorithm for piecewise rigid motion correction of calcium imaging data. J. Neurosci. Methods 291, 83–94 (2017).

    Article  CAS  PubMed  Google Scholar 

  23. Giovannucci, A. et al. OnACID: online analysis of calcium imaging data in real time. In Guyon, I. et al. (eds) Advances in Neural Information Processing Systems 30, 2381–2391 (Curran Associates, 2017). http://papers.nips.cc/paper/6832-onacid-online-analysis-of-calcium-imaging-data-in-real-time.pdf

  24. Friedrich, J., Zhou, P. & Paninski, L. Fast online deconvolution of calcium imaging data. PLoS Comput. Biol. 13, e1005423 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  25. Bao, Y., Soltanian-Zadeh, S., Farsiu, S. & Gong, Y. Segmentation of neurons from fluorescence calcium recordings beyond real time. Nat. Mach. Intell. 3, 590–600 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  26. Friedrich, J., Giovannucci, A. & Pnevmatikakis, E. A. Online analysis of microendoscopic 1-photon calcium imaging data streams. PLoS Comput. Biol. 17, e1008565 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Giovannucci, A. et al. CaImAn: an open source tool for scalable calcium imaging data analysis. eLife 8, e38173 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  28. Mitani, A. & Komiyama, T. Real-time processing of two-photon calcium imaging data including lateral motion artifact correction. Front. Neuroinform. 12, 98 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  29. Chen, Z., Blair, G. J., Blair, H. T. & Cong, J. BLINK: bit-sparse LSTM inference kernel enabling efficient calcium trace extraction for neurofeedback devices. In Proceedings of the ACM/IEEE International Symposium on Low Power Electronics and Design, 217–222 (2020).

  30. Yang, W., Carrillo-Reid, L., Bando, Y., Peterka, D. S. & Yuste, R. Simultaneous two-photon imaging and two-photon optogenetics of cortical circuits in three dimensions. Elife 7, e32671 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  31. Pnevmatikakis, E. A. et al. Simultaneous denoising, deconvolution, and demixing of calcium imaging data. Neuron 89, 285–299 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  32. Guizar-Sicairos, M., Thurman, S. T. & Fienup, J. R. Efficient subpixel image registration algorithms. Opt. Lett. 33, 156–158 (2008).

    Article  PubMed  Google Scholar 

  33. Abdou, I. E. Practical approach to the registration of multiple frames of video images. In Visual Communications and Image Processing’99, vol. 3653, 371–382 (International Society for Optics and Photonics, 1998).

  34. Abadi, M. et al. TensorFlow: a system for large-scale machine learning. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), 265–283 (2016).

  35. Lawson, C. L. & Hanson, R. J. Solving Least Squares Problems, vol. 15 (Siam, 1995).

  36. Tseng, P. Approximation accuracy, gradient methods, and error bound for structured convex optimization. Math. Program. 125, 263–295 (2010).

    Article  Google Scholar 

  37. Pachitariu, M. et al. Suite2p: beyond 10,000 neurons with standard two-photon microscopy. Preprint at bioRxiv https://doi.org/10.1101/061507 (2017).

  38. Xie, M. E. et al. High-fidelity estimates of spikes and subthreshold waveforms from 1-photon voltage imaging in vivo. Cell Rep. 35, 108954 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  39. Plitt, M. H. & Giocomo, L. M. Experience-dependent contextual codes in the hippocampus. Nat. Neurosci. 24, 705–714 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  40. Zhu, F. et al. A deep learning framework for inference of single-trial neural population dynamics from calcium imaging with subframe temporal resolution. Nat. Neurosci. 25, 1724–1734 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  41. Vladimirov, N. et al. Brain-wide circuit interrogation at the cellular level guided by online analysis of neuronal function. Nat. Methods 15, 1117–1125 (2018).

    Article  CAS  PubMed  Google Scholar 

  42. Che, S. et al. A performance study of general-purpose applications on graphics processors using CUDA. J. Parallel Distrib. Computing 68, 1370–1380 (2008).

    Article  Google Scholar 

  43. Buchanan, E. K. et al. Penalized matrix decomposition for denoising, compression, and improved demixing of functional imaging data. Preprint at bioRxiv https://doi.org/10.1101/334706 (2019).

  44. Deneux, T. et al. Accurate spike estimation from noisy calcium signals for ultrafast three-dimensional imaging of large neuronal populations in vivo. Nat. Commun. 7, 12190 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  45. Liu, Z. et al. Sustained deep-tissue voltage recording using a fast indicator evolved for two-photon microscopy. Cell 185, 3408–3425 (2022).

    Article  Google Scholar 

  46. Smith, J. O. Introduction to Digital Filters: With Audio Applications, vol. 2 (Julius Smith, 2007).

  47. Rozsa, M., Singh, A. & Svoboda, K. Simultaneous Voltron (1.0) Imaging and Whole-cell Patch-clamp Recordings of Somatosensory Cortex Layer 1 Interneurons In Vivo (Janelia Research Campus, 2021). https://janelia.figshare.com/collections/Simultaneous_Voltron_1_0_imaging_and_whole-cell_patch-clamp_recordings_of_somatosensory_cortex_layer_1_interneurons_in_vivo/5325254/1

  48. Kuhn, H. W. The Hungarian method for the assignment problem. Naval Res. Logistics Q 2, 83–97 (1955).

    Article  Google Scholar 

  49. Koay, S. A., Charles, A. S., Thiberge, S. Y., Brody, C. D. & Tank, D. W. Sequential and efficient neural-population coding of complex task information. Neuron 110, 328–349 (2022).

    Article  CAS  PubMed  Google Scholar 

  50. Najafi, F., Giovannucci, A., Wang, S. S.-H. & Medina, J. F. Sensory-driven enhancement of calcium signals in individual Purkinje cell dendrites of awake mice. Cell Rep. 6, 792–798 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  51. Walker, E. Y. et al. Inception loops discover what excites neurons most using deep predictive models. Nat. Neurosci. 22, 2060–2065 (2019).

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

The authors thank A. M. S. Ang from University of Mons for useful discussions and for the formalism of Supplementary Note 3. The authors also thank P. Gunn from the Flatiron Institute, Simons Foundation for valuable suggestions and help with the GPU experiments, K. Podgorski, K. Svoboda and A. Singh from Janelia Research Campus for the ground truth datasets, K. Podgorski for useful discussions, M. Xie and A. Cohen from Harvard University for useful discussions, and J. Tabet and W. Heffley from UNC for assistance with the manuscript. A.G. is supported by the Beckman Young Investigator award.

Author information

Author notes

  1. These authors contributed equally: Changjia Cai, Cynthia Dong.

Authors and Affiliations

  1. Joint Department of Biomedical Engineering UNC/NCSU, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Changjia Cai, Cynthia Dong & Andrea Giovannucci

  2. Department of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Cynthia Dong

  3. Flatiron Institute, Simons Foundation, New York, NY, USA

    Johannes Friedrich & Eftychios A. Pnevmatikakis

  4. Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA, USA

    Marton Rozsa

  5. Neuroscience Center, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Andrea Giovannucci

  6. Closed-Loop Engineering for Advanced Rehabilitation (CLEAR), North Carolina State University, Raleigh, NC, USA

    Andrea Giovannucci

Authors
  1. Changjia Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cynthia Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Johannes Friedrich
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marton Rozsa
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Eftychios A. Pnevmatikakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Andrea Giovannucci
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

C.C. and A.G. designed the study with input from C.D. and E.A.P. Data acquisition was done by M.R. for simultaneous voltage imaging and electrophysiology. C.C., C.D., J.F. and A.G. wrote the code and performed the data analysis. C.C., C.D. and A.G. wrote the manuscript, with feedback from J.F., E.A.P. and M.R.

Corresponding author

Correspondence to Andrea Giovannucci.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Methods thanks Philipp Berens and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available. Primary Handling Editor: Nina Vogt, in collaboration with the Nature Methods team.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Table 1 Calcium and voltage datasets detail
Full size table

Supplementary information

Supplementary Information

Supplementary Notes 1–7, Supplementary Figs. 1–17 and Supplementary Tables 1–14.

Reporting Summary

Peer Review File

Source data

Source Data Fig. 2

Result of motion correction.

Source Data Fig. 3

Result of source separation.

Source Data Fig. 4

Result of spike extraction for voltage imaging.

Source Data Fig. 5

Result of timing performance.

Source Data Fig. 6

Result of simulated experiments.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cai, C., Dong, C., Friedrich, J. et al. FIOLA: an accelerated pipeline for fluorescence imaging online analysis. Nat Methods 20, 1417–1425 (2023). https://doi.org/10.1038/s41592-023-01964-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41592-023-01964-2

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics