Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Systematic errors in connectivity inferred from activity in strongly recurrent networks

Nature Neuroscience volume 23, pages 1286–1296 (2020)Cite this article

Subjects

Abstract

Understanding the mechanisms of neural computation and learning will require knowledge of the underlying circuitry. Because it is difficult to directly measure the wiring diagrams of neural circuits, there has long been an interest in estimating them algorithmically from multicell activity recordings. We show that even sophisticated methods, applied to unlimited data from every cell in the circuit, are biased toward inferring connections between unconnected but highly correlated neurons. This failure to ‘explain away’ connections occurs when there is a mismatch between the true network dynamics and the model used for inference, which is inevitable when modeling the real world. Thus, causal inference suffers when variables are highly correlated, and activity-based estimates of connectivity should be treated with special caution in strongly connected networks. Finally, performing inference on the activity of circuits pushed far out of equilibrium by a simple low-dimensional suppressive drive might ameliorate inference bias.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Structure and dynamics of the generative network.
Fig. 2: Quality of circuit inference using 108 spikes from a fully observed ring network, as a function of weight strength.
Fig. 3: Inference quality as a function of data volume.
Fig. 4: Results extend to different inference methods.
Fig. 5: Results extend to different networks.
Fig. 6: Circuit-to-activity map is inherently less invertible when correlations are strong.
Fig. 7: Inference bias due to unobserved neurons is exacerbated at strong weights.
Fig. 8: Sampling non-equilibrium data mitigates inference bias.

Similar content being viewed by others

Data and code availability

The data and code that support the findings of this study are available from the corresponding author upon reasonable request.

References

  1. Burns, R., et al. The Open Connectome Project data cluster: scalable analysis and vision for high-throughput neuroscience. In Proceedings of the 25th International Conference on Scientific and Statistical Database Management, 1–11 (Association for Computing Machinery, 2013).

  2. Xu, C. S. et al. A connectome of the adult Drosophila central brain. Preprint at bioRxiv https://doi.org/10.1101/2020.01.21.911859 (2020).

  3. Helmstaedter, M. et al. Connectomic reconstruction of the inner plexiform layer in the mouse retina. Nature 500, 168 (2013).

    CAS  PubMed  Google Scholar 

  4. Takemura, S.-Y. et al. A visual motion detection circuit suggested by Drosophila connectomics. Nature 500, 175–181 (2013).

    CAS  PubMed  PubMed Central  Google Scholar 

  5. Lee, W.-C. A. et al. Anatomy and function of an excitatory network in the visual cortex. Nature 532, 370–374 (2016).

    CAS  PubMed  PubMed Central  Google Scholar 

  6. Pillow, J. W. et al. Spatio-temporal correlations and visual signalling in a complete neuronal population. Nature 454, 995–999 (2008).

    CAS  PubMed  PubMed Central  Google Scholar 

  7. Schneidman, E., Berry, M. J., Segev, R. & Bialek, W. Weak pairwise correlations imply strongly correlated network states in a neural population. Nature 440, 1007–1012 (2006).

    CAS  PubMed  PubMed Central  Google Scholar 

  8. Friston, K. J. Functional and effective connectivity: a review. Brain Connect. 1, 13–36 (2011).

    PubMed  Google Scholar 

  9. Pakman, A., Huggins, J., Smith, C. & Paninski, L. Fast state-space methods for inferring dendritic synaptic connectivity. J. Comput. Neurosci. 36, 415–443 (2014).

    PubMed  Google Scholar 

  10. Ravikumar, P. et al. High-dimensional Ising model selection using l1-regularized logistic regression. Ann. Stat. 38, 1287–1319 (2010).

  11. Nelder, J. A. & Wedderburn, R. W. M. Generalized linear models. J. R. Stat. Soc. Series A 135, 370–384 (1972).

    Google Scholar 

  12. Truccolo, W., Eden, U. T., Fellows, M. R., Donoghue, J. P. & Brown, E. N. A point process framework for relating neural spiking activity to spiking history, neural ensemble, and extrinsic covariate effects. J. Neurophysiol. 93, 1074–1089 (2005).

    PubMed  Google Scholar 

  13. Shlens, J. et al. The structure of multi-neuron firing patterns in primate retina. J. Neurosci. 26, 8254–8266 (2006).

    CAS  PubMed  PubMed Central  Google Scholar 

  14. Pillow, J. W. and Latham, P. E. Neural characterization in partially observed populations of spiking neurons. In Advances in Neural Information Processing Systems 1161–1168 (NIPS, 2008).

  15. Mishchencko, Y., Vogelstein, J. T. & Paninski, L. A Bayesian approach for inferring neuronal connectivity from calcium fluorescent imaging data. Ann. Appl. Stat. 5, 1229–1261 (2011).

    Google Scholar 

  16. Ramirez, A. D. & Paninski, L. Fast inference in generalized linear models via expected log-likelihoods. J. Comput. Neurosci. 36, 215–234 (2014).

    PubMed  Google Scholar 

  17. Nykamp, D. Q. Revealing pairwise coupling in linear–nonlinear networks. SIAM J. Appl. Math. 65, 2005–2032 (2005).

    Google Scholar 

  18. Soudry, D. et al. Efficient “shotgun” inference of neural connectivity from highly sub-sampled activity data. PLoS Comput. Biol. 11, e1004464 (2015).

    PubMed  PubMed Central  Google Scholar 

  19. Kulkarni, J. E. & Paninski, L. Common-input models for multiple neural spike-train data. Network 18, 375–407 (2007).

    PubMed  Google Scholar 

  20. Vidne, M. et al. Modeling the impact of common noise inputs on the network activity of retinal ganglion cells. J. Comput. Neurosci. 33, 97–121 (2012).

    PubMed  Google Scholar 

  21. Brinkman, B. A. W., Rieke, F., Shea-Brown, E. & Buice, M. A. Predicting how and when hidden neurons skew measured synaptic interactions. PLoS Comput. Biol. 14, e1006490 (2018).

    PubMed  PubMed Central  Google Scholar 

  22. Dunn, B. & Battistin, C. The appropriateness of ignorance in the inverse kinetic ising model. J. Phys. A Math. Theor. 50, 124002 (2017).

  23. Mehler, D. M. A. and Kording, K. P. The lure of causal statements: rampant mis-inference of causality in estimated connectivity. Preprint at arXiv https://arxiv.org/abs/1812.03363 (2018).

  24. Burak, Y. & Fiete, I. R. Accurate path integration in continuous attractor network models of grid cells. PLoS Comput. Biol. 5, e1000291 (2009).

    PubMed  PubMed Central  Google Scholar 

  25. Gutnisky, D. A. & Dragoi, V. Adaptive coding of visual information in neural populations. Nature 452, 220–224 (2008).

    CAS  PubMed  Google Scholar 

  26. Poort, J. & Roelfsema, P. R. Noise correlations have little influence on the coding of selective attention in area V1. Cereb. Cortex 19, 543–553 (2009).

    PubMed  Google Scholar 

  27. Samonds, J. M., Potetz, B. R. & Lee, T. S. Cooperative and competitive interactions facilitate stereo computations in macaque primary visual cortex. J. Neurosci. 29, 15780–15795 (2009).

    CAS  PubMed  PubMed Central  Google Scholar 

  28. Cohen, M. R. & Maunsell, J. H. R. Attention improves performance primarily by reducing interneuronal correlations. Nat. Neurosci. 12, 1594 (2009).

    CAS  PubMed  PubMed Central  Google Scholar 

  29. Mitchell, J. F., Sundberg, K. A. & Reynolds, J. H. Spatial attention decorrelates intrinsic activity fluctuations in macaque area v4. Neuron 63, 879–888 (2009).

    CAS  PubMed  PubMed Central  Google Scholar 

  30. Huang, X. & Lisberger, S. G. Noise correlations in cortical area MT and their potential impact on trial-by-trial variation in the direction and speed of smooth-pursuit eye movements. J. Neurophysiol. 101, 3012–3030 (2009).

    PubMed  PubMed Central  Google Scholar 

  31. Cohen, M. R. & Newsome, W. T. Context-dependent changes in functional circuitry in visual area MT. Neuron 60, 162–173 (2008).

    CAS  PubMed  PubMed Central  Google Scholar 

  32. Zohary, E., Shadlen, M. N. & Newsome, W. T. Correlated neuronal discharge rate and its implications for psychophysical performance. Nature 370, 140 (1994).

    CAS  PubMed  Google Scholar 

  33. Bair, W., Zohary, E. & Newsome, W. T. Correlated firing in macaque visual area MT: time scales and relationship to behavior. J. Neurosci. 21, 1676–1697 (2001).

    CAS  PubMed  PubMed Central  Google Scholar 

  34. Kemere, C., Carr, M. F., Karlsson, M. P. & Frank, L. M. Rapid and continuous modulation of hippocampal network state during exploration of new places. PLoS ONE 8, e73114 (2013).

    CAS  PubMed  PubMed Central  Google Scholar 

  35. Burak, Y. & Fiete, I. R. Fundamental limits on persistent activity in networks of noisy neurons. Proc. Natl Acad. Sci. USA 109, 17645–17650 (2012).

    CAS  PubMed  Google Scholar 

  36. Mastromatteo, I. & Marsili, M. On the criticality of inferred models. J. Stat. Mech. 2011, P10012 (2011).

    Google Scholar 

  37. Barahona, F. On the computational complexity of Ising spin glass models. J. Phys. A Math. Gen. 15, 3241 (1982).

  38. Sohl-Dickstein, J., Battaglino, P. B. & DeWeese, M. R. New method for parameter estimation in probabilistic models: minimum probability flow. Phys. Rev. Lett. 107, 220601 (2011).

    PubMed  Google Scholar 

  39. Thouless, D. J., Anderson, P. W. & Palmer, R. G. Solution of ‘Solvable model of a spin glass’. Philos. Mag. 35, 593–601 (1977).

  40. Roudi, Y., Tyrcha, J. & Hertz, J. Ising model for neural data: model quality and approximate methods for extracting functional connectivity. Phys. Rev. E 79, 051915 (2009).

    Google Scholar 

  41. Sessak, V. & Monasson, R. Small-correlation expansions for the inverse Ising problem. J. Phys. A Math. Theor. 42, 055001 (2009).

    Google Scholar 

  42. Lee, S.-I., Lee, H., Abbeel, P. & Ng, A. Y. Efficient L1 regularized logistic regression. In The Twenty-First National Conference on Artificial Intelligence 6, 401–408 (Association for the Advancement of Artificial Intelligence, 2006).

  43. Sompolinsky, H., Crisanti, A. & Sommers, H.-J. Chaos in random neural networks. Phys. Rev. Lett. 61, 259 (1988).

    CAS  PubMed  Google Scholar 

  44. Widrow, B. & Hoff, M. E. Adaptive switching circuits. Technical Report No. 1553-1. https://apps.dtic.mil/dtic/tr/fulltext/u2/241531.pdf (Office of Naval Research, 1960).

  45. Williams, R. J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 8, 229–256 (1992).

    Google Scholar 

  46. Fiete, I. R. & Seung, H. S. Gradient learning in spiking neural networks by dynamic perturbation of conductances. Phys. Rev. Lett. 97, 048104 (2006).

    PubMed  Google Scholar 

  47. Shababo, B., Paige, B., Pakman, A. & Paninski, L. Bayesian inference and online experimental design for mapping neural microcircuits. In Advances in Neural Information Processing Systems 1304–1312 (NIPS, 2013).

  48. Aronov, D., Veit, L., Goldberg, J. H. & Fee, M. S. Two distinct modes of forebrain circuit dynamics underlie temporal patterning in the vocalizations of young songbirds. J. Neurosci. 31, 16353–16368 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  49. Casadiego, J., Nitzan, M., Hallerberg, S. & Timme, M. Model-free inference of direct network interactions from nonlinear collective dynamics. Nat. Commun. 8, 2192 (2017).

    PubMed  PubMed Central  Google Scholar 

  50. Widloski, J., Marder, M. P. & Fiete, I. R. Inferring circuit mechanisms from sparse neural recording and global perturbation in grid cells. eLife 7, e33503 (2018).

    PubMed  PubMed Central  Google Scholar 

  51. West, M., Harrison, P. J. & Helio, S. M. Dynamic generalized linear models and bayesian forecasting. J. Am. Stat. Assoc. 80, 73–83 (1985).

    Google Scholar 

  52. Federico, R.-T. The Bethe approximation for solving the inverse Ising problem: a comparison with other inference methods. J. Stat. Mech. 2012, P08015 (2012).

    Google Scholar 

  53. Nemenman, I., Shafee, F. & Bialek, W. Entropy and inference, revisited. In Advances in Neural Information Processing Systems 471–478 (NIPS, 2002).

  54. Paninski, L. Estimating entropy on m bins given fewer than m samples. IEEE Trans. Inf. Theory 50, 2200–2203 (2004).

    Google Scholar 

  55. Jeffreys, H. An invariant form for the prior probability in estimation problems. Proc. Roy. Soc. Lond. Math. Phys. Sci. 186, 453–461 (1946).

    CAS  Google Scholar 

Download references

Acknowledgements

We thank R. Chaudhuri, B. Gerçek, I. Kanitscheider, T. Taillefumier, D. Schwab, M. Bethge and P. Dayan for helpful discussions. This project was funded in part by the ONR, the Howard Hughes Medical Institute through the Faculty Scholars Program and the Simons Collaboration on the Global Brain through the Simons Foundation.

Author information

Authors and Affiliations

  1. Department of Physics, The University of Texas, Austin, TX, USA

    Abhranil Das & Ila R. Fiete

  2. Center for Learning and Memory, The University of Texas, Austin, TX, USA

    Abhranil Das & Ila R. Fiete

  3. Department of Brain and Cognitive Sciences and McGovern Institute, Massachusetts Institute of Technology, Cambridge, MA, USA

    Ila R. Fiete

Authors
  1. Abhranil Das
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ila R. Fiete
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ila R. Fiete.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Inference with matched models.

a, Using an Ising model for both generation and inference. Top: superposed inferred weights from each node to the rest (line marks zero). Bottom: pattern coherence, and inference error with different data volumes, against weight strength. b, Squared total, variance and bias errors against data volume at weak and strong weights. c, Using a generalized linear model with an exponential nonlinearity for both generation and inference (see Methods). Top: pattern coherence, and inference error with different data volumes, against weight strength. Bottom: Superposed inferred weights from each node to the rest.

Extended Data Fig. 2 Tuning firing thresholds of different generative networks to control the average network inter-spike-interval as recurrent weight strength is varied.

a, Firing thresholds to hold the average ISI of the ring network with local and non-local synapses, and the local ring network with rectifying LNP dynamics, at 16.0 ± 0.1 ms, and that of the random, fully-connected, symmetric network at 34.0 ± 0.1 ms. b, Firing rate coefficients α (see eq. (7)) to hold the average ISI of the ring network with GLM dynamics (logarithmic link function) at 16.0 ± 0.1 ms. c, Firing thresholds to hold the average ISI of the sparse, non-symmetric random network at 16.0 ± 0.1 ms.

Extended Data Fig. 3 Binning affects noise correlations and inference.

a, Noise correlations between neuron pairs (top: full matrix, bottom: superposed rows) for binned vs unbinned spikes from the ring network. The optimal bin-width groups causally related spikes together, and noise correlations at an intermediate r then reflect underlying weights. b, Inference error using inverse Ising with MPF on spike data binned at different widths. c, Fraction of spikes discarded when binarizing binned spike data for Inverse Ising inference.

Extended Data Fig. 4 Distribution of inference errors of individual weights in the ring circuit, at different recurrent weight strengths.

a, Histograms of the inference errors (relative to the length of the ground-truth weight vector). At weak weights, errors are random and normally distributed. As the weight increases, errors first shrink as noise weakens and SNR grows, then they become increasingly non-normal due to bias. b, Negentropy of the error distribution (see Methods) against weight strength.

Extended Data Fig. 5 Power-law decay of variance error of inferring the ring circuit, with increasing data volume.

a, The fitted exponents α of \({\Delta }_{v}^{2} \sim {D}^{\alpha }\) when using a generalized linear model for inference. Error-bands are 95% confidence intervals using 20 data points. The theoretical exponent is -1. b, The exponent α of the decay of total inference error Δ2 ~ Dα when using the Ising model for both data generation and inference. Error-bands are 95% confidence intervals using 20 data points. Here inference error is almost entirely due to variance, thus decays as the power law.

Extended Data Fig. 6 Circuit inference using logistic regression is not improved by l1 regularization.

a, Example ring network weight profiles inferred using logistic regression with zero, optimal and excessive regularization penalties. When weights are weak, regularization reduces some noise and marginally improves inference. At high weights, regularization suppresses both the spurious off-diagonal stripes and the true coupling shape, so is not helpful. b, Inference error vs weight strength using logistic regression with and without l1 regularization. c, Optimal l1 penalties (that produce the lowest inference errors) at each weight. Regularization improves inference in the strong and weak weight regimes, but barely.

Extended Data Fig. 7 Circuit inference using neural CCGs.

(a-d) Inferring the strong weight (r = 0.025) ring circuit using short-lag peaks in neural CCGs. a, Pearson’s cross-correlation of a connected and an unconnected neuron pair. b, Left: matrix of absolute lags of the CCG peaks. This partly reveals the circuit: directly connected neurons exhibit short lags. Right: binary matrix connecting neuron pairs with short lags. c, Top: CCG-based weight matrix: weight is set to the peak cross-correlation if the neurons are connected (lag < τ), zero otherwise. Bottom: avg. weight profile. GLM fares better. d, Combining CCG and GLM. Top: matrix of GLM-inferred weights if neuron pairs have short CCG lag, zero otherwise. Bottom: avg. weight profile. Relative to pure GLM (Δ = 0.23), this method (Δ = 0.32) removes some biases but introduces others. (e-j) Spike CCGs from the sparse, non-symmetric, strong weight (rRSA = 0.1) random network. e, Unconnected pair. CCG has no sharp features around 0, indicating no direct connection. f, Unconnected pair. Broad symmetric peak, indicating multiple indirect influences through mutual connections, is discounted. g, Connected pair. Sharp asymmetric dip at 0 reveals direct (inhibitory) connection. h, Unconnected pair. CCG is indistinguishable from the previous, and passes the criterion. i, Connected pair, but no CCG features. j, Connected pair, but broad symmetric peak is discounted.

Extended Data Fig. 8 Results of inference using a GLM on data generated by a linear-nonlinear-Poisson model with a rectifying linear response (see Methods).

a, Pattern coherence against weight strength for the generative LNP network. b, Inferred weight matrices (top) and superposition of rows (bottom, line marks zero), at several weight strengths. c, Inference error and bias fraction against weight strength. Optimal inference is at the point of pattern onset.

Extended Data Fig. 9 Activity and inference in the random balanced networks that are fully or sparsely connected.

a, Top: waterfall plots of neural fields of the full network at weak weights (when activity decays) and strong weights (when activity is chaotic), in response to a brief uniform feed-forward pulse. Bottom: corresponding inferences. (b-d) Inference on the sparse network. b, Left: true and inferred weights (using logistic regression) for the network with no noisy drive, at rRSB = 0.6. Some zero weights are inferred to be non-zero, while some nonzero weights are underestimated. Right: true and inferred weights (using only logistic regression, and augmented with CCG information) for the same condition, but when the network is noise-driven. c, Inference error (using the two methods) vs recurrent weight strength, and data volume, on the noise-driven network. d, Example CCG (top) and its time-derivative (bottom) of a connected neuron pair in the noise-driven network.

Extended Data Fig. 10 Entropy of spiking activity states of neural circuits.

a, Entropy of the distributions of 22-neuron spike sub-states from the true local ring circuit W and the non-local circuit \({\bf{W}}^{\prime}\). b, Entropies of the spike sub-states of the two circuits computed with different data fractions across weight strengths. At all weights, the computed entropies converge as the data approaches the total volume. c: Example slice of plot b at the weakest weights, where entropy convergence takes the longest.

Supplementary information

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Das, A., Fiete, I.R. Systematic errors in connectivity inferred from activity in strongly recurrent networks. Nat Neurosci 23, 1286–1296 (2020). https://doi.org/10.1038/s41593-020-0699-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41593-020-0699-2

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing