Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Systematic errors in connectivity inferred from activity in strongly recurrent networks

Abstract

Understanding the mechanisms of neural computation and learning will require knowledge of the underlying circuitry. Because it is difficult to directly measure the wiring diagrams of neural circuits, there has long been an interest in estimating them algorithmically from multicell activity recordings. We show that even sophisticated methods, applied to unlimited data from every cell in the circuit, are biased toward inferring connections between unconnected but highly correlated neurons. This failure to ‘explain away’ connections occurs when there is a mismatch between the true network dynamics and the model used for inference, which is inevitable when modeling the real world. Thus, causal inference suffers when variables are highly correlated, and activity-based estimates of connectivity should be treated with special caution in strongly connected networks. Finally, performing inference on the activity of circuits pushed far out of equilibrium by a simple low-dimensional suppressive drive might ameliorate inference bias.

Access options

from\$8.99

All prices are NET prices.

Data and code availability

The data and code that support the findings of this study are available from the corresponding author upon reasonable request.

References

1. 1.

Burns, R., et al. The Open Connectome Project data cluster: scalable analysis and vision for high-throughput neuroscience. In Proceedings of the 25th International Conference on Scientific and Statistical Database Management, 1–11 (Association for Computing Machinery, 2013).

2. 2.

Xu, C. S. et al. A connectome of the adult Drosophila central brain. Preprint at bioRxiv https://doi.org/10.1101/2020.01.21.911859 (2020).

3. 3.

Helmstaedter, M. et al. Connectomic reconstruction of the inner plexiform layer in the mouse retina. Nature 500, 168 (2013).

4. 4.

Takemura, S.-Y. et al. A visual motion detection circuit suggested by Drosophila connectomics. Nature 500, 175–181 (2013).

5. 5.

Lee, W.-C. A. et al. Anatomy and function of an excitatory network in the visual cortex. Nature 532, 370–374 (2016).

6. 6.

Pillow, J. W. et al. Spatio-temporal correlations and visual signalling in a complete neuronal population. Nature 454, 995–999 (2008).

7. 7.

Schneidman, E., Berry, M. J., Segev, R. & Bialek, W. Weak pairwise correlations imply strongly correlated network states in a neural population. Nature 440, 1007–1012 (2006).

8. 8.

Friston, K. J. Functional and effective connectivity: a review. Brain Connect. 1, 13–36 (2011).

9. 9.

Pakman, A., Huggins, J., Smith, C. & Paninski, L. Fast state-space methods for inferring dendritic synaptic connectivity. J. Comput. Neurosci. 36, 415–443 (2014).

10. 10.

Ravikumar, P. et al. High-dimensional Ising model selection using l1-regularized logistic regression. Ann. Stat. 38, 1287–1319 (2010).

11. 11.

Nelder, J. A. & Wedderburn, R. W. M. Generalized linear models. J. R. Stat. Soc. Series A 135, 370–384 (1972).

12. 12.

Truccolo, W., Eden, U. T., Fellows, M. R., Donoghue, J. P. & Brown, E. N. A point process framework for relating neural spiking activity to spiking history, neural ensemble, and extrinsic covariate effects. J. Neurophysiol. 93, 1074–1089 (2005).

13. 13.

Shlens, J. et al. The structure of multi-neuron firing patterns in primate retina. J. Neurosci. 26, 8254–8266 (2006).

14. 14.

Pillow, J. W. and Latham, P. E. Neural characterization in partially observed populations of spiking neurons. In Advances in Neural Information Processing Systems 1161–1168 (NIPS, 2008).

15. 15.

Mishchencko, Y., Vogelstein, J. T. & Paninski, L. A Bayesian approach for inferring neuronal connectivity from calcium fluorescent imaging data. Ann. Appl. Stat. 5, 1229–1261 (2011).

16. 16.

Ramirez, A. D. & Paninski, L. Fast inference in generalized linear models via expected log-likelihoods. J. Comput. Neurosci. 36, 215–234 (2014).

17. 17.

Nykamp, D. Q. Revealing pairwise coupling in linear–nonlinear networks. SIAM J. Appl. Math. 65, 2005–2032 (2005).

18. 18.

Soudry, D. et al. Efficient “shotgun” inference of neural connectivity from highly sub-sampled activity data. PLoS Comput. Biol. 11, e1004464 (2015).

19. 19.

Kulkarni, J. E. & Paninski, L. Common-input models for multiple neural spike-train data. Network 18, 375–407 (2007).

20. 20.

Vidne, M. et al. Modeling the impact of common noise inputs on the network activity of retinal ganglion cells. J. Comput. Neurosci. 33, 97–121 (2012).

21. 21.

Brinkman, B. A. W., Rieke, F., Shea-Brown, E. & Buice, M. A. Predicting how and when hidden neurons skew measured synaptic interactions. PLoS Comput. Biol. 14, e1006490 (2018).

22. 22.

Dunn, B. & Battistin, C. The appropriateness of ignorance in the inverse kinetic ising model. J. Phys. A Math. Theor. 50, 124002 (2017).

23. 23.

Mehler, D. M. A. and Kording, K. P. The lure of causal statements: rampant mis-inference of causality in estimated connectivity. Preprint at arXiv https://arxiv.org/abs/1812.03363 (2018).

24. 24.

Burak, Y. & Fiete, I. R. Accurate path integration in continuous attractor network models of grid cells. PLoS Comput. Biol. 5, e1000291 (2009).

25. 25.

Gutnisky, D. A. & Dragoi, V. Adaptive coding of visual information in neural populations. Nature 452, 220–224 (2008).

26. 26.

Poort, J. & Roelfsema, P. R. Noise correlations have little influence on the coding of selective attention in area V1. Cereb. Cortex 19, 543–553 (2009).

27. 27.

Samonds, J. M., Potetz, B. R. & Lee, T. S. Cooperative and competitive interactions facilitate stereo computations in macaque primary visual cortex. J. Neurosci. 29, 15780–15795 (2009).

28. 28.

Cohen, M. R. & Maunsell, J. H. R. Attention improves performance primarily by reducing interneuronal correlations. Nat. Neurosci. 12, 1594 (2009).

29. 29.

Mitchell, J. F., Sundberg, K. A. & Reynolds, J. H. Spatial attention decorrelates intrinsic activity fluctuations in macaque area v4. Neuron 63, 879–888 (2009).

30. 30.

Huang, X. & Lisberger, S. G. Noise correlations in cortical area MT and their potential impact on trial-by-trial variation in the direction and speed of smooth-pursuit eye movements. J. Neurophysiol. 101, 3012–3030 (2009).

31. 31.

Cohen, M. R. & Newsome, W. T. Context-dependent changes in functional circuitry in visual area MT. Neuron 60, 162–173 (2008).

32. 32.

Zohary, E., Shadlen, M. N. & Newsome, W. T. Correlated neuronal discharge rate and its implications for psychophysical performance. Nature 370, 140 (1994).

33. 33.

Bair, W., Zohary, E. & Newsome, W. T. Correlated firing in macaque visual area MT: time scales and relationship to behavior. J. Neurosci. 21, 1676–1697 (2001).

34. 34.

Kemere, C., Carr, M. F., Karlsson, M. P. & Frank, L. M. Rapid and continuous modulation of hippocampal network state during exploration of new places. PLoS ONE 8, e73114 (2013).

35. 35.

Burak, Y. & Fiete, I. R. Fundamental limits on persistent activity in networks of noisy neurons. Proc. Natl Acad. Sci. USA 109, 17645–17650 (2012).

36. 36.

Mastromatteo, I. & Marsili, M. On the criticality of inferred models. J. Stat. Mech. 2011, P10012 (2011).

37. 37.

Barahona, F. On the computational complexity of Ising spin glass models. J. Phys. A Math. Gen. 15, 3241 (1982).

38. 38.

Sohl-Dickstein, J., Battaglino, P. B. & DeWeese, M. R. New method for parameter estimation in probabilistic models: minimum probability flow. Phys. Rev. Lett. 107, 220601 (2011).

39. 39.

Thouless, D. J., Anderson, P. W. & Palmer, R. G. Solution of ‘Solvable model of a spin glass’. Philos. Mag. 35, 593–601 (1977).

40. 40.

Roudi, Y., Tyrcha, J. & Hertz, J. Ising model for neural data: model quality and approximate methods for extracting functional connectivity. Phys. Rev. E 79, 051915 (2009).

41. 41.

Sessak, V. & Monasson, R. Small-correlation expansions for the inverse Ising problem. J. Phys. A Math. Theor. 42, 055001 (2009).

42. 42.

Lee, S.-I., Lee, H., Abbeel, P. & Ng, A. Y. Efficient L1 regularized logistic regression. In The Twenty-First National Conference on Artificial Intelligence 6, 401–408 (Association for the Advancement of Artificial Intelligence, 2006).

43. 43.

Sompolinsky, H., Crisanti, A. & Sommers, H.-J. Chaos in random neural networks. Phys. Rev. Lett. 61, 259 (1988).

44. 44.

Widrow, B. & Hoff, M. E. Adaptive switching circuits. Technical Report No. 1553-1. https://apps.dtic.mil/dtic/tr/fulltext/u2/241531.pdf (Office of Naval Research, 1960).

45. 45.

Williams, R. J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 8, 229–256 (1992).

46. 46.

Fiete, I. R. & Seung, H. S. Gradient learning in spiking neural networks by dynamic perturbation of conductances. Phys. Rev. Lett. 97, 048104 (2006).

47. 47.

Shababo, B., Paige, B., Pakman, A. & Paninski, L. Bayesian inference and online experimental design for mapping neural microcircuits. In Advances in Neural Information Processing Systems 1304–1312 (NIPS, 2013).

48. 48.

Aronov, D., Veit, L., Goldberg, J. H. & Fee, M. S. Two distinct modes of forebrain circuit dynamics underlie temporal patterning in the vocalizations of young songbirds. J. Neurosci. 31, 16353–16368 (2011).

49. 49.

Casadiego, J., Nitzan, M., Hallerberg, S. & Timme, M. Model-free inference of direct network interactions from nonlinear collective dynamics. Nat. Commun. 8, 2192 (2017).

50. 50.

Widloski, J., Marder, M. P. & Fiete, I. R. Inferring circuit mechanisms from sparse neural recording and global perturbation in grid cells. eLife 7, e33503 (2018).

51. 51.

West, M., Harrison, P. J. & Helio, S. M. Dynamic generalized linear models and bayesian forecasting. J. Am. Stat. Assoc. 80, 73–83 (1985).

52. 52.

Federico, R.-T. The Bethe approximation for solving the inverse Ising problem: a comparison with other inference methods. J. Stat. Mech. 2012, P08015 (2012).

53. 53.

Nemenman, I., Shafee, F. & Bialek, W. Entropy and inference, revisited. In Advances in Neural Information Processing Systems 471–478 (NIPS, 2002).

54. 54.

Paninski, L. Estimating entropy on m bins given fewer than m samples. IEEE Trans. Inf. Theory 50, 2200–2203 (2004).

55. 55.

Jeffreys, H. An invariant form for the prior probability in estimation problems. Proc. Roy. Soc. Lond. Math. Phys. Sci. 186, 453–461 (1946).

Acknowledgements

We thank R. Chaudhuri, B. Gerçek, I. Kanitscheider, T. Taillefumier, D. Schwab, M. Bethge and P. Dayan for helpful discussions. This project was funded in part by the ONR, the Howard Hughes Medical Institute through the Faculty Scholars Program and the Simons Collaboration on the Global Brain through the Simons Foundation.

Author information

Authors

Corresponding author

Correspondence to Ila R. Fiete.

Ethics declarations

Competing interests

The authors declare no competing interests.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Inference with matched models.

a, Using an Ising model for both generation and inference. Top: superposed inferred weights from each node to the rest (line marks zero). Bottom: pattern coherence, and inference error with different data volumes, against weight strength. b, Squared total, variance and bias errors against data volume at weak and strong weights. c, Using a generalized linear model with an exponential nonlinearity for both generation and inference (see Methods). Top: pattern coherence, and inference error with different data volumes, against weight strength. Bottom: Superposed inferred weights from each node to the rest.

Extended Data Fig. 2 Tuning firing thresholds of different generative networks to control the average network inter-spike-interval as recurrent weight strength is varied.

a, Firing thresholds to hold the average ISI of the ring network with local and non-local synapses, and the local ring network with rectifying LNP dynamics, at 16.0 ± 0.1 ms, and that of the random, fully-connected, symmetric network at 34.0 ± 0.1 ms. b, Firing rate coefficients α (see eq. (7)) to hold the average ISI of the ring network with GLM dynamics (logarithmic link function) at 16.0 ± 0.1 ms. c, Firing thresholds to hold the average ISI of the sparse, non-symmetric random network at 16.0 ± 0.1 ms.

Extended Data Fig. 3 Binning affects noise correlations and inference.

a, Noise correlations between neuron pairs (top: full matrix, bottom: superposed rows) for binned vs unbinned spikes from the ring network. The optimal bin-width groups causally related spikes together, and noise correlations at an intermediate r then reflect underlying weights. b, Inference error using inverse Ising with MPF on spike data binned at different widths. c, Fraction of spikes discarded when binarizing binned spike data for Inverse Ising inference.

Extended Data Fig. 4 Distribution of inference errors of individual weights in the ring circuit, at different recurrent weight strengths.

a, Histograms of the inference errors (relative to the length of the ground-truth weight vector). At weak weights, errors are random and normally distributed. As the weight increases, errors first shrink as noise weakens and SNR grows, then they become increasingly non-normal due to bias. b, Negentropy of the error distribution (see Methods) against weight strength.

Extended Data Fig. 5 Power-law decay of variance error of inferring the ring circuit, with increasing data volume.

a, The fitted exponents α of $${\Delta }_{v}^{2} \sim {D}^{\alpha }$$ when using a generalized linear model for inference. Error-bands are 95% confidence intervals using 20 data points. The theoretical exponent is -1. b, The exponent α of the decay of total inference error Δ2 ~ Dα when using the Ising model for both data generation and inference. Error-bands are 95% confidence intervals using 20 data points. Here inference error is almost entirely due to variance, thus decays as the power law.

Extended Data Fig. 6 Circuit inference using logistic regression is not improved by l1 regularization.

a, Example ring network weight profiles inferred using logistic regression with zero, optimal and excessive regularization penalties. When weights are weak, regularization reduces some noise and marginally improves inference. At high weights, regularization suppresses both the spurious off-diagonal stripes and the true coupling shape, so is not helpful. b, Inference error vs weight strength using logistic regression with and without l1 regularization. c, Optimal l1 penalties (that produce the lowest inference errors) at each weight. Regularization improves inference in the strong and weak weight regimes, but barely.

Extended Data Fig. 7 Circuit inference using neural CCGs.

(a-d) Inferring the strong weight (r = 0.025) ring circuit using short-lag peaks in neural CCGs. a, Pearson’s cross-correlation of a connected and an unconnected neuron pair. b, Left: matrix of absolute lags of the CCG peaks. This partly reveals the circuit: directly connected neurons exhibit short lags. Right: binary matrix connecting neuron pairs with short lags. c, Top: CCG-based weight matrix: weight is set to the peak cross-correlation if the neurons are connected (lag < τ), zero otherwise. Bottom: avg. weight profile. GLM fares better. d, Combining CCG and GLM. Top: matrix of GLM-inferred weights if neuron pairs have short CCG lag, zero otherwise. Bottom: avg. weight profile. Relative to pure GLM (Δ = 0.23), this method (Δ = 0.32) removes some biases but introduces others. (e-j) Spike CCGs from the sparse, non-symmetric, strong weight (rRSA = 0.1) random network. e, Unconnected pair. CCG has no sharp features around 0, indicating no direct connection. f, Unconnected pair. Broad symmetric peak, indicating multiple indirect influences through mutual connections, is discounted. g, Connected pair. Sharp asymmetric dip at 0 reveals direct (inhibitory) connection. h, Unconnected pair. CCG is indistinguishable from the previous, and passes the criterion. i, Connected pair, but no CCG features. j, Connected pair, but broad symmetric peak is discounted.

Extended Data Fig. 8 Results of inference using a GLM on data generated by a linear-nonlinear-Poisson model with a rectifying linear response (see Methods).

a, Pattern coherence against weight strength for the generative LNP network. b, Inferred weight matrices (top) and superposition of rows (bottom, line marks zero), at several weight strengths. c, Inference error and bias fraction against weight strength. Optimal inference is at the point of pattern onset.

Extended Data Fig. 9 Activity and inference in the random balanced networks that are fully or sparsely connected.

a, Top: waterfall plots of neural fields of the full network at weak weights (when activity decays) and strong weights (when activity is chaotic), in response to a brief uniform feed-forward pulse. Bottom: corresponding inferences. (b-d) Inference on the sparse network. b, Left: true and inferred weights (using logistic regression) for the network with no noisy drive, at rRSB = 0.6. Some zero weights are inferred to be non-zero, while some nonzero weights are underestimated. Right: true and inferred weights (using only logistic regression, and augmented with CCG information) for the same condition, but when the network is noise-driven. c, Inference error (using the two methods) vs recurrent weight strength, and data volume, on the noise-driven network. d, Example CCG (top) and its time-derivative (bottom) of a connected neuron pair in the noise-driven network.

Extended Data Fig. 10 Entropy of spiking activity states of neural circuits.

a, Entropy of the distributions of 22-neuron spike sub-states from the true local ring circuit W and the non-local circuit $${\bf{W}}^{\prime}$$. b, Entropies of the spike sub-states of the two circuits computed with different data fractions across weight strengths. At all weights, the computed entropies converge as the data approaches the total volume. c: Example slice of plot b at the weakest weights, where entropy convergence takes the longest.

Rights and permissions

Reprints and Permissions

Das, A., Fiete, I.R. Systematic errors in connectivity inferred from activity in strongly recurrent networks. Nat Neurosci 23, 1286–1296 (2020). https://doi.org/10.1038/s41593-020-0699-2

• Accepted:

• Published:

• Issue Date:

• Automatic and Reliable Quantification of Tonic Dopamine Concentrations In Vivo Using a Novel Probabilistic Inference Method

• Jaekyung Kim
• , Abhijeet S. Barath
• , Aaron E. Rusheen
• , Juan M. Rojas Cabrera
• , J. Blair Price
• , Hojin Shin
• , Abhinav Goyal
• , Jason W. Yuen
• , Danielle E. Jondal
• , Charles D. Blaha
• , Kendall H. Lee
• , Dong Pyo Jang
•  & Yoonbae Oh

ACS Omega (2021)

• Targeted photostimulation uncovers circuit motifs supporting short-term memory

• Kayvon Daie
• , Karel Svoboda
•  & Shaul Druckmann

Nature Neuroscience (2021)

• Inferring functional connectivity through graphical directed information

• Joseph Young
• , Curtis L Neveu
• , John H Byrne
•  & Behnaam Aazhang

Journal of Neural Engineering (2021)

• A convolutional neural network for estimating synaptic connectivity from spike trains

• Daisuke Endo
• , Ryota Kobayashi
• , Ramon Bartolo
• , Bruno B. Averbeck
• , Yasuko Sugase-Miyamoto
• , Kazuko Hayashi
• , Kenji Kawano
• , Barry J. Richmond
•  & Shigeru Shinomoto

Scientific Reports (2021)

• Inferring causal networks of dynamical systems through transient dynamics and perturbation

• George Stepaniants
• , Bingni W. Brunton
•  & J. Nathan Kutz

Physical Review E (2020)