Perspective | Published:

Quasi-experimental causality in neuroscience and behavioural research

Nature Human Behaviourvolume 2pages891898 (2018) | Download Citation

Abstract

In many scientific domains, causality is the key question. For example, in neuroscience, we might ask whether a medication affects perception, cognition or action. Randomized controlled trials are the gold standard to establish causality, but they are not always practical. The field of empirical economics has developed rigorous methods to establish causality even when randomized controlled trials are not available. Here we review these quasi-experimental methods and highlight how neuroscience and behavioural researchers can use them to do research that can credibly demonstrate causal effects.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  1. 1.

    Pearl, J. Causality (Cambridge Univ. Press, New York, 2009).

  2. 2.

    The Notorious B.I.G. Mo’ Money Mo’ Problems (Bad Boy Records, 1997).

  3. 3.

    Grodstein, F. et al. A prospective, observational study of postmenopausal hormone therapy and primary prevention of cardiovascular disease. Ann. Intern. Med. 133, 933–941 (2000).

  4. 4.

    Manson, J. E. et al. Estrogen plus progestin and the risk of coronary heart disease. N. Engl. J. Med. 349, 523–534 (2003).

  5. 5.

    Humphrey, L. L., Chan, B. K. & Sox, H. C. Postmenopausal hormone replacement therapy and the primary prevention of cardiovascular disease. Ann. Intern. Med. 137, 273–284 (2002).

  6. 6.

    Greenland, S. Randomization, statistics, and causal inference. Epidemiology 1, 421–429 (1990).

  7. 7.

    Ismail-Beigi, F. et al. Effect of intensive treatment of hyperglycaemia on microvascular outcomes in type 2 diabetes: an analysis of the ACCORD randomised trial. Lancet 376, 419–430 (2010).

  8. 8.

    Officers, T. A. Major outcomes in high-risk hypertensive patients randomized to or calcium channel blocker vs diuretic. J. Am. Med. Assoc. 288, 2981–2997 (2002).

  9. 9.

    Group, S. R. A randomized trial of intensive versus standard blood-pressure control. N. Engl. J. Med. 373, 2103–2116 (2015).

  10. 10.

    Granger, C. W. Investigating causal relations by econometric models and cross-spectral methods. Econometrica 37, 424–438 (1969).

  11. 11.

    Angrist, J. D. & Pischke, J.-S. Mostly Harmless Econometrics: An Empiricist’s Companion (Princeton Univ. Press, Princeton, 2008).

  12. 12.

    Leamer, E. E. Let’s take the con out of econometrics. Am. Econ. Rev. 73, 31–43 (1983).

  13. 13.

    Thistlethwaite, D. L. & Campbell, D. T. Regression-discontinuity analysis: an alternative to the ex-post facto experiment. J. Educ. Psychol. 51, 309–317 (1960).

  14. 14.

    Imbens, G. W. & Lemieux, T. Regression discontinuity designs: a guide to practice. J. Econom. 142, 615–635 (2008).

  15. 15.

    Angrist, J., Azoulay, P., Ellison, G., Hill, R. & Lu, S. F. Economic research evolves: fields and styles. Am. Econ. Rev. 107, 293–297 (2017).

  16. 16.

    Angrist, J., Azoulay, P., Ellison, G., Hill, R. & Lu, S. F. Inside Job or Deep Impact? Using Extramural Citations to Assess Economic Scholarship (National Bureau of Economic Research, 2017).

  17. 17.

    McCrary, J. Manipulation of the running variable in the regression discontinuity design: a density test. J. Econom. 142, 698–714 (2008).

  18. 18.

    Trochim, W. M. Research Design for Program Evaluation: The Regression-Discontinuity Approach (Sage Publications, Beverly Hills, 1984).

  19. 19.

    Jacob, R., Zhu, P., Somers, M. A., & Bloom, H. A practical guide to regression discontinuity. MDRC https://www.mdrc.org/publication/practical-guide-regression-discontinuity (2012).

  20. 20.

    Lansdell, B. & Kording, K. Spiking allows neurons to estimate their causal effect. Preprint at bioRxiv https://doi.org/10.1101/253351 (2018).

  21. 21.

    Moscoe, E., Bor, J. & Bärnighausen, T. Regression discontinuity designs are underutilized in medicine, epidemiology, and public health: a review of current and best practice. J. Clin. Epidemiol. 68, 122–133 (2015).

  22. 22.

    Pischke, J. S. The impact of length of the school year on student performance and earnings: evidence from the German short school years. Econ. J. 117, 1216–1242 (2007).

  23. 23.

    Athey, S. & Imbens, G. W. Identification and inference in nonlinear difference-in-differences models. Econometrica 74, 431–497 (2006).

  24. 24.

    Angrist, J. D. et al. Identification of causal effects using instrumental variables. J. Am. Stat. Assoc. 91, 444–455 (2016).

  25. 25.

    Evans, W. N. & Ringel, J. S. Can higher cigarette taxes improve birth outcomes? J. Public. Econ. 72, 135–154 (1999).

  26. 26.

    Stock, J. H. & Yogo, M. Testing for Weak Instruments in Linear IV Regression. (National Bureau of Economic Research, Cambridge, 2002).

  27. 27.

    Li, X., Yamawaki, N., Barrett, J. M., Körding, K. P. & Shepherd, G. Scaling of optogenetically evoked signaling in a higher-order corticocortical pathway in the anesthetized mouse. Front. Syst. Neurosci. 12, 16 (2018).

  28. 28.

    Koller, D. & Friedman, N. Probabilistic Graphical Models: Principles and Techniques (MIT Press, Cambridge, 2009).

  29. 29.

    Ullman, J. B., & Bentler, P. M. in Handbook of Psychology 2nd edn (eds Schinka, J. A. & Velicer, W. F.) Ch. 23 (John Wiley & Sons, Hoboken, 2012).

  30. 30.

    Dehejia, R. H. & Wahba, S. Propensity score-matching methods for nonexperimental causal studies. Rev. Econ. Stat. 84, 151–161 (2002).

  31. 31.

    Rosenbaum, P. R. & Rubin, D. B. The central role of the propensity score in observational studies for causal effects. Biometrika 70, 41–55 (1983).

  32. 32.

    Imbens, G. W. & Rubin, D. B. Causal Inference in Statistics, Social, and Biomedical Sciences (Cambridge Univ. Press, New York, 2015).

  33. 33.

    Hoyer, P. O., Janzing, D., Mooij, J. M., Peters, J. & Schölkopf, B. Nonlinear causal discovery with additive noise models. Adv. Neural Inf. Process. Syst. 21, 689–696 (2009).

  34. 34.

    Abadie, A., Diamond, A. & Hainmueller, J. Synthetic control methods for comparative case studies: estimating the effect of California’s tobacco control program. J. Am. Stat. Assoc. 105, 493–505 (2010).

  35. 35.

    Ramsey, J. D. et al. Six problems for causal inference from fMRI. NeuroImage 49, 1545–1558 (2010).

  36. 36.

    Jonas, E. & Kording, K. P. Could a neuroscientist understand a microprocessor? PLoS Comput. Biol. 13, e1005268 (2017).

  37. 37.

    Valdes-Sosa, P. A., Roebroeck, A., Daunizeau, J. & Friston, K. Effective connectivity: influence, causality and biophysical modeling. NeuroImage 58, 339–361 (2011).

  38. 38.

    Smith, S. M., Miller, K. L., Salimi-Khorshidi, G., Webster, M. & Beckmann, C. F. et al. Network modelling methods for FMRI. NeuroImage 54, 875–891 (2011).

  39. 39.

    Kwak, H., Lee, C., Park, H. & Moon, S. What is Twitter, a social network or a news media? In Proc. 19th International Conference on World Wide Web 591–600 (ACM, 2010).

  40. 40.

    Bem, J. Using match confidence to adjust a performance threshold. Google Patent 7346615 (2008).

  41. 41.

    Slemrod, J. Buenas notches: lines and notches in tax system design. EJ Tax Res. 11, 259–283 (2013).

  42. 42.

    Angrist, J. D. & Pischke, J.-S. The credibility revolution in empirical economics: how better research design is taking the con out of econometrics. J. Econ. Perspect. 24, 3–30 (2010).

  43. 43.

    Krakauer, J. W., Ghazanfar, A. A., Gomez-Marin, A., MacIver, M. A. & Poeppel, D. Neuroscience needs behavior: correcting a reductionist bias. Neuron (in the press).

  44. 44.

    Pillow, J. W. et al. Spatio-temporal correlations and visual signalling in a complete neuronal population. Nature 454, 995–999 (2008).

  45. 45.

    Stevenson, I. H. & Körding, K. P. On the similarity of functional connectivity between neurons estimated across timescales. PLoS ONE 5, e9206 (2010).

  46. 46.

    Sakkalis, V. Review of advanced techniques for the estimation of brain connectivity measured with EEG/MEG. Comput. Biol. Med. 41, 1110–1117 (2011).

  47. 47.

    Bressler, S. L. & Seth, A. K. Wiener–Granger causality: a well established methodology. Neuroimage 58, 323–329 (2011).

  48. 48.

    Ding, M., Chen, Y. & Bressler, S. L. in Handbook of Time Series Analysis: Recent Theoretical Developments and Applications (eds Schelter, B., Winterhalder, M. & Timmer, J.) 335–368 (Wiley-VCH, Weinheim, 2006).

  49. 49.

    Hiemstra, C. & Jones, J. D. Testing for linear and nonlinear Granger causality in the stock price — volume relation. J. Finance 49, 1639–1664 (1994).

  50. 50.

    Chen, Z. Advanced State Space Methods for Neural and Clinical Data (Cambridge Univ. Press, Cambridge, 2015).

  51. 51.

    Shumway, R. H. & Stoffer, D. S. in Time Series Analysis and its Applications (Shumway, R. H. & Stoffer, D. S.) 319–404 (Springer, New York, 2011).

  52. 52.

    Friston, K. J., Harrison, L. & Penny, W. Dynamic causal modelling. NeuroImage 19, 1273–1302 (2003).

  53. 53.

    Semedo, J., Zandvakili, A., Kohn, A., Machens, C. K. & Byron, M. Y. Extracting latent structure from multiple interacting neural populations. Adv. Neural Inf. Process. Syst. 27, 2942–2950 (2014).

  54. 54.

    Daunizeau, J., David, O. & Stephan, K. E. Dynamic causal modelling: a critical review of the biophysical and statistical foundations. NeuroImage 58, 312–322 (2009).

  55. 55.

    Latimer, K. W., Yates, J. L., Meister, M. L. R., Huk, A. C. & Pillow, J. W. Single-trial spike trains in parietal cortex reveal discrete steps during decision-making. Science 349, 184–187 (2015).

  56. 56.

    Nevo, A. & Whinston, M. D. Taking the dogma out of econometrics: structural modeling and credible inference. J. Econ. Perspect. 24, 69–82 (2010).

  57. 57.

    Song, L., Kolar, M. & Xing, E. P. Time-varying dynamic bayesian networks. Adv. Neural Inf. Process. Syst. 22, 1732–1740 (2009).

  58. 58.

    Goodman, N. D., Ullman, T. D. & Tenenbaum, J. B. Learning a theory of causality. Psychol. Rev. 118, 110–119 (2011).

  59. 59.

    Gopnik, A. et al. A Theory of causal learning in children: causal maps and Bayes nets. Psychol. Rev. 111, 3–32 (2004).

  60. 60.

    Gopnik, A. & Tenenbaum, J. B. Bayesian networks, Bayesian learning and cognitive development. Dev. Sci. 10, 281–287 (2007).

  61. 61.

    Körding, K. P. et al. Causal inference in multisensory perception. PLoS ONE 2, e943 (2007).

Download references

Author information

Affiliations

  1. Department of Social Policy and Practice, University of Pennsylvania, Philadelphia, PA, USA

    • Ioana E. Marinescu
  2. Division of Neurology, Children’s Hospital of Philadelphia, Philadelphia, PA, USA

    • Patrick N. Lawlor
  3. Departments of Neuroscience and Bioengineering, Leonard Davis Institute, Warren Center for Network Science, Wharton Neuroscience Initiative, University of Pennsylvania, Philadelphia, PA, USA

    • Konrad P. Kording
  4. Canadian Institute For Advanced Research, Toronto, Ontario, Canada

    • Konrad P. Kording

Authors

  1. Search for Ioana E. Marinescu in:

  2. Search for Patrick N. Lawlor in:

  3. Search for Konrad P. Kording in:

Competing interests

The authors declare no competing interests.

Corresponding author

Correspondence to Ioana E. Marinescu.

About this article

Publication history

Received

Accepted

Published

Issue Date

DOI

https://doi.org/10.1038/s41562-018-0466-5