Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Quasi-experimental causality in neuroscience and behavioural research

Nature Human Behaviour volume 2, pages 891–898 (2018)Cite this article

Subjects

Abstract

In many scientific domains, causality is the key question. For example, in neuroscience, we might ask whether a medication affects perception, cognition or action. Randomized controlled trials are the gold standard to establish causality, but they are not always practical. The field of empirical economics has developed rigorous methods to establish causality even when randomized controlled trials are not available. Here we review these quasi-experimental methods and highlight how neuroscience and behavioural researchers can use them to do research that can credibly demonstrate causal effects.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Rent or buy this article

Get just this article for as long as you need it

$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Regression discontinuity design.
Fig. 2: Difference-in-differences.
Fig. 3: Instrumental variables.

References

  1. Pearl, J. Causality (Cambridge Univ. Press, New York, 2009).

  2. The Notorious B.I.G. Mo’ Money Mo’ Problems (Bad Boy Records, 1997).

  3. Grodstein, F. et al. A prospective, observational study of postmenopausal hormone therapy and primary prevention of cardiovascular disease. Ann. Intern. Med. 133, 933–941 (2000).

    Article  CAS  Google Scholar 

  4. Manson, J. E. et al. Estrogen plus progestin and the risk of coronary heart disease. N. Engl. J. Med. 349, 523–534 (2003).

    Article  CAS  Google Scholar 

  5. Humphrey, L. L., Chan, B. K. & Sox, H. C. Postmenopausal hormone replacement therapy and the primary prevention of cardiovascular disease. Ann. Intern. Med. 137, 273–284 (2002).

    Article  CAS  Google Scholar 

  6. Greenland, S. Randomization, statistics, and causal inference. Epidemiology 1, 421–429 (1990).

    Article  CAS  Google Scholar 

  7. Ismail-Beigi, F. et al. Effect of intensive treatment of hyperglycaemia on microvascular outcomes in type 2 diabetes: an analysis of the ACCORD randomised trial. Lancet 376, 419–430 (2010).

    Article  Google Scholar 

  8. Officers, T. A. Major outcomes in high-risk hypertensive patients randomized to or calcium channel blocker vs diuretic. J. Am. Med. Assoc. 288, 2981–2997 (2002).

    Article  Google Scholar 

  9. Group, S. R. A randomized trial of intensive versus standard blood-pressure control. N. Engl. J. Med. 373, 2103–2116 (2015).

    Article  Google Scholar 

  10. Granger, C. W. Investigating causal relations by econometric models and cross-spectral methods. Econometrica 37, 424–438 (1969).

    Article  Google Scholar 

  11. Angrist, J. D. & Pischke, J.-S. Mostly Harmless Econometrics: An Empiricist’s Companion (Princeton Univ. Press, Princeton, 2008).

  12. Leamer, E. E. Let’s take the con out of econometrics. Am. Econ. Rev. 73, 31–43 (1983).

    Google Scholar 

  13. Thistlethwaite, D. L. & Campbell, D. T. Regression-discontinuity analysis: an alternative to the ex-post facto experiment. J. Educ. Psychol. 51, 309–317 (1960).

    Article  Google Scholar 

  14. Imbens, G. W. & Lemieux, T. Regression discontinuity designs: a guide to practice. J. Econom. 142, 615–635 (2008).

    Article  Google Scholar 

  15. Angrist, J., Azoulay, P., Ellison, G., Hill, R. & Lu, S. F. Economic research evolves: fields and styles. Am. Econ. Rev. 107, 293–297 (2017).

    Article  Google Scholar 

  16. Angrist, J., Azoulay, P., Ellison, G., Hill, R. & Lu, S. F. Inside Job or Deep Impact? Using Extramural Citations to Assess Economic Scholarship (National Bureau of Economic Research, 2017).

  17. McCrary, J. Manipulation of the running variable in the regression discontinuity design: a density test. J. Econom. 142, 698–714 (2008).

    Article  Google Scholar 

  18. Trochim, W. M. Research Design for Program Evaluation: The Regression-Discontinuity Approach (Sage Publications, Beverly Hills, 1984).

  19. Jacob, R., Zhu, P., Somers, M. A., & Bloom, H. A practical guide to regression discontinuity. MDRC https://www.mdrc.org/publication/practical-guide-regression-discontinuity (2012).

  20. Lansdell, B. & Kording, K. Spiking allows neurons to estimate their causal effect. Preprint at bioRxiv https://doi.org/10.1101/253351 (2018).

  21. Moscoe, E., Bor, J. & Bärnighausen, T. Regression discontinuity designs are underutilized in medicine, epidemiology, and public health: a review of current and best practice. J. Clin. Epidemiol. 68, 122–133 (2015).

    Article  Google Scholar 

  22. Pischke, J. S. The impact of length of the school year on student performance and earnings: evidence from the German short school years. Econ. J. 117, 1216–1242 (2007).

    Article  Google Scholar 

  23. Athey, S. & Imbens, G. W. Identification and inference in nonlinear difference-in-differences models. Econometrica 74, 431–497 (2006).

    Article  Google Scholar 

  24. Angrist, J. D. et al. Identification of causal effects using instrumental variables. J. Am. Stat. Assoc. 91, 444–455 (2016).

    Article  Google Scholar 

  25. Evans, W. N. & Ringel, J. S. Can higher cigarette taxes improve birth outcomes? J. Public. Econ. 72, 135–154 (1999).

    Article  Google Scholar 

  26. Stock, J. H. & Yogo, M. Testing for Weak Instruments in Linear IV Regression. (National Bureau of Economic Research, Cambridge, 2002).

    Book  Google Scholar 

  27. Li, X., Yamawaki, N., Barrett, J. M., Körding, K. P. & Shepherd, G. Scaling of optogenetically evoked signaling in a higher-order corticocortical pathway in the anesthetized mouse. Front. Syst. Neurosci. 12, 16 (2018).

    Article  Google Scholar 

  28. Koller, D. & Friedman, N. Probabilistic Graphical Models: Principles and Techniques (MIT Press, Cambridge, 2009).

  29. Ullman, J. B., & Bentler, P. M. in Handbook of Psychology 2nd edn (eds Schinka, J. A. & Velicer, W. F.) Ch. 23 (John Wiley & Sons, Hoboken, 2012).

  30. Dehejia, R. H. & Wahba, S. Propensity score-matching methods for nonexperimental causal studies. Rev. Econ. Stat. 84, 151–161 (2002).

    Article  Google Scholar 

  31. Rosenbaum, P. R. & Rubin, D. B. The central role of the propensity score in observational studies for causal effects. Biometrika 70, 41–55 (1983).

    Article  Google Scholar 

  32. Imbens, G. W. & Rubin, D. B. Causal Inference in Statistics, Social, and Biomedical Sciences (Cambridge Univ. Press, New York, 2015).

  33. Hoyer, P. O., Janzing, D., Mooij, J. M., Peters, J. & Schölkopf, B. Nonlinear causal discovery with additive noise models. Adv. Neural Inf. Process. Syst. 21, 689–696 (2009).

    Google Scholar 

  34. Abadie, A., Diamond, A. & Hainmueller, J. Synthetic control methods for comparative case studies: estimating the effect of California’s tobacco control program. J. Am. Stat. Assoc. 105, 493–505 (2010).

    Article  CAS  Google Scholar 

  35. Ramsey, J. D. et al. Six problems for causal inference from fMRI. NeuroImage 49, 1545–1558 (2010).

    Article  CAS  Google Scholar 

  36. Jonas, E. & Kording, K. P. Could a neuroscientist understand a microprocessor? PLoS Comput. Biol. 13, e1005268 (2017).

    Article  Google Scholar 

  37. Valdes-Sosa, P. A., Roebroeck, A., Daunizeau, J. & Friston, K. Effective connectivity: influence, causality and biophysical modeling. NeuroImage 58, 339–361 (2011).

    Article  Google Scholar 

  38. Smith, S. M., Miller, K. L., Salimi-Khorshidi, G., Webster, M. & Beckmann, C. F. et al. Network modelling methods for FMRI. NeuroImage 54, 875–891 (2011).

    Article  Google Scholar 

  39. Kwak, H., Lee, C., Park, H. & Moon, S. What is Twitter, a social network or a news media? In Proc. 19th International Conference on World Wide Web 591–600 (ACM, 2010).

  40. Bem, J. Using match confidence to adjust a performance threshold. Google Patent 7346615 (2008).

  41. Slemrod, J. Buenas notches: lines and notches in tax system design. EJ Tax Res. 11, 259–283 (2013).

    Google Scholar 

  42. Angrist, J. D. & Pischke, J.-S. The credibility revolution in empirical economics: how better research design is taking the con out of econometrics. J. Econ. Perspect. 24, 3–30 (2010).

    Article  Google Scholar 

  43. Krakauer, J. W., Ghazanfar, A. A., Gomez-Marin, A., MacIver, M. A. & Poeppel, D. Neuroscience needs behavior: correcting a reductionist bias. Neuron (in the press).

  44. Pillow, J. W. et al. Spatio-temporal correlations and visual signalling in a complete neuronal population. Nature 454, 995–999 (2008).

    Article  CAS  Google Scholar 

  45. Stevenson, I. H. & Körding, K. P. On the similarity of functional connectivity between neurons estimated across timescales. PLoS ONE 5, e9206 (2010).

    Article  Google Scholar 

  46. Sakkalis, V. Review of advanced techniques for the estimation of brain connectivity measured with EEG/MEG. Comput. Biol. Med. 41, 1110–1117 (2011).

    Article  CAS  Google Scholar 

  47. Bressler, S. L. & Seth, A. K. Wiener–Granger causality: a well established methodology. Neuroimage 58, 323–329 (2011).

    Article  Google Scholar 

  48. Ding, M., Chen, Y. & Bressler, S. L. in Handbook of Time Series Analysis: Recent Theoretical Developments and Applications (eds Schelter, B., Winterhalder, M. & Timmer, J.) 335–368 (Wiley-VCH, Weinheim, 2006).

  49. Hiemstra, C. & Jones, J. D. Testing for linear and nonlinear Granger causality in the stock price — volume relation. J. Finance 49, 1639–1664 (1994).

    Google Scholar 

  50. Chen, Z. Advanced State Space Methods for Neural and Clinical Data (Cambridge Univ. Press, Cambridge, 2015).

  51. Shumway, R. H. & Stoffer, D. S. in Time Series Analysis and its Applications (Shumway, R. H. & Stoffer, D. S.) 319–404 (Springer, New York, 2011).

  52. Friston, K. J., Harrison, L. & Penny, W. Dynamic causal modelling. NeuroImage 19, 1273–1302 (2003).

    Article  CAS  Google Scholar 

  53. Semedo, J., Zandvakili, A., Kohn, A., Machens, C. K. & Byron, M. Y. Extracting latent structure from multiple interacting neural populations. Adv. Neural Inf. Process. Syst. 27, 2942–2950 (2014).

    Google Scholar 

  54. Daunizeau, J., David, O. & Stephan, K. E. Dynamic causal modelling: a critical review of the biophysical and statistical foundations. NeuroImage 58, 312–322 (2009).

    Article  Google Scholar 

  55. Latimer, K. W., Yates, J. L., Meister, M. L. R., Huk, A. C. & Pillow, J. W. Single-trial spike trains in parietal cortex reveal discrete steps during decision-making. Science 349, 184–187 (2015).

    Article  CAS  Google Scholar 

  56. Nevo, A. & Whinston, M. D. Taking the dogma out of econometrics: structural modeling and credible inference. J. Econ. Perspect. 24, 69–82 (2010).

    Article  Google Scholar 

  57. Song, L., Kolar, M. & Xing, E. P. Time-varying dynamic bayesian networks. Adv. Neural Inf. Process. Syst. 22, 1732–1740 (2009).

    Google Scholar 

  58. Goodman, N. D., Ullman, T. D. & Tenenbaum, J. B. Learning a theory of causality. Psychol. Rev. 118, 110–119 (2011).

    Article  Google Scholar 

  59. Gopnik, A. et al. A Theory of causal learning in children: causal maps and Bayes nets. Psychol. Rev. 111, 3–32 (2004).

    Article  Google Scholar 

  60. Gopnik, A. & Tenenbaum, J. B. Bayesian networks, Bayesian learning and cognitive development. Dev. Sci. 10, 281–287 (2007).

    Article  Google Scholar 

  61. Körding, K. P. et al. Causal inference in multisensory perception. PLoS ONE 2, e943 (2007).

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Social Policy and Practice, University of Pennsylvania, Philadelphia, PA, USA

    Ioana E. Marinescu

  2. Division of Neurology, Children’s Hospital of Philadelphia, Philadelphia, PA, USA

    Patrick N. Lawlor

  3. Departments of Neuroscience and Bioengineering, Leonard Davis Institute, Warren Center for Network Science, Wharton Neuroscience Initiative, University of Pennsylvania, Philadelphia, PA, USA

    Konrad P. Kording

  4. Canadian Institute For Advanced Research, Toronto, Ontario, Canada

    Konrad P. Kording

Authors
  1. Ioana E. Marinescu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Patrick N. Lawlor
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Konrad P. Kording
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ioana E. Marinescu.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Marinescu, I.E., Lawlor, P.N. & Kording, K.P. Quasi-experimental causality in neuroscience and behavioural research. Nat Hum Behav 2, 891–898 (2018). https://doi.org/10.1038/s41562-018-0466-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41562-018-0466-5

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing