Abstract

Brain decoders use neural recordings to infer the activity or intent of a user. To train a decoder, one generally needs to infer the measured variables of interest (covariates) from simultaneously measured neural activity. However, there are cases for which obtaining supervised data is difficult or impossible. Here, we describe an approach for movement decoding that does not require access to simultaneously measured neural activity and motor outputs. We use the statistics of movement—much like cryptographers use the statistics of language—to find a mapping between neural activity and motor variables, and then align the distribution of decoder outputs with the typical distribution of motor outputs by minimizing their Kullback–Leibler divergence. By using datasets collected from the motor cortex of three non-human primates performing either a reaching task or an isometric force-production task, we show that the performance of such a distribution-alignment decoding algorithm is comparable to the performance of supervised approaches. Distribution-alignment decoding promises to broaden the set of potential applications of brain decoding.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  1. 1.

    Kay, K. N. et al. Identifying natural images from human brain activity. Nature 452, 352–355 (2008).

  2. 2.

    Cowen, A. S. et al. Neural portraits of perception: reconstructing face images from evoked brain activity. Neuroimage 94, 12–22 (2014).

  3. 3.

    Mitchell, T. M. et al. Predicting human brain activity associated with the meanings of nouns. Science 320, 1191–1195 (2008).

  4. 4.

    Haynes, J. et al. Decoding mental states from brain activity in humans. Nat. Rev. Neurosci. 7, 523–534 (2006).

  5. 5.

    Serruya, M. D. et al. Brain–machine interface: instant neural control of a movement signal. Nature 416, 141–142 (2002).

  6. 6.

    Kemere, C. et al. Model-based neural decoding of reaching movements: a maximum likelihood approach. IEEE Trans. Biomed. Eng. 51, 925–932 (2004).

  7. 7.

    Tkach, D. et al. Observation-based learning for brain–machine interfaces. Curr. Opin. Neurobiol. 18, 589–594 (2008).

  8. 8.

    Anderson, D. J. et al. Toward a science of computational ethology. Neuron 84, 18–31 (2014).

  9. 9.

    Donoghue, J. P. Connecting cortex to machines: recent advances in brain interfaces. Nat. Neurosci. 5, 1085–1088 (2002).

  10. 10.

    Bose, R. Information Theory, Coding and Cryptography (McGraw-Hill Education, New Delhi, India, 2008).

  11. 11.

    Chaudhari, M. P. et al. A survey on cryptography algorithms. Int. J. Adv. Res. Comput. Sci. Manag. Stud. 2, 100–104 (2014).

  12. 12.

    Wang, L. et al. Automatic gait recognition based on statistical shape analysis. IEEE Trans. Image Process. 12, 1120–1131 (2003).

  13. 13.

    Belić, J. J. et al. Decoding of human hand actions to handle missing limbs in neuroprosthetics. Front. Comput. Neuro. https://doi.org/10.3389/fncom.2015.00027.

  14. 14.

    Ishiduka, S. et al. Kinematic analysis of low dimensional structure in walking and running. In 2015 International Symposium on Micro-NanoMechatronics and Human Science (MHS) 1–6 (IEEE, 2015).

  15. 15.

    Damavsevivcius, R. et al. Smartphone user identity verification using gait characteristics. Symmetry 8, 100 (2016).

  16. 16.

    Kaufman, M. T. et al. The largest response component in the motor cortex reflects movement timing but not movement type. eNeuro 3, ENEURO–0085 (2016).

  17. 17.

    Marblestone, A. H. et al. Physical principles for scalable neural recording. Front. Comput. Neurosci. https://doi.org/10.3389/fncom.2013.00137.

  18. 18.

    Georgopoulos, A. P. et al. Neuronal population coding of movement direction. Science 233, 1416–1419 (1986).

  19. 19.

    Pillow, J. W. et al. Spatio-temporal correlations and visual signaling in a complete neuronal population. Nature 454, 995–999 (2008).

  20. 20.

    Dyer, E. L. et al. Greedy feature selection for subspace clustering. J. Mach. Learn. Res. 14, 2487–2517 (2013).

  21. 21.

    Schölkopf, B. et al. Kernel Methods in Computational Biology (MIT Press, Cambridge, MA, USA, 2004).

  22. 22.

    Ingram, J. N. et al. The statistics of natural hand movements. Exp. Brain Res. 188, 223–236 (2008).

  23. 23.

    Ejaz, N. et al. Hand use predicts the structure of representations in sensorimotor cortex. Nat. Neurosci18, 1034–1040 (2015).

  24. 24.

    Byron, M. Y. et al. Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity. In Advances in Neural Information Processing Systems (eds Koller, D. et al.) 1881–1888 (MIT Press, 2009).

  25. 25.

    Fernandes, H. L. et al. Saliency and saccade encoding in the frontal eye field during natural scene search. Cereb. Cortex 24, 3232–3245 (2014).

  26. 26.

    Li, Z. et al. Adaptive decoding for brain–machine interfaces through Bayesian parameter updates. Neural Comput. 23, 3162–3204 (2011).

  27. 27.

    Gilja, V. et al. A high-performance neural prosthesis enabled by control algorithm design. Nat. Neurosci. 15, 1752–1757 (2012).

  28. 28.

    Orsborn, A. L. et al. Closed-loop decoder adaptation on intermediate time-scales facilitates rapid bmi performance improvements independent of decoder initialization conditions. IEEE Trans. Neural Syst. Rehabil. Eng. 20, 468–477 (2012).

  29. 29.

    Orsborn, A. L. et al. Closed-loop decoder adaptation shapes neural plasticity for skillful neuroprosthetic control. Neuron 82, 1380–1393 (2014).

  30. 30.

    Ashe, J. et al. Movement parameters and neural activity in motor cortex and area 5. Cereb. Cortex 4, 590–600 (1994).

  31. 31.

    Averbeck, B. B. et al. Parietal representation of hand velocity in a copy task. J. Neurophysio. 93, 508–518 (2005).

  32. 32.

    Stevenson, I. H. et al. Statistical assessment of the stability of neural movement representations. J. Neurophys io106, 764–774 (2011).

  33. 33.

    Stevenson, I. H. et al. Functional connectivity and tuning curves in populations of simultaneously recorded neurons. PLoS Comput. Biol. 8, e1002775 (2012).

  34. 34.

    Jagabathula, S. et al. Inferring rankings using constrained sensing. IEEE Trans. Inform. Theory 57, 7288–7306 (2011).

  35. 35.

    Van Der Maaten, L. et al. Dimensionality reduction: a comparative. J. Mach. Learn. Res. 10, 66–71 (2009).

  36. 36.

    Tenenbaum, J. B. et al. A global geometric framework for nonlinear dimensionality reduction. Science 290, 2319–2323 (2000).

  37. 37.

    van der Maaten, L. et al. Visualizing data using t-sne. J. Mach. Learn. Res. 9, 2579–2605 (2008).

  38. 38.

    Bishop, C. M. Bayesian PCA. In Advances in Neural Information Processing Systems (eds Kearns, M. J. et al.) 382–388 (MIT Press, 1999).

  39. 39.

    Macke, J. H. et al. Empirical models of spiking in neural populations. In Advances in Neural Information Processing Systems (eds Shawe-Taylor, J. et al.) 1350–1358 (MIT Press, 2011).

  40. 40.

    Póczos, B. et al. On the estimation of alpha-divergences. In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics 609–617 (2011).

  41. 41.

    Loftsgaarden, D. O. et al. A nonparametric estimate of a multivariate density function. Ann. Math. Stat. 36, 1049–1051 (1965).

Download references

Acknowledgements

The authors would like to thank B. Dekleva, P. Ramkumar, J. Glaser, D. Acuna, P. Lawlor, S. Saeb, L. Lonini, S. Solla and B. Yu for discussions and advice. This work was supported by NINDS R01 NS053603, NINDS R01 NS074044, U01 MH109100 and R01 NS074044.

Author information

Affiliations

  1. Department of Biomedical Engineering, Georgia Institute of Technology & Emory University, Atlanta, GA, USA

    • Eva L. Dyer
  2. Department of Physical Medicine and Rehabilitation, Northwestern University, Chicago, IL, USA

    • Mohammad Gheshlaghi Azar
    • , Hugo L. Fernandes
    •  & Lee E. Miller
  3. Sensory Motor Performance Program, Rehabilitation Institute of Chicago, Chicago, IL, USA

    • Mohammad Gheshlaghi Azar
    •  & Hugo L. Fernandes
  4. Department of Biomedical Engineering, Northwestern University, Evanston, IL, USA

    • Matthew G. Perich
    • , Stephanie Naufel
    •  & Lee E. Miller
  5. Department of Physiology, Northwestern University, Chicago, IL, USA

    • Lee E. Miller
  6. Department of Biomedical Engineering, University of Pennsylvania, Philadelphia, PA, USA

    • Konrad P. Körding

Authors

  1. Search for Eva L. Dyer in:

  2. Search for Mohammad Gheshlaghi Azar in:

  3. Search for Matthew G. Perich in:

  4. Search for Hugo L. Fernandes in:

  5. Search for Stephanie Naufel in:

  6. Search for Lee E. Miller in:

  7. Search for Konrad P. Körding in:

Contributions

E.L.D., M.G.A. and K.P.K. designed the research. E.L.D. and M.G.A. implemented the algorithms. E.L.D., H.L.F. and M.G.A. tested the decoders on neural recordings and analysed the results. M.G.P. and S.N. collected and sorted the NHP data. K.P.K. and L.M. managed and advised on the project. E.L.D., M.G.A. and K.P.K. wrote the manuscript. All authors provided feedback on the manuscript.

Competing interests

The authors declare no competing financial interests.

Corresponding author

Correspondence to Eva L. Dyer.

Supplementary information

About this article

Publication history

Received

Accepted

Published

DOI

https://doi.org/10.1038/s41551-017-0169-7

Further reading