Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

A cryptography-based approach for movement decoding

Nature Biomedical Engineering volume 1, pages 967–976 (2017)Cite this article

Subjects

Abstract

Brain decoders use neural recordings to infer the activity or intent of a user. To train a decoder, one generally needs to infer the measured variables of interest (covariates) from simultaneously measured neural activity. However, there are cases for which obtaining supervised data is difficult or impossible. Here, we describe an approach for movement decoding that does not require access to simultaneously measured neural activity and motor outputs. We use the statistics of movement—much like cryptographers use the statistics of language—to find a mapping between neural activity and motor variables, and then align the distribution of decoder outputs with the typical distribution of motor outputs by minimizing their Kullback–Leibler divergence. By using datasets collected from the motor cortex of three non-human primates performing either a reaching task or an isometric force-production task, we show that the performance of such a distribution-alignment decoding algorithm is comparable to the performance of supervised approaches. Distribution-alignment decoding promises to broaden the set of potential applications of brain decoding.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Kay, K. N. et al. Identifying natural images from human brain activity. Nature452, 352–355 (2008).

    Article  CAS  Google Scholar 

  2. Cowen, A. S. et al. Neural portraits of perception: reconstructing face images from evoked brain activity. Neuroimage94, 12–22 (2014).

    Article  Google Scholar 

  3. Mitchell, T. M. et al. Predicting human brain activity associated with the meanings of nouns. Science320, 1191–1195 (2008).

    Article  CAS  Google Scholar 

  4. Haynes, J. et al. Decoding mental states from brain activity in humans. Nat. Rev. Neurosci.7, 523–534 (2006).

    Article  CAS  Google Scholar 

  5. Serruya, M. D. et al. Brain–machine interface: instant neural control of a movement signal. Nature416, 141–142 (2002).

    Article  CAS  Google Scholar 

  6. Kemere, C. et al. Model-based neural decoding of reaching movements: a maximum likelihood approach. IEEE Trans. Biomed. Eng.51, 925–932 (2004).

    Article  Google Scholar 

  7. Tkach, D. et al. Observation-based learning for brain–machine interfaces. Curr. Opin. Neurobiol.18, 589–594 (2008).

    Article  CAS  Google Scholar 

  8. Anderson, D. J. et al. Toward a science of computational ethology. Neuron84, 18–31 (2014).

    Article  CAS  Google Scholar 

  9. Donoghue, J. P. Connecting cortex to machines: recent advances in brain interfaces. Nat. Neurosci.5, 1085–1088 (2002).

    Article  CAS  Google Scholar 

  10. Bose, R. Information Theory, Coding and Cryptography (McGraw-Hill Education, New Delhi, India, 2008).

  11. Chaudhari, M. P. et al. A survey on cryptography algorithms. Int. J. Adv. Res. Comput. Sci. Manag. Stud.2, 100–104 (2014).

  12. Wang, L. et al. Automatic gait recognition based on statistical shape analysis. IEEE Trans. Image Process.12, 1120–1131 (2003).

    Article  Google Scholar 

  13. Belić, J. J. et al. Decoding of human hand actions to handle missing limbs in neuroprosthetics. Front. Comput. Neuro. https://doi.org/10.3389/fncom.2015.00027.

  14. Ishiduka, S. et al. Kinematic analysis of low dimensional structure in walking and running. In 2015 International Symposium on Micro-NanoMechatronics and Human Science (MHS) 1–6 (IEEE, 2015).

  15. Damavsevivcius, R. et al. Smartphone user identity verification using gait characteristics. Symmetry8, 100 (2016).

    Article  Google Scholar 

  16. Kaufman, M. T. et al. The largest response component in the motor cortex reflects movement timing but not movement type. eNeuro3, ENEURO–0085 (2016).

    Article  Google Scholar 

  17. Marblestone, A. H. et al. Physical principles for scalable neural recording. Front. Comput. Neurosci. https://doi.org/10.3389/fncom.2013.00137.

  18. Georgopoulos, A. P. et al. Neuronal population coding of movement direction. Science233, 1416–1419 (1986).

    Article  CAS  Google Scholar 

  19. Pillow, J. W. et al. Spatio-temporal correlations and visual signaling in a complete neuronal population. Nature454, 995–999 (2008).

    Article  CAS  Google Scholar 

  20. Dyer, E. L. et al. Greedy feature selection for subspace clustering. J. Mach. Learn. Res.14, 2487–2517 (2013).

    Google Scholar 

  21. Schölkopf, B. et al. Kernel Methods in Computational Biology (MIT Press, Cambridge, MA, USA, 2004).

  22. Ingram, J. N. et al. The statistics of natural hand movements. Exp. Brain Res.188, 223–236 (2008).

    Article  Google Scholar 

  23. Ejaz, N. et al. Hand use predicts the structure of representations in sensorimotor cortex. Nat. Neurosci. 18, 1034–1040 (2015).

    Article  CAS  Google Scholar 

  24. Byron, M. Y. et al. Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity. In Advances in Neural Information Processing Systems (eds Koller, D. et al.) 1881–1888 (MIT Press, 2009).

  25. Fernandes, H. L. et al. Saliency and saccade encoding in the frontal eye field during natural scene search. Cereb. Cortex24, 3232–3245 (2014).

    Article  Google Scholar 

  26. Li, Z. et al. Adaptive decoding for brain–machine interfaces through Bayesian parameter updates. Neural Comput.23, 3162–3204 (2011).

    Article  Google Scholar 

  27. Gilja, V. et al. A high-performance neural prosthesis enabled by control algorithm design. Nat. Neurosci.15, 1752–1757 (2012).

    Article  CAS  Google Scholar 

  28. Orsborn, A. L. et al. Closed-loop decoder adaptation on intermediate time-scales facilitates rapid bmi performance improvements independent of decoder initialization conditions. IEEE Trans. Neural Syst. Rehabil. Eng.20, 468–477 (2012).

    Article  Google Scholar 

  29. Orsborn, A. L. et al. Closed-loop decoder adaptation shapes neural plasticity for skillful neuroprosthetic control. Neuron82, 1380–1393 (2014).

    Article  CAS  Google Scholar 

  30. Ashe, J. et al. Movement parameters and neural activity in motor cortex and area 5. Cereb. Cortex4, 590–600 (1994).

    Article  CAS  Google Scholar 

  31. Averbeck, B. B. et al. Parietal representation of hand velocity in a copy task. J. Neurophysio.93, 508–518 (2005).

    Article  Google Scholar 

  32. Stevenson, I. H. et al. Statistical assessment of the stability of neural movement representations. J. Neurophysio. 106, 764–774 (2011).

    Article  Google Scholar 

  33. Stevenson, I. H. et al. Functional connectivity and tuning curves in populations of simultaneously recorded neurons. PLoS Comput. Biol.8, e1002775 (2012).

    Article  CAS  Google Scholar 

  34. Jagabathula, S. et al. Inferring rankings using constrained sensing. IEEE Trans. Inform. Theory57, 7288–7306 (2011).

    Article  Google Scholar 

  35. Van Der Maaten, L. et al. Dimensionality reduction: a comparative. J. Mach. Learn. Res.10, 66–71 (2009).

    Google Scholar 

  36. Tenenbaum, J. B. et al. A global geometric framework for nonlinear dimensionality reduction. Science290, 2319–2323 (2000).

    Article  CAS  Google Scholar 

  37. van der Maaten, L. et al. Visualizing data using t-sne. J. Mach. Learn. Res.9, 2579–2605 (2008).

    Google Scholar 

  38. Bishop, C. M. Bayesian PCA. In Advances in Neural Information Processing Systems (eds Kearns, M. J. et al.) 382–388 (MIT Press, 1999).

  39. Macke, J. H. et al. Empirical models of spiking in neural populations. In Advances in Neural Information Processing Systems (eds Shawe-Taylor, J. et al.) 1350–1358 (MIT Press, 2011).

  40. Póczos, B. et al. On the estimation of alpha-divergences. In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics 609–617 (2011).

  41. Loftsgaarden, D. O. et al. A nonparametric estimate of a multivariate density function. Ann. Math. Stat.36, 1049–1051 (1965).

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank B. Dekleva, P. Ramkumar, J. Glaser, D. Acuna, P. Lawlor, S. Saeb, L. Lonini, S. Solla and B. Yu for discussions and advice. This work was supported by NINDS R01 NS053603, NINDS R01 NS074044, U01 MH109100 and R01 NS074044.

Author information

Authors and Affiliations

  1. Department of Biomedical Engineering, Georgia Institute of Technology & Emory University, Atlanta, GA, USA

    Eva L. Dyer

  2. Department of Physical Medicine and Rehabilitation, Northwestern University, Chicago, IL, USA

    Mohammad Gheshlaghi Azar, Hugo L. Fernandes & Lee E. Miller

  3. Sensory Motor Performance Program, Rehabilitation Institute of Chicago, Chicago, IL, USA

    Mohammad Gheshlaghi Azar & Hugo L. Fernandes

  4. Department of Biomedical Engineering, Northwestern University, Evanston, IL, USA

    Matthew G. Perich, Stephanie Naufel & Lee E. Miller

  5. Department of Physiology, Northwestern University, Chicago, IL, USA

    Lee E. Miller

  6. Department of Biomedical Engineering, University of Pennsylvania, Philadelphia, PA, USA

    Konrad P. Körding

Authors
  1. Eva L. Dyer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammad Gheshlaghi Azar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Matthew G. Perich
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hugo L. Fernandes
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Stephanie Naufel
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Lee E. Miller
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Konrad P. Körding
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

E.L.D., M.G.A. and K.P.K. designed the research. E.L.D. and M.G.A. implemented the algorithms. E.L.D., H.L.F. and M.G.A. tested the decoders on neural recordings and analysed the results. M.G.P. and S.N. collected and sorted the NHP data. K.P.K. and L.M. managed and advised on the project. E.L.D., M.G.A. and K.P.K. wrote the manuscript. All authors provided feedback on the manuscript.

Corresponding author

Correspondence to Eva L. Dyer.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dyer, E.L., Gheshlaghi Azar, M., Perich, M.G. et al. A cryptography-based approach for movement decoding. Nat Biomed Eng 1, 967–976 (2017). https://doi.org/10.1038/s41551-017-0169-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41551-017-0169-7

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing