A cryptography-based approach for movement decoding

Abstract

Brain decoders use neural recordings to infer the activity or intent of a user. To train a decoder, one generally needs to infer the measured variables of interest (covariates) from simultaneously measured neural activity. However, there are cases for which obtaining supervised data is difficult or impossible. Here, we describe an approach for movement decoding that does not require access to simultaneously measured neural activity and motor outputs. We use the statistics of movement—much like cryptographers use the statistics of language—to find a mapping between neural activity and motor variables, and then align the distribution of decoder outputs with the typical distribution of motor outputs by minimizing their Kullback–Leibler divergence. By using datasets collected from the motor cortex of three non-human primates performing either a reaching task or an isometric force-production task, we show that the performance of such a distribution-alignment decoding algorithm is comparable to the performance of supervised approaches. Distribution-alignment decoding promises to broaden the set of potential applications of brain decoding.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

References

  1. 1.

    Kay, K. N. et al. Identifying natural images from human brain activity. Nature452, 352–355 (2008).

    CAS  Article  Google Scholar 

  2. 2.

    Cowen, A. S. et al. Neural portraits of perception: reconstructing face images from evoked brain activity. Neuroimage94, 12–22 (2014).

    Article  Google Scholar 

  3. 3.

    Mitchell, T. M. et al. Predicting human brain activity associated with the meanings of nouns. Science320, 1191–1195 (2008).

    CAS  Article  Google Scholar 

  4. 4.

    Haynes, J. et al. Decoding mental states from brain activity in humans. Nat. Rev. Neurosci.7, 523–534 (2006).

    CAS  Article  Google Scholar 

  5. 5.

    Serruya, M. D. et al. Brain–machine interface: instant neural control of a movement signal. Nature416, 141–142 (2002).

    CAS  Article  Google Scholar 

  6. 6.

    Kemere, C. et al. Model-based neural decoding of reaching movements: a maximum likelihood approach. IEEE Trans. Biomed. Eng.51, 925–932 (2004).

    Article  Google Scholar 

  7. 7.

    Tkach, D. et al. Observation-based learning for brain–machine interfaces. Curr. Opin. Neurobiol.18, 589–594 (2008).

    CAS  Article  Google Scholar 

  8. 8.

    Anderson, D. J. et al. Toward a science of computational ethology. Neuron84, 18–31 (2014).

    CAS  Article  Google Scholar 

  9. 9.

    Donoghue, J. P. Connecting cortex to machines: recent advances in brain interfaces. Nat. Neurosci.5, 1085–1088 (2002).

    CAS  Article  Google Scholar 

  10. 10.

    Bose, R. Information Theory, Coding and Cryptography (McGraw-Hill Education, New Delhi, India, 2008).

  11. 11.

    Chaudhari, M. P. et al. A survey on cryptography algorithms. Int. J. Adv. Res. Comput. Sci. Manag. Stud.2, 100–104 (2014).

  12. 12.

    Wang, L. et al. Automatic gait recognition based on statistical shape analysis. IEEE Trans. Image Process.12, 1120–1131 (2003).

    Article  Google Scholar 

  13. 13.

    Belić, J. J. et al. Decoding of human hand actions to handle missing limbs in neuroprosthetics. Front. Comput. Neuro. https://doi.org/10.3389/fncom.2015.00027.

  14. 14.

    Ishiduka, S. et al. Kinematic analysis of low dimensional structure in walking and running. In 2015 International Symposium on Micro-NanoMechatronics and Human Science (MHS) 1–6 (IEEE, 2015).

  15. 15.

    Damavsevivcius, R. et al. Smartphone user identity verification using gait characteristics. Symmetry8, 100 (2016).

    Article  Google Scholar 

  16. 16.

    Kaufman, M. T. et al. The largest response component in the motor cortex reflects movement timing but not movement type. eNeuro3, ENEURO–0085 (2016).

    Article  Google Scholar 

  17. 17.

    Marblestone, A. H. et al. Physical principles for scalable neural recording. Front. Comput. Neurosci. https://doi.org/10.3389/fncom.2013.00137.

  18. 18.

    Georgopoulos, A. P. et al. Neuronal population coding of movement direction. Science233, 1416–1419 (1986).

    CAS  Article  Google Scholar 

  19. 19.

    Pillow, J. W. et al. Spatio-temporal correlations and visual signaling in a complete neuronal population. Nature454, 995–999 (2008).

    CAS  Article  Google Scholar 

  20. 20.

    Dyer, E. L. et al. Greedy feature selection for subspace clustering. J. Mach. Learn. Res.14, 2487–2517 (2013).

    Google Scholar 

  21. 21.

    Schölkopf, B. et al. Kernel Methods in Computational Biology (MIT Press, Cambridge, MA, USA, 2004).

  22. 22.

    Ingram, J. N. et al. The statistics of natural hand movements. Exp. Brain Res.188, 223–236 (2008).

    Article  Google Scholar 

  23. 23.

    Ejaz, N. et al. Hand use predicts the structure of representations in sensorimotor cortex. Nat. Neurosci18, 1034–1040 (2015).

    CAS  Article  Google Scholar 

  24. 24.

    Byron, M. Y. et al. Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity. In Advances in Neural Information Processing Systems (eds Koller, D. et al.) 1881–1888 (MIT Press, 2009).

  25. 25.

    Fernandes, H. L. et al. Saliency and saccade encoding in the frontal eye field during natural scene search. Cereb. Cortex24, 3232–3245 (2014).

    Article  Google Scholar 

  26. 26.

    Li, Z. et al. Adaptive decoding for brain–machine interfaces through Bayesian parameter updates. Neural Comput.23, 3162–3204 (2011).

    Article  Google Scholar 

  27. 27.

    Gilja, V. et al. A high-performance neural prosthesis enabled by control algorithm design. Nat. Neurosci.15, 1752–1757 (2012).

    CAS  Article  Google Scholar 

  28. 28.

    Orsborn, A. L. et al. Closed-loop decoder adaptation on intermediate time-scales facilitates rapid bmi performance improvements independent of decoder initialization conditions. IEEE Trans. Neural Syst. Rehabil. Eng.20, 468–477 (2012).

    Article  Google Scholar 

  29. 29.

    Orsborn, A. L. et al. Closed-loop decoder adaptation shapes neural plasticity for skillful neuroprosthetic control. Neuron82, 1380–1393 (2014).

    CAS  Article  Google Scholar 

  30. 30.

    Ashe, J. et al. Movement parameters and neural activity in motor cortex and area 5. Cereb. Cortex4, 590–600 (1994).

    CAS  Article  Google Scholar 

  31. 31.

    Averbeck, B. B. et al. Parietal representation of hand velocity in a copy task. J. Neurophysio.93, 508–518 (2005).

    Article  Google Scholar 

  32. 32.

    Stevenson, I. H. et al. Statistical assessment of the stability of neural movement representations. J. Neurophysio106, 764–774 (2011).

    Article  Google Scholar 

  33. 33.

    Stevenson, I. H. et al. Functional connectivity and tuning curves in populations of simultaneously recorded neurons. PLoS Comput. Biol.8, e1002775 (2012).

    CAS  Article  Google Scholar 

  34. 34.

    Jagabathula, S. et al. Inferring rankings using constrained sensing. IEEE Trans. Inform. Theory57, 7288–7306 (2011).

    Article  Google Scholar 

  35. 35.

    Van Der Maaten, L. et al. Dimensionality reduction: a comparative. J. Mach. Learn. Res.10, 66–71 (2009).

    Google Scholar 

  36. 36.

    Tenenbaum, J. B. et al. A global geometric framework for nonlinear dimensionality reduction. Science290, 2319–2323 (2000).

    CAS  Article  Google Scholar 

  37. 37.

    van der Maaten, L. et al. Visualizing data using t-sne. J. Mach. Learn. Res.9, 2579–2605 (2008).

    Google Scholar 

  38. 38.

    Bishop, C. M. Bayesian PCA. In Advances in Neural Information Processing Systems (eds Kearns, M. J. et al.) 382–388 (MIT Press, 1999).

  39. 39.

    Macke, J. H. et al. Empirical models of spiking in neural populations. In Advances in Neural Information Processing Systems (eds Shawe-Taylor, J. et al.) 1350–1358 (MIT Press, 2011).

  40. 40.

    Póczos, B. et al. On the estimation of alpha-divergences. In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics 609–617 (2011).

  41. 41.

    Loftsgaarden, D. O. et al. A nonparametric estimate of a multivariate density function. Ann. Math. Stat.36, 1049–1051 (1965).

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank B. Dekleva, P. Ramkumar, J. Glaser, D. Acuna, P. Lawlor, S. Saeb, L. Lonini, S. Solla and B. Yu for discussions and advice. This work was supported by NINDS R01 NS053603, NINDS R01 NS074044, U01 MH109100 and R01 NS074044.

Author information

Affiliations

Authors

Contributions

E.L.D., M.G.A. and K.P.K. designed the research. E.L.D. and M.G.A. implemented the algorithms. E.L.D., H.L.F. and M.G.A. tested the decoders on neural recordings and analysed the results. M.G.P. and S.N. collected and sorted the NHP data. K.P.K. and L.M. managed and advised on the project. E.L.D., M.G.A. and K.P.K. wrote the manuscript. All authors provided feedback on the manuscript.

Corresponding author

Correspondence to Eva L. Dyer.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Dyer, E.L., Gheshlaghi Azar, M., Perich, M.G. et al. A cryptography-based approach for movement decoding. Nat Biomed Eng 1, 967–976 (2017). https://doi.org/10.1038/s41551-017-0169-7

Download citation

Further reading

Search

Sign up for the Nature Briefing newsletter for a daily update on COVID-19 science.
Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing