Stabilization of a brain–computer interface via the alignment of low-dimensional spaces of neural activity

Abstract

The instability of neural recordings can render clinical brain–computer interfaces (BCIs) uncontrollable. Here, we show that the alignment of low-dimensional neural manifolds (low-dimensional spaces that describe specific correlation patterns between neurons) can be used to stabilize neural activity, thereby maintaining BCI performance in the presence of recording instabilities. We evaluated the stabilizer with non-human primates during online cursor control via intracortical BCIs in the presence of severe and abrupt recording instabilities. The stabilized BCIs recovered proficient control under different instability conditions and across multiple days. The stabilizer does not require knowledge of user intent and can outperform supervised recalibration. It stabilized BCIs even when neural activity contained little information about the direction of cursor movement. The stabilizer may be applicable to other neural interfaces and may improve the clinical viability of BCIs.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: Stabilized BCI framework.
Fig. 2: Manifold-based stabilization intuition and design.
Fig. 3: Examples of neural recording instabilities.
Fig. 4: Representative experimental session.
Fig. 5: Summary of the single-day experimental sessions.
Fig. 6: Manifold-based stabilization restores performance in the presence of instabilities.
Fig. 7: Manifold-based stabilization maintains performance across multiple days.
Fig. 8: Manifold-based stabilization can outperform supervised recalibration.

Data availability

The main data supporting the results in this study are available within the paper and its Supplementary Information. Experimental data for the stabilization of brain–computer interfaces are available at https://github.com/alandegenhart/stabilizedbci.

Code availability

The MATLAB code for the stabilization of brain–computer interfaces is available at https://github.com/alandegenhart/stabilizedbci.

References

  1. 1.

    Collinger, J. L. et al. High-performance neuroprosthetic control by an individual with tetraplegia. Lancet 381, 557–564 (2012).

  2. 2.

    Hochberg, L. R. et al. Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature 485, 372–375 (2012).

  3. 3.

    Bouton, C. E. et al. Restoring cortical control of functional movement in a human with quadriplegia. Nature 533, 247–250 (2016).

  4. 4.

    Ajiboye, A. B. et al. Restoration of reaching and grasping movements through brain-controlled muscle stimulation in a person with tetraplegia: a proof-of-concept demonstration. Lancet 389, 1821–1830 (2017).

  5. 5.

    Pandarinath, C. et al. High performance communication by people with paralysis using an intracortical brain–computer interface. eLife 6, e18554 (2017).

  6. 6.

    Sakellaridi, S. et al. Intrinsic variable learning for brain–machine interface control by human anterior intraparietal cortex. Neuron 102, 694–705 (2019).

  7. 7.

    Perge, J. A. et al. Intra-day signal instabilities affect decoding performance in an intracortical neural interface system. J. Neural Eng. 10, 036004 (2013).

  8. 8.

    Turner, J. N. et al. Cerebral astrocyte response to micromachined silicon implants. Exp. Neurol. 156, 33–49 (1999).

  9. 9.

    Biran, R., Martin, D. C. & Tresco, P. A. Neuronal cell loss accompanies the brain tissue response to chronically implanted silicon microelectrode arrays. Exp. Neurol. 195, 115–126 (2005).

  10. 10.

    Moffitt, M. A. & McIntyre, C. C. Model-based analysis of cortical recording with silicon microelectrodes. Clin. Neurophysiol. 116, 2240–2250 (2005).

  11. 11.

    McConnell, G. C. et al. Implanted neural electrodes cause chronic, local inflammation that is correlated with local neurodegeneration. J. Neural Eng. 6, 056003 (2009).

  12. 12.

    Nuyujukian, P. et al. Performance sustaining intracortical neural prostheses. J. Neural Eng. 11, 066003 (2014).

  13. 13.

    Jarosiewicz, B. et al. Virtual typing by people with tetraplegia using a self-calibrating intracortical brain–computer interface. Sci. Transl. Med. 7, 313ra179 (2015).

  14. 14.

    Downey, J. E., Schwed, N., Chase, S. M., Schwartz, A. B. & Collinger, J. L. Intracortical recording stability in human brain–computer interface users. J. Neural Eng. 15, 046016 (2018).

  15. 15.

    Li, Z., O’Doherty, J. E., Lebedev, M. A. & Nicolelis, M. A. L. Adaptive decoding for brain–machine interfaces through Bayesian parameter updates. Neural Comput. 23, 3162–3204 (2011).

  16. 16.

    Zhang, Y. & Chase, S. M. A stabilized dual Kalman filter for adaptive tracking of brain–computer interface decoding parameters. In 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) 7100–7103 (IEEE, 2013).

  17. 17.

    Bishop, W. et al. Self-recalibrating classifiers for intracortical brain–computer interfaces. J. Neural Eng. 11, 026001 (2014).

  18. 18.

    Homer, M. L. et al. Adaptive offset correction for intracortical brain–computer interfaces. IEEE Trans. Neural Syst. Rehab. Eng. 22, 239–248 (2014).

  19. 19.

    Sussillo, D., Stavisky, S. D., Kao, J. C., Ryu, S. I. & Shenoy, K. V. Making brain–machine interfaces robust to future neural variability. Nat. Commun. 7, 13749 (2016).

  20. 20.

    Yu, B. et al. Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity. J. Neurophysiol. 102, 614–635 (2009).

  21. 21.

    Cunningham, J. P. & Yu, B. M. Dimensionality reduction for large-scale neural recordings. Nat. Neurosci. 17, 1500–1509 (2014).

  22. 22.

    Gao, P. & Ganguli, S. On simplicity and complexity in the brave new world of large-scale neuroscience. Curr. Opin. Neurobiol. 32, 148–155 (2015).

  23. 23.

    Gallego, J. A., Perich, M. G., Miller, L. E. & Solla, S. A. Neural manifolds for the control of movement. Neuron 94, 978–984 (2017).

  24. 24.

    Luczak, A., Barthó, P. & Harris, K. D. Spontaneous events outline the realm of possible sensory responses in neocortical populations. Neuron 62, 413–425 (2009).

  25. 25.

    Sadtler, P. T. et al. Neural constraints on learning. Nature 512, 423–426 (2014).

  26. 26.

    Degenhart, A. D. et al. Self-Recalibrating Brain-Computer Interfaces based on Population Subspace Alignment Abstr. 334.13 (Society of Neuroscience, 2016).

  27. 27.

    Bishop, W. E. et al. Extracting Stable Representations of Neural Population State from Unstable Neural Recordings (COSYNE, 2017).

  28. 28.

    Chase, S. M. Neural manifolds: from basic science to practical improvements in brain-computer interfaces. In IEEE 7th International Winter Conference on Brain-Computer Interface (BCI) 1–2 (IEEE, 2019).

  29. 29.

    Santhanam, G. et al. Factor-analysis methods for higher-performance neural prostheses. J. Neurophysiol. 102, 1315–1330 (2009).

  30. 30.

    Wu, W., Gao, Y., Bienenstock, E., Donoghue, J. P. & Black, M. J. Bayesian population decoding of motor cortical activity using a Kalman filter. Neural Comput. 18, 80–118 (2006).

  31. 31.

    Fraser, G. W. & Schwartz, A. B. Recording from the same neurons chronically in motor cortex. J. Neurophysiol. 107, 1970–1978 (2012).

  32. 32.

    Pandarinath, C. et al. Inferring single-trial neural population dynamics using sequential auto-encoders. Nat. Methods 15, 805–815 (2018).

  33. 33.

    Gallego, J. A., Perich, M. G., Chowdhury, R. H., Solla, S. A. & Miller, L. E. Long-term stability of cortical population dynamics underlying consistent behavior. Nat. Neurosci. 23, 260–270 (2020).

  34. 34.

    Chestek, C. A. et al. Single-neuron stability during repeated reaching in macaque premotor cortex. J. Neurosci. 27, 10742–10750 (2007).

  35. 35.

    Stevenson, I. H. et al. Statistical assessment of the stability of neural movement representations. J. Neurophysiol. 106, 764–774 (2011).

  36. 36.

    Flint, R. D., Scheid, M. R., Wright, Z. A., Solla, S. A. & Slutzky, M. W. Long-term stability of motor cortical activity: implications for brain machine interfaces and optimal feedback control. J. Neurosci. 36, 3623–3632 (2016).

  37. 37.

    Ruff, D. A., Ni, A. M. & Cohen, M. R. Cognition as a window into neuronal population space. Annu. Rev. Neurosci. 41, 77–97 (2018).

  38. 38.

    Tolias, A. S. et al. Recording chronically from the same neurons in awake, behaving primates. J. Neurophysiol. 98, 3780–3790 (2007).

  39. 39.

    Dickey, A. S., Suminski, A., Amit, Y. & Hatsopoulos, N. G. Single-unit stability using chronically implanted multielectrode arrays. J. Neurophysiol. 102, 1331–1339 (2009).

  40. 40.

    Cunningham, J. P. & Ghahramani, Z. Linear dimensionality reduction: survey, insights, and generalizations. J. Mach. Learn. Res. 16, 2859–2900 (2015).

  41. 41.

    Gilja, V. et al. A high-performance neural prosthesis enabled by control algorithm design. Nat. Neurosci. 15, 1752–1757 (2012).

  42. 42.

    Chase, S. M., Schwartz, A. B. & Kass, R. E. Bias, optimal linear estimation, and the differences between open-loop simulation and closed-loop performance of spiking-based brain–computer interface algorithms. Neural Netw. 22, 1203–1213 (2009).

  43. 43.

    Carmena, J. M. et al. Learning to control a brain–machine interface for reaching and grasping by primates. PLoS Biol. 1, E42 (2003).

  44. 44.

    So, K., Dangi, S., Orsborn, A. L., Gastpar, M. C. & Carmena, J. M. Subject-specific modulation of local field potential spectral power during brain–machine interface control in primates. J. Neural Eng. 11, 026002 (2014).

  45. 45.

    Leuthardt, E. C., Schalk, G., Wolpaw, J. R., Ojemann, J. G. & Moran, D. W. A brain–computer interface using electrocorticographic signals in humans. J. Neural Eng. 1, 63–71 (2004).

  46. 46.

    Degenhart, A. D. et al. Remapping cortical modulation for electrocorticographic brain–computer interfaces: a somatotopy-based approach in individuals with upper-limb paralysis. J. Neural Eng. 15, 026021 (2018).

  47. 47.

    Wolpaw, J. R. & McFarland, D. J. Control of a two-dimensional movement signal by a noninvasive brain–computer interface in humans. Proc. Natl Acad. Sci. USA 101, 17849–17854 (2004).

  48. 48.

    Picton, T. W. & Hillyard, S. A. Cephalic skin potentials in electroencephalography. Electroencephalogr. Clin. Neurophysiol. 33, 419–424 (1972).

  49. 49.

    Degenhart, A. D. et al. Histological evaluation of a chronically-implanted electrocorticographic electrode grid in a non-human primate. J. Neural Eng. 13, 046019 (2016).

  50. 50.

    Miller, K. J. et al. Spectral changes in cortical surface potentials during motor movement. J. Neurosci. 27, 2424–2432 (2007).

  51. 51.

    Siems, M., Pape, A.-A., Hipp, J. F. & Siegel, M. Measuring the cortical correlation structure of spontaneous oscillatory activity with EEG and MEG. NeuroImage 129, 345–355 (2016).

  52. 52.

    Piccione, F. et al. P300-based brain computer interface: reliability and performance in healthy and paralysed participants. Clin. Neurophysiol. 117, 531–537 (2006).

  53. 53.

    Gunduz, A., Ozturk, M., Sanchez, J. & Principe, J. Echo state networks for motor control of human ECoG neuroprosthetics. In 3rd International IEEE/EMBS Conference on Neural Engineering 514–517 (IEEE, 2007).

  54. 54.

    Kim, J.-H., Bießmann, F. & Lee, S.-W. Decoding three-dimensional trajectory of executed and imagined arm movements from electroencephalogram signals. IEEE Trans. Neural Syst. Rehab. Eng. 23, 867–876 (2015).

  55. 55.

    Bishop, W. E. & Yu, S. M. Deterministic symmetric positive semidefinite matrix completion. Adv. Neural Inf. Process. Syst. 27, 2762–2770 (2014).

  56. 56.

    Cowley, B. R., Smith, M. A., Kohn, A. & Yu, B. M. Stimulus-driven population activity patterns in macaque primary visual cortex. PLoS Comput. Biol. 12, e1005185 (2016).

  57. 57.

    Wodlinger, B. et al. Ten-dimensional anthropomorphic arm control in a human brain–machine interface: difficulties, solutions, and limitations. J. Neural Eng. 12, 016011 (2014).

  58. 58.

    Fiser, J., Chiu, C. & Weliky, M. Small modulation of ongoing cortical dynamics by sensory input during natural vision. Nature 431, 573–578 (2004).

  59. 59.

    Berkes, P., Orbán, G., Lengyel, M. & Fiser, J. Spontaneous cortical activity reveals hallmarks of an optimal internal model of the environment. Science 331, 83–87 (2011).

  60. 60.

    Kiani, R. et al. Natural grouping of neural responses reveals spatially segregated clusters in prearcuate cortex. Neuron 85, 1359–1373 (2015).

  61. 61.

    Gallego, J. A. et al. Cortical population activity within a preserved neural manifold underlies multiple motor behaviors. Nat. Commun. 9, 4233 (2018).

  62. 62.

    Oby, E. R. et al. New neural activity patterns emerge with long-term learning. Proc. Natl Acad. Sci. USA 7, 201820296 (2019).

  63. 63.

    Zhou, X., Tien, R. N., Ravikumar, S. & Chase, S. M. Distinct types of neural reorganization during long-term learning. J. Neurophysiol. 121, 1329–1341 (2019).

  64. 64.

    Golub, M. D. et al. Learning by neural reassociation. Nat. Neurosci. 21, 607–616 (2018).

  65. 65.

    Shenoy, K. V. & Carmena, J. M. Combining decoder design and neural adaptation in brain–machine interfaces. Neuron 84, 665–680 (2014).

  66. 66.

    Zhang, Y. & Chase, S. M. Optimizing the usability of brain–computer interfaces. Neural Comput. 30, 1323–1358 (2018).

  67. 67.

    Dyer, E. L. et al. A cryptography-based approach for movement decoding. Nat. Biomed. Eng. 1, 967–976 (2017).

  68. 68.

    Kao, J. C., Ryu, S. I. & Shenoy, K. V. Leveraging neural dynamics to extend functional lifetime of brain–machine interfaces. Sci. Rep. 7, 7395 (2017).

  69. 69.

    Hamilton, L. S., Edwards, E. & Chang, E. F. A spatial map of onset and sustained responses to speech in the human superior temporal gyrus. Curr. Biol. 28, 1860–1871 (2018).

  70. 70.

    Riva-Posse, P. et al. A connectomic approach for subcallosal cingulate deep brain stimulation surgery: prospective targeting in treatment-resistant depression. Mol. Psychiatry 23, 843–849 (2018).

  71. 71.

    Williamson, R. C. et al. Scaling properties of dimensionality reduction for neural populations and network models. PLoS Comput. Biol. 12, e1005141 (2016).

  72. 72.

    Dempster, A. P., Laird, N. M. & Rubin, D. B. Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. B 39, 1–38 (1977).

  73. 73.

    Schönemann, P. H. A generalized solution of the orthogonal procrustes problem. Psychometrika 31, 1–10 (1966).

  74. 74.

    Kao, J. C. et al. Single-trial dynamics of motor cortex and their applications to brain–machine interfaces. Nat. Commun. 6, 7759 (2015).

  75. 75.

    Tkach, D., Reimer, J. & Hatsopoulos, N. G. Congruent activity during action and action observation in motor cortex. J. Neurosci. 27, 13241–13250 (2007).

  76. 76.

    Golub, M. D., Yu, B. M. & Chase, S. M. Internal models for interpreting neural population activity during sensorimotor control. eLife 4, e10015 (2015).

Download references

Acknowledgements

This work was supported by the Craig H. Neilsen Foundation (280028 to B.M.Y., S.M.C. and A.P.B.), NIH T32 NS086749 (to A.D.D. and E.R.O.), the DSF Charitable Foundation (132RA03 to A.D.D.), NIH R01 HD071686 (to A.P.B., B.M.Y. and S.M.C.), NSF NCS BCS1533672 (to S.M.C., A.P.B. and B.M.Y.), NIH CRCNS R01 NS105318 (to B.M.Y. and A.P.B.), the PA Department of Health (Research Formula Grant SAP 4100077048 to S.M.C. and B.M.Y.), NSF CAREER Award IOS1553252 (to S.M.C.), NSF NCS BCS1734916 (to B.M.Y.) and the Simons Foundation (364994 and 543065 to B.M.Y.).

Author information

Affiliations

Authors

Contributions

A.D.D., W.E.B., E.R.O., S.M.C., A.P.B. and B.M.Y. designed the experiments and interpreted the results. A.D.D. performed the experiments with input from W.E.B. W.E.B. and B.M.Y. designed the stabilization method. A.D.D. and W.E.B. developed the real-time implementation of the stabilized BCI. A.D.D. and W.E.B. performed the analyses and wrote the manuscript. E.R.O., E.C.T.-K. and A.D.D. implanted the electrode arrays used for the experiments. All authors provided feedback on the manuscript.

Corresponding author

Correspondence to Byron M. Yu.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Figs. 1–8 and reference.

Reporting Summary

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Degenhart, A.D., Bishop, W.E., Oby, E.R. et al. Stabilization of a brain–computer interface via the alignment of low-dimensional spaces of neural activity. Nat Biomed Eng (2020). https://doi.org/10.1038/s41551-020-0542-9

Download citation