Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Letter
  • Published:

Reach and grasp by people with tetraplegia using a neurally controlled robotic arm

Nature volume 485, pages 372–375 (2012)Cite this article

Subjects

Abstract

Paralysis following spinal cord injury, brainstem stroke, amyotrophic lateral sclerosis and other disorders can disconnect the brain from the body, eliminating the ability to perform volitional movements. A neural interface system1,2,3,4,5 could restore mobility and independence for people with paralysis by translating neuronal activity directly into control signals for assistive devices. We have previously shown that people with long-standing tetraplegia can use a neural interface system to move and click a computer cursor and to control physical devices6,7,8. Able-bodied monkeys have used a neural interface system to control a robotic arm9, but it is unknown whether people with profound upper extremity paralysis or limb loss could use cortical neuronal ensemble signals to direct useful arm actions. Here we demonstrate the ability of two people with long-standing tetraplegia to use neural interface system-based control of a robotic arm to perform three-dimensional reach and grasp movements. Participants controlled the arm and hand over a broad space without explicit training, using signals decoded from a small, local population of motor cortex (MI) neurons recorded from a 96-channel microelectrode array. One of the study participants, implanted with the sensor 5 years earlier, also used a robotic arm to drink coffee from a bottle. Although robotic reach and grasp actions were not as fast or accurate as those of an able-bodied person, our results demonstrate the feasibility for people with tetraplegia, years after injury to the central nervous system, to recreate useful multidimensional control of complex devices directly from a small sample of neural signals.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Experimental setup and performance metrics.
Figure 2: Participant S3 drinking from a bottle using the DLR robotic arm.
Figure 3: Examples of neural signals from three sessions and two participants.

Similar content being viewed by others

References

  1. Donoghue, J. P. Bridging the brain to the world: a perspective on neural interface systems. Neuron 60, 511–521 (2008)

    Article  CAS  Google Scholar 

  2. Gilja, V. et al. Challenges and opportunities for next-generation intra-cortically based neural prostheses. IEEE Trans. Biomed. Eng. 58, 1891–1899 (2011)

    Article  Google Scholar 

  3. Schwartz, A. B., Cui, X. T., Weber, D. J. & Moran, D. W. Brain-controlled interfaces: movement restoration with neural prosthetics. Neuron 52, 205–220 (2006)

    Article  CAS  Google Scholar 

  4. Nicolelis, M. A. L. & Lebedev, M. A. Principles of neural ensemble physiology underlying the operation of brain-machine interfaces. Nature Rev. Neurosci. 10, 530–540 (2009)

    Article  CAS  Google Scholar 

  5. Green, A. M. & Kalaska, J. F. Learning to move machines with the mind. Trends Neurosci. 34, 61–75 (2011)

    Article  CAS  Google Scholar 

  6. Hochberg, L. R. et al. Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature 442, 164–171 (2006)

    Article  ADS  CAS  Google Scholar 

  7. Simeral, J. D., Kim, S. P., Black, M. J., Donoghue, J. P. & Hochberg, L. R. Neural control of cursor trajectory and click by a human with tetraplegia 1000 days after implant of an intracortical microelectrode array. J. Neural Eng. 8, 025027 (2011)

    Article  ADS  CAS  Google Scholar 

  8. Kim, S. P. et al. Point-and-click cursor control with an intracortical neural interface system by humans with tetraplegia. IEEE Trans. Neural Syst. Rehabil. Eng. 19, 193–203 (2011)

    Article  Google Scholar 

  9. Velliste, M., Perel, S., Spalding, M. C., Whitford, A. S. & Schwartz, A. B. Cortical control of a prosthetic arm for self-feeding. Nature 453, 1098–1101 (2008)

    Article  ADS  CAS  Google Scholar 

  10. Albu-Schaffer, A. et al. The DLR lightweight robot: design and control concepts for robots in human environments. Ind. Rob. 34, 376–385 (2007)

    Article  Google Scholar 

  11. Resnik, L. Research update: VA study to optimize the DEKA Arm. J. Rehabil. Res. Dev. 47, ix–x (2010)

  12. Wu, W., Gao, Y., Bienenstock, E., Donoghue, J. P. & Black, M. J. Bayesian population decoding of motor cortical activity using a Kalman filter. Neural Comput. 18, 80–118 (2006)

    Article  MathSciNet  Google Scholar 

  13. Suner, S., Fellows, M. R., Vargas-Irwin, C., Nakata, G. K. & Donoghue, J. P. Reliability of signals from a chronically implanted, silicon-based electrode array in non-human primate primary motor cortex. IEEE Trans. Neural Syst. Rehabil. Eng. 13, 524–541 (2005)

    Article  Google Scholar 

  14. Chestek, C. A. et al. Long-term stability of neural prosthetic control signals from silicon cortical arrays in rhesus macaque motor cortex. J. Neural. Eng. 8, 045005 (2011)

    Article  ADS  Google Scholar 

  15. Kruger, J., Caruana, F., Volta, R. D. & Rizzolatti, G. Seven years of recording from monkey cortex with a chronically implanted multiple microelectrode. Front. Neuroeng. 3, 6 (2010)

    PubMed  PubMed Central  Google Scholar 

  16. Kim, S. P., Simeral, J. D., Hochberg, L. R., Donoghue, J. P. & Black, M. J. Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia. J. Neural Eng. 5, 455–476 (2008)

    Article  ADS  Google Scholar 

  17. Burrow, M., Dugger, J., Humphrey, D. R., Reed, D. J. & Hochberg, L. R. in Proc. ICORR ’97: Int. Conf. Rehabilitation Robotics 83–86 (Bath Institute of Medical Engineering, 1997)

    Google Scholar 

  18. Shin, H. C., Aggarwal, V., Acharya, S., Schieber, M. H. & Thakor, N. V. Neural decoding of finger movements using Skellam-based maximum-likelihood decoding. IEEE Trans. Biomed. Eng. 57, 754–760 (2010)

    Article  Google Scholar 

  19. Vargas-Irwin, C. E. et al. Decoding complete reach and grasp actions from local primary motor cortex populations. J. Neurosci. 30, 9659–9669 (2010)

    Article  CAS  Google Scholar 

  20. Mehring, C. et al. Inference of hand movements from local field potentials in monkey motor cortex. Nat. Neurosci. 6, 1253–1254 (2003)

    Article  CAS  Google Scholar 

  21. Stark, E. & Abeles, M. Predicting movement from multiunit activity. J. Neurosci. 27, 8387–8394 (2007)

    Article  CAS  Google Scholar 

  22. Bansal, A. K., Vargas-Irwin, C. E., Truccolo, W. & Donoghue, J. P. Relationships among low-frequency local field potentials, spiking activity, and three-dimensional reach and grasp kinematics in primary motor and ventral premotor cortices. J. Neurophysiol. 105, 1603–1619 (2011)

    Article  Google Scholar 

  23. Musallam, S., Corneil, B. D., Greger, B., Scherberger, H. & Andersen, R. A. Cognitive control signals for neural prosthetics. Science 305, 258–262 (2004)

    Article  ADS  CAS  Google Scholar 

  24. Mulliken, G. H., Musallam, S. & Andersen, R. A. Decoding trajectories from posterior parietal cortex ensembles. J. Neurosci. 28, 12913–12926 (2008)

    Article  CAS  Google Scholar 

  25. Santhanam, G., Ryu, S. I., Yu, B. M., Afshar, A. & Shenoy, K. V. A high-performance brain–computer interface. Nature 442, 195–198 (2006)

    Article  ADS  CAS  Google Scholar 

  26. Moritz, C. T., Perlmutter, S. I. & Fetz, E. E. Direct control of paralysed muscles by cortical neurons. Nature 456, 639–642 (2008)

    Article  ADS  CAS  Google Scholar 

  27. Pohlmeyer, E. A. et al. Toward the restoration of hand use to a paralyzed monkey: brain-controlled functional electrical stimulation of forearm muscles. PLoS One 4, e5924 (2009)

    Article  ADS  Google Scholar 

  28. Chadwick, E. K. et al. Continuous neuronal ensemble control of simulated arm reaching by a human with tetraplegia. J. Neural Eng. 8, 034003 (2011)

    Article  ADS  CAS  Google Scholar 

  29. Kuiken, T. A. et al. Targeted reinnervation for enhanced prosthetic arm function in a woman with a proximal amputation: a case study. Lancet 369, 371–380 (2007)

    Article  Google Scholar 

  30. Fraser, G. W., Chase, S. M., Whitford, A. & Schwartz, A. B. Control of a brain–computer interface without spike sorting. J. Neural Eng. 6, 055004 (2009)

    Article  ADS  Google Scholar 

  31. Kim, S. P. et al. Multi-state decoding of point-and-click control signals from motor cortical activity in a human with tetraplegia. 3rd Int. IEEE/EMBS Conf. Neural Eng. 486–489 (2007).

  32. Albu-Schaffer, A. et al. Soft robotics: from torque feedback controlled light-weight robots to intrinsically compliant systems. Robot. Automat. Mag. 15, 20–30 (2008)

    Article  Google Scholar 

  33. Liu, H. et al. Multisensory five-finger dexterous hand: The DLR/HIT Hand II. IEEE/RSJ Int. Conf. Intell. Robots Systems 3692–3697 (2008).

  34. Haddadin, S., Albu-Schaeffer, A. & Hirzinger, G. Requirements for safe robots: measurements, analysis and new insights. Int. J. Robot. Res. 28, 1507–1527 (2009)

    Article  Google Scholar 

  35. Quian Quiroga, R. What is the real shape of extracellular spikes? J. Neurosci. Methods 177, 194–198 (2009)

    Article  CAS  Google Scholar 

  36. Malik, W. Q., Truccolo, W., Brown, E. N. & Hochberg, L. R. Efficient decoding with steady-state Kalman filter in neural interface systems. IEEE Trans. Neural Syst. Rehabil. Eng. 19, 25–34 (2011)

    Article  Google Scholar 

  37. Taylor, D. M., Tillery, S. I. & Schwartz, A. B. Direct cortical control of 3D neuroprosthetic devices. Science 296, 1829–1832 (2002)

    Article  ADS  CAS  Google Scholar 

  38. Jarosiewicz, B. et al. Functional network reorganization during learning in a brain-computer interface paradigm. Proc. Natl Acad. Sci. USA 105, 19486–19491 (2008)

    Article  ADS  CAS  Google Scholar 

Download references

Acknowledgements

We thank participants S3 and T2 for their dedication to this research. We thank M. Black for initial guidance in the BrainGate–DLR research. We thank E. Gallivan, E. Berhanu, D. Rosler, L. Barefoot, K. Centrella and B. King for their contributions to this research. We thank G. Friehs and E. Eskandar for their surgical contributions. We thank K. Knoper for assistance with illustrations. We thank D. Van Der Merwe and DEKA Research and Development for their technical support. The contents do not represent the views of the Department of Veterans Affairs or the United States Government. The research was supported by the Rehabilitation Research and Development Service, Office of Research and Development, Department of Veterans Affairs (Merit Review Awards B6453R and A6779I; Career Development Transition Award B6310N). Support was also provided by the National Institutes of Health: NINDS/NICHD (RC1HD063931), NIDCD (R01DC009899), NICHD-NCMRR (N01HD53403 and N01HD10018), NIBIB (R01EB007401), NINDS-Javits (NS25074); a Memorandum of Agreement between the Defense Advanced Research Projects Agency (DARPA) and the Department of Veterans Affairs; the Doris Duke Charitable Foundation; the MGH-Deane Institute for Integrated Research on Atrial Fibrillation and Stroke; Katie Samson Foundation; Craig H. Neilsen Foundation; the European Commission’s Seventh Framework Programme through the project The Hand Embodied (grant 248587). The pilot clinical trial into which participant S3 was recruited was sponsored in part by Cyberkinetics Neurotechnology Systems (CKI).

Author information

Author notes

  1. Daniel Bacher, Beata Jarosiewicz, Nicolas Y. Masse, John D. Simeral and Joern Vogel: These authors contributed equally to this work.

Authors and Affiliations

  1. Department of Veterans Affairs, Rehabilitation Research & Development Service, Providence, 02908, Rhode Island, USA

    Leigh R. Hochberg, Beata Jarosiewicz, John D. Simeral, Jie Liu & John P. Donoghue

  2. School of Engineering and Institute for Brain Science, Brown University, Providence, 02912, Rhode Island, USA

    Leigh R. Hochberg, Daniel Bacher, John D. Simeral, Jie Liu & John P. Donoghue

  3. Department of Neurology, Massachusetts General Hospital, Boston, 02114, Massachusetts, USA

    Leigh R. Hochberg, John D. Simeral & Sydney S. Cash

  4. Harvard Medical School, Boston, 02115, Massachusetts, USA

    Leigh R. Hochberg & Sydney S. Cash

  5. Department of Neuroscience and Institute for Brain Science, Brown University, Providence, 02912, Rhode Island, USA

    Beata Jarosiewicz, Nicolas Y. Masse & John P. Donoghue

  6. German Aerospace Center, Institute of Robotics and Mechatronics (DLR, Oberpfaffenhofen) 82230, Germany

    Joern Vogel, Sami Haddadin & Patrick van der Smagt

Authors
  1. Leigh R. Hochberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel Bacher
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Beata Jarosiewicz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nicolas Y. Masse
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. John D. Simeral
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Joern Vogel
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Sami Haddadin
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jie Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Sydney S. Cash
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Patrick van der Smagt
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. John P. Donoghue
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

J.P.D. and L.R.H. conceived, planned and directed the BrainGate research and the DEKA sessions. J.P.D., L.R.H. and P.v.d.S. conceived, planned and directed the DLR robot control sessions. J.P.D. and P.v.d.S. are co-senior authors. D.B., B.J., N.Y.M., J.D.S. and J.V. contributed equally to this work and are listed alphabetically. J.D.S., J.V. and D.B. developed the BrainGate–DLR interface. D.B., J.D.S. and J.L. developed the BrainGate–DEKA interface. D.B. and J.V. created the three-dimensional motorized target placement system. B.J., N.Y.M. and D.B. designed the behavioural task, the neural signal processing approach, the filter building approach and the performance metrics. B.J., N.Y.M. and D.B. performed data analysis, further guided by L.R.H., J.D.S. and J.P.D. N.Y.M., L.R.H. and J.P.D. drafted the manuscript, which was further edited by all authors. D.B. and J.D.S. engineered the BrainGate neural interface system/assistive technology system. J.V. and S.H. developed the reactive planner for the Light-Weight Robot III (LWR). S.H. developed the internal control framework of the Light-Weight Robot III. The internal control framework of the DEKA arm was developed by DEKA. L.R.H. is principal investigator of the pilot clinical trial. S.S.C. is clinical co-investigator of the pilot clinical trial and assisted in the clinical oversight of these participants.

Corresponding authors

Correspondence to Leigh R. Hochberg or John P. Donoghue.

Ethics declarations

Competing interests

J.P.D. is a former chief scientific officer and director of CKI; he held stocks and received compensation. L.R.H. received research support from Massachusetts General and Spaulding Rehabilitation Hospitals, which in turn received clinical trial support from CKI. J.D.S. received compensation as a consultant for CKI. CKI ceased operations in 2009, before the collection of data reported in this manuscript. The BrainGate pilot clinical trials are now administered by Massachusetts General Hospital.

Supplementary information

Supplementary Information

This file contains Supplementary Figures 1-9 and Supplementary Table 1 providing additional results, a Supplementary Discussion regarding multidimensional robotic and prosthetic limb control and neural signals, Supplementary Movie Legends demonstrating neural control of reach and grasp by people with tetraplegia and Supplementary References. (PDF 17887 kb)

Supplementary Movie 1

This file contains Supplementary Movie 1 which shows neuronal ensemble control of the DLR robot arm and hand for three-dimensional reach and grasp by a woman with tetraplegia, trial day 1959. (MOV 24495 kb)

Supplementary Movie 2

This file contains Supplementary Movie 2 which shows neuronal ensemble control of the DEKA prosthetic arm and hand by a woman with tetraplegia, trial day 1974. (MOV 25133 kb)

Supplementary Movie 3

This file contains Supplementary Movie 3 which shows neuronal ensemble control of the DEKA prosthetic arm and hand by a gentleman with tetraplegia (Participant T2), trial day 166. (MOV 25604 kb)

Supplementary Movie 4

This file contains Supplementary Movie 4 which shows BrainGate-enabled drinking from a bottle by S3, using neurally-controlled 2-D movement and hand state control of the DLR robot arm, trial day 1959. (MOV 24748 kb)

PowerPoint slides

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hochberg, L., Bacher, D., Jarosiewicz, B. et al. Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature 485, 372–375 (2012). https://doi.org/10.1038/nature11076

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nature11076

This article is cited by

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing