Article | Published:

Neural population partitioning and a concurrent brain-machine interface for sequential motor function

Nature Neuroscience volume 15, pages 17151722 (2012) | Download Citation

Abstract

Although brain-machine interfaces (BMIs) have focused largely on performing single-targeted movements, many natural tasks involve planning a complete sequence of such movements before execution. For these tasks, a BMI that can concurrently decode the full planned sequence before its execution may also consider the higher-level goal of the task to reformulate and perform it more effectively. Using population-wide modeling, we discovered two distinct subpopulations of neurons in the rhesus monkey premotor cortex that allow two planned targets of a sequential movement to be simultaneously held in working memory without degradation. Such marked stability occurred because each subpopulation encoded either only currently held or only newly added target information irrespective of the exact sequence. On the basis of these findings, we developed a BMI that concurrently decodes a full motor sequence in advance of movement and can then accurately execute it as desired.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

References

  1. 1.

    , , & Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex. Nat. Neurosci. 2, 664–670 (1999).

  2. 2.

    et al. Real-time prediction of hand trajectory by ensembles of cortical neurons in primates. Nature 408, 361–365 (2000).

  3. 3.

    , , , & Instant neural control of a movement signal. Nature 416, 141–142 (2002).

  4. 4.

    et al. Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature 442, 164–171 (2006).

  5. 5.

    et al. Learning to control a brain-machine interface for reaching and grasping by primates. PLoS Biol. 1, e42 (2003).

  6. 6.

    , & Direct cortical control of 3D neuroprosthetic devices. Science 296, 1829–1832 (2002).

  7. 7.

    & Emergence of a stable cortical map for neuroprosthetic control. PLoS Biol. 7, e1000153 (2009).

  8. 8.

    & Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. Proc. Natl. Acad. Sci. USA 101, 17849–17854 (2004).

  9. 9.

    , , , & Cortical control of a prosthetic arm for self-feeding. Nature 453, 1098–1101 (2008).

  10. 10.

    , & Direct control of paralysed muscles by cortical neurons. Nature 456, 639–642 (2008).

  11. 11.

    , & Decoding trajectories from posterior parietal cortex ensembles. J. Neurosci. 28, 12913–12926 (2008).

  12. 12.

    , , , & Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia. J. Neural Eng. 5, 455–476 (2008).

  13. 13.

    et al. Unscented Kalman filter for brain-machine interfaces. PLoS ONE 4, e6243 (2009).

  14. 14.

    , & Bias, optimal linear estimation, and the differences between open-loop simulation and closed-loop performance of spiking-based brain-computer interface algorithms. Neural Netw. 22, 1203–1213 (2009).

  15. 15.

    , , , & Cognitive control signals for neural prosthetics. Science 305, 258–262 (2004).

  16. 16.

    , , , & A high-performance brain-computer interface. Nature 442, 195–198 (2006).

  17. 17.

    , , & Feedback-controlled parallel point process filter for estimation of goal-directed movements from neural signals. IEEE Trans. Neural Syst. Rehabil. Eng. published online, doi:10.1109/TNSRE.2012.2221743 (2 October 2012).

  18. 18.

    , , & A brain-machine interface combining target and trajectory information using optimal feedback control. in Computational and Systems Neuroscience (COSYNE) Meeting (Salt Lake City, USA, 2011).

  19. 19.

    Premotor cortex of monkeys: set- and movement-related activity reflecting amplitude and direction of wrist movements. J. Neurophysiol. 69, 187–200 (1993).

  20. 20.

    & Covariation of primate dorsal premotor cell activity with direction and amplitude during a memorized-delay reaching task. J. Neurophysiol. 84, 152–165 (2000).

  21. 21.

    & Modulation of preparatory neuronal activity in dorsal premotor cortex due to stimulus-response compatibility. J. Neurophysiol. 71, 1281–1284 (1994).

  22. 22.

    & Gaze effects in the cerebral cortex: reference frames for space coding and action. Exp. Brain Res. 128, 170–180 (1999).

  23. 23.

    & Differential relation of discharge in primary motor cortex and premotor cortex to movements versus actively maintained postures during a reaching task. Exp. Brain Res. 108, 45–61 (1996).

  24. 24.

    & Prior information in motor and premotor cortex: activity during the delay period and effect on pre-movement activity. J. Neurophysiol. 84, 986–1005 (2000).

  25. 25.

    , , & Target-, limb-, and context-dependent neural activity in the cingulate and supplementary motor areas of the monkey. Exp. Brain Res. 158, 278–288 (2004).

  26. 26.

    & Effects of hand movement path on motor cortical activity in awake, behaving rhesus monkeys. Exp. Brain Res. 83, 285–302 (1991).

  27. 27.

    & The parietal reach region codes the next planned movement in a sequential reach task. J. Neurophysiol. 85, 539–544 (2001).

  28. 28.

    , & Representation of the temporal order of visual objects in the primate lateral prefrontal cortex. J. Neurophysiol. 89, 2868–2873 (2003).

  29. 29.

    & Role for supplementary motor area cells in planning several movements ahead. Nature 371, 413–416 (1994).

  30. 30.

    , , & Categorization of behavioural sequences in the prefrontal cortex. Nature 445, 315–318 (2007).

  31. 31.

    & Neuronal activity in the supplementary and presupplementary motor areas for temporal organization of multiple movements. J. Neurophysiol. 84, 2148–2160 (2000).

  32. 32.

    , & The posterior parietal cortex encodes in parallel both goals for double-reach sequences. J. Neurosci. 28, 10081–10089 (2008).

  33. 33.

    , & Activity in prefrontal cortex during dynamic selection of action sequences. Nat. Neurosci. 9, 276–282 (2006).

  34. 34.

    , , , & Activity in the lateral prefrontal cortex reflects multiple steps of future events in action plans. Neuron 50, 631–641 (2006).

  35. 35.

    , & Conversion of working memory to motor sequence in the monkey premotor cortex. Science 301, 233–236 (2003).

  36. 36.

    , & Control of remembered reaching sequences in monkey. II. Storage and preparation before movement in motor and premotor cortex. Exp. Brain Res. 112, 347–358 (1996).

  37. 37.

    & Anticipatory activity in primary motor cortex codes memorized movement sequences. Neuron 45, 967–973 (2005).

  38. 38.

    , , & Covert representation of second-next movement in the pre-supplementary motor area of monkeys. J. Neurophysiol. 101, 1883–1889 (2009).

  39. 39.

    , , & Parallel processing of serial movements in prefrontal cortex. Proc. Natl. Acad. Sci. USA 99, 13172–13177 (2002).

  40. 40.

    , , , & Representation of immediate and final behavioral goals in the monkey prefrontal cortex during an instructed delay period. Cereb. Cortex 15, 1535–1546 (2005).

  41. 41.

    , & Selective coding of motor sequence in the supplementary motor area of the monkey cerebral cortex. Exp. Brain Res. 82, 208–210 (1990).

  42. 42.

    et al. State-space algorithms for estimating spike rate functions. Comput. Intell. Neurosci. published online, doi:10.1155/2010/426539 (5 November 2009).

  43. 43.

    Sequential organization of multiple movements: involvement of cortical motor areas. Annu. Rev. Neurosci. 24, 631–651 (2001).

  44. 44.

    & Neural correlates of reaching decisions in dorsal premotor cortex: specification of multiple direction choices and final selection of action. Neuron 45, 801–814 (2005).

  45. 45.

    & Integration of target and body-part information in the premotor cortex when planning action. Nature 408, 466–470 (2000).

  46. 46.

    & Contrasting neuronal activity in the dorsal and ventral premotor areas during preparation to reach. J. Neurophysiol. 87, 1123–1128 (2002).

  47. 47.

    & Differential involvement of neurons in the dorsal and ventral premotor cortex during processing of visual signals for action planning. J. Neurophysiol. 95, 3596–3616 (2006).

  48. 48.

    & Both supplementary and presupplementary motor areas are crucial for the temporal organization of multiple movements. J. Neurophysiol. 80, 3247–3260 (1998).

  49. 49.

    , , & Likelihood methods for neural data analysis. in Computational Neuroscience: A Comprehensive Approach (ed. Feng, J.) 253–286 (CRC Press, 2003).

  50. 50.

    , , , & A point process framework for relating neural spiking activity to spiking history, neural ensemble, and extrinsic covariate effects. J. Neurophysiol. 93, 1074–1089 (2005).

  51. 51.

    & Estimating a state-space model from point process observations. Neural Comput. 15, 965–991 (2003).

  52. 52.

    , & Maximum likelihood from incomplete data via the EM algorithm. J. Royal Stat. Soc., Series B 39, 1–38 (1977).

  53. 53.

    On the exact variance of products. J. Am. Stat. Assoc. 55, 708–713 (1960).

Download references

Acknowledgements

R.C.H. is funded by the Neuroscience Research and Education Foundation, E.N.B. is funded by US National Institutes of Health (NIH) DP1 OD003646, and Z.M.W. is funded by NIH 5R01-HD059852, a Presidential Early Career Award for Scientists and Engineers and the Whitehall Foundation.

Author information

Affiliations

  1. Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA.

    • Maryam M Shanechi
    •  & Gregory W Wornell
  2. Department of Neurosurgery, Massachusetts General Hospital, Boston, Massachusetts, USA.

    • Maryam M Shanechi
    • , Rollin C Hu
    • , Marissa Powers
    •  & Ziv M Williams
  3. Harvard Medical School, Boston, Massachusetts, USA.

    • Maryam M Shanechi
    • , Rollin C Hu
    • , Emery N Brown
    •  & Ziv M Williams
  4. Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA.

    • Emery N Brown
  5. Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Boston, Massachusetts, USA.

    • Emery N Brown
  6. Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA.

    • Emery N Brown
  7. Harvard-MIT Health Sciences and Technology Program, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA.

    • Emery N Brown

Authors

  1. Search for Maryam M Shanechi in:

  2. Search for Rollin C Hu in:

  3. Search for Marissa Powers in:

  4. Search for Gregory W Wornell in:

  5. Search for Emery N Brown in:

  6. Search for Ziv M Williams in:

Contributions

M.M.S. developed the BMI real-time decoder, conceived and performed the computational analysis, assisted with animal recordings and wrote the manuscript. R.C.H. and M.P. assisted with animal training and recordings. G.W.W. and E.N.B. were involved in the computational methodological development and writing of the manuscript. Z.M.W. conceived and designed the study, developed the BMI system, performed the animal training and recordings, helped implement the computational models and wrote the manuscript.

Competing interests

The authors declare no competing financial interests.

Corresponding author

Correspondence to Ziv M Williams.

Supplementary information

PDF files

  1. 1.

    Supplementary Text and Figures

    Supplementary Figures 1–10 and Supplementary Modeling

About this article

Publication history

Received

Accepted

Published

DOI

https://doi.org/10.1038/nn.3250

Further reading