Signals derived from the rat motor cortex can be used for controlling one-dimensional movements of a robot arm1. It remains unknown, however, whether real-time processing of cortical signals can be employed to reproduce, in a robotic device, the kind of complex arm movements used by primates to reach objects in space. Here we recorded the simultaneous activity of large populations of neurons, distributed in the premotor, primary motor and posterior parietal cortical areas, as non-human primates performed two distinct motor tasks. Accurate real-time predictions of one- and three-dimensional arm movement trajectories were obtained by applying both linear and nonlinear algorithms to cortical neuronal ensemble activity recorded from each animal. In addition, cortically derived signals were successfully used for real-time control of robotic devices, both locally and through the Internet. These results suggest that long-term control of complex prosthetic robot arm movements can be achieved by simple real-time transformations of neuronal population signals derived from multiple cortical areas in primates.
Access optionsAccess options
Subscribe to Journal
Get full journal access for 1 year
only $3.90 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
Chapin, J. K., Moxon, K. A., Markowitz, R. S. & Nicolelis, M. A. L. Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex. Nature Neurosci. 2, 664 –670 (1999).
Evarts, E. V. Pyramidal tract activity associated with a conditioned hand movement in the monkey. J. Neurophysiol. 29, 1011– 1027 (1966).
Mountcastle, V. B., Lynch, J. C., Georgopoulos, A., Sakata, H. & Acuna, C. Posterior parietal association cortex of the monkey: command functions for operation within extrapersonal space. J. Neurophysiol. 38, 871– 908 (1975).
Fetz, E. E. & Cheney, P. D. Muscle fields of primate corticomotoneuronal cells. J. Physiol. (Paris) 74, 239– 245 (1978).
Georgopoulos, A. P., Kalaska, J. F., Caminiti, R. & Massey, J. T. On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex. J. Neurosci. 2, 1527–1537 (1982).
Weinrich, M. & Wise, S. P. The premotor cortex of the monkey. J. Neurosci. 2, 1329–1345 (1982).
Wise, S. P., Boussaoud, D., Johnson, P. B. & Caminiti, R. Premotor and parietal cortex: corticocortical connectivity and combinatorial computations. Annu. Rev. Neurosci. 20, 25 –42 (1997).
Batista, A. P., Buneo, C. A., Snyder, L. H. & Andersen, R. A. Reach plans in eye-centered coordinates. Science 285 , 257–260 (1999).
Mitz, A. R., Godschalk, M. & Wise, S. P. Learning-dependent neuronal activity in the premotor cortex: activity during the acquisition of conditional motor associations. J. Neurosci. 11, 1855– 1872 (1991).
Humphrey, D. R., Schmidt, E. M. & Thompson, W. D. Predicting measures of motor performance from multiple cortical spike trains. Science 170, 758– 762 (1970).
Georgopoulos, A. P., Schwartz, A. B. & Kettner, R. E. Neuronal population coding of movement direction. Science 233, 1416–1419 (1986).
Schwartz, A. Direct cortical representation of drawing. Science 265, 540–542 (1994).
Fetz, E. E. & Cheney, P. D. Postspike facilitation of forelimb muscle activity by primate corticomotoneuronal cells. J. Neurophysiol. 44, 751–772 ( 1980).
Nicolelis, A. L. et al. Simultaneous representation of tactile information by distinct primate cortical areas rely on different encoding strategies. Nature Neurosci. 1, 621–630 (1998).
Laubach, M., Wessberg, J. & Nicolelis, M. A. L. Cortical ensemble activity increasingly predicts behaviour outcomes during learning of a motor task. Nature 405, 567–571 (2000).
Nicolelis, M. A. L., Ghazanfar, A. A., Faggin, B. M., Votaw, S. & Oliveira, L. M. Reconstructing the engram: simultaneous, multisite, many single neuron recordings. Neuron 18 , 529–537 (1997).
Stepniewska, I., Preuss, T. M. & Kaas, J. H. Architectonics, somatotopic organization, and ipsilateral cortical connections of the primary motor area (M1) of owl monkeys. J. Comp. Neurol. 330, 238–271 (1993).
Preuss, T. M., Stepniewska, I. & Kaas, J. H. Movement representation in the dorsal and ventral premotor areas of owl monkeys: a microstimulation study. J. Comp. Neurol. 371, 649–676 ( 1996).
Brillinger, D. R. Time Series. Data Analysis and Theory (Holden-Day, San Fransisco, 1981).
Bendat, J. S. & Piersol, A. G. Random Data. Analysis and Measurement Procedures (Wiley, New York, 1986).
Halliday, D. M. et al. The Fourier approach to the analysis of mixed time series/point process data. Theory and application to the study of physiological tremor, single motor unit discharges and electromyograms. Prog. Biophys. Mol. Biol. 64, 237–278 ( 1995).
Powell, M. J. D. Restart procedures for the conjugate gradient method. Math. Program. 12, 241–254 ( 1977).
Ghazanfar, A. A., Stambaugh, C. R. & Nicolelis, M. A. L. Encoding of tactile stimulus location by somatosensory thalamocortical ensembles. J. Neurosci. 20, 3761–3775 (2000).
Salisbury, J. K. & Srinivasan, M. A. Phantom-based haptic interaction with virtual objects. IEEE Comput. Graph. Appl. 17, 6–10 (1997 ).
Ferraina, S. et al. Combination of hand and gaze signals during reaching: Activity in parietal area 7m of the monkey. J. Neurophysiol. 77, 1034–1038 (1997).
Mussa-Ivaldi, F. A. Do neurons in the motor cortex encode movement direction? An alternative hypothesis. Neurosci. Lett. 91, 106– 111 (1988).
Scott, S. H., Sergio, L. E. & Kalaska, J. F. Reaching movements with similar hand paths but different arm orientations. II. Activity of individual cells in dorsal premotor cortex and parietal area 5. J. Neurophysiol. 78, 2413–2426 (1997).
Schmidt, E. M. Single neuron recording from motor cortex as a possible source of signals for control of external devices. Ann. Biomed. Eng. 8, 339–349 (1980).
Kennedy P. R. & Bakay, R. A. Restoration of neural output from a paralyzed patient by a direct brain connection. Neuroreport 9, 1707–1711 (1998).
Nicolelis, M. A. L. Hybrid brain-machine interfaces for translating thoughts into action. Nature (in the press).
This work was supported by grants from the National Institutes of Health and DARPA-ONR to M.A.L.N., J.K.C. and M.A.S., and the National Science Foundation to M.A.L.N. J.W. was supported by The Swedish Foundation for International Cooperation in Research and Higher Education, and the Swedish Medical Research Council. J.K., P.B. and M.L. were supported by NIH postdoctoral fellowships.
About this article
Journal of Neural Engineering (2019)
A High Definition Noninvasive Neuromuscular Electrical Stimulation System for Cortical Control of Combinatorial Rotary Hand Movements in a Human With Tetraplegia
IEEE Transactions on Biomedical Engineering (2019)
Frontiers in Neural Circuits (2019)
Robotics and Autonomous Systems (2019)
Transactions of Japan Society of Kansei Engineering (2019)