Mood variations decoded from multi-site intracranial human brain activity

Abstract

The ability to decode mood state over time from neural activity could enable closed-loop systems to treat neuropsychiatric disorders. However, this decoding has not been demonstrated, partly owing to the difficulty of modeling distributed mood-relevant neural dynamics while dealing with the sparsity of mood state measurements. Here we develop a modeling framework to decode mood state variations from multi-site intracranial recordings in seven human subjects with epilepsy who self-reported their mood state intermittently over multiple days. We built dynamic neural encoding models of mood state and corresponding decoders for each individual and demonstrated that mood state variations over time can be decoded from neural activity. Across subjects, the decoders largely recruited neural signals from limbic regions, whose spectro-spatial features were tuned to mood variations. The dynamic models also provided an analytical tool to compute the timescales of the decoded mood state. These results provide an initial line of evidence indicating the feasibility of mood state decoding.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Figure 1: Illustration of decoder training and evaluation using cross-validation.
Figure 2: Mood state could be decoded from neural activity in each subject.
Figure 3: Mood-predictive networks selected for decoding were largely within the limbic regions in all subjects.
Figure 4: Spectro-spatial features within the mood-predictive networks were tuned to mood state variations over time.

References

  1. 1

    Calhoon, G.G. & Tye, K.M. Resolving the neural circuits of anxiety. Nat. Neurosci. 18, 1394–1404 (2015).

    CAS  PubMed  Article  Google Scholar 

  2. 2

    Dan, R. et al. Separate neural representations of depression, anxiety and apathy in Parkinson's disease. Sci. Rep. 7, 12164 (2017).

    PubMed  PubMed Central  Article  CAS  Google Scholar 

  3. 3

    Dmochowski, J.P., Sajda, P., Dias, J. & Parra, L.C. Correlated components of ongoing EEG point to emotionally laden attention - a possible marker of engagement? Front. Hum. Neurosci. 6, 112 (2012).

    PubMed  PubMed Central  Article  Google Scholar 

  4. 4

    Drevets, W.C. Neuroimaging and neuropathological studies of depression: implications for the cognitive-emotional features of mood disorders. Curr. Opin. Neurobiol. 11, 240–249 (2001).

    CAS  Article  Google Scholar 

  5. 5

    Drysdale, A.T. et al. Resting-state connectivity biomarkers define neurophysiological subtypes of depression. Nat. Med. 23, 28–38 (2017).

    CAS  PubMed  Article  Google Scholar 

  6. 6

    Haroush, K. & Williams, Z.M. Neuronal prediction of opponent's behavior during cooperative social interchange in primates. Cell 160, 1233–1245 (2015).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  7. 7

    Kragel, P.A., Knodt, A.R., Hariri, A.R. & LaBar, K.S. Decoding spontaneous emotional states in the human brain. PLoS Biol. 14, e2000106 (2016).

    PubMed  PubMed Central  Article  CAS  Google Scholar 

  8. 8

    Kupfer, D.J., Frank, E. & Phillips, M.L. Major depressive disorder: new clinical, neurobiological, and treatment perspectives. Lancet 379, 1045–1055 (2012).

    PubMed  Article  Google Scholar 

  9. 9

    Mayberg, H.S. Modulating dysfunctional limbic-cortical circuits in depression: towards development of brain-based algorithms for diagnosis and optimised treatment. Br. Med. Bull. 65, 193–207 (2003).

    Article  Google Scholar 

  10. 10

    Mayberg, H.S. et al. Reciprocal limbic-cortical function and negative mood: converging PET findings in depression and normal sadness. Am. J. Psychiatry 156, 675–682 (1999).

    CAS  PubMed  PubMed Central  Google Scholar 

  11. 11

    Zeng, L.-L. et al. Identifying major depression using whole-brain functional connectivity: a multivariate pattern analysis. Brain 135, 1498–1507 (2012).

    PubMed  PubMed Central  Article  Google Scholar 

  12. 12

    Clark, L.A. & Watson, D. Tripartite model of anxiety and depression: psychometric evidence and taxonomic implications. J. Abnorm. Psychol. 100, 316–336 (1991).

    CAS  PubMed  Article  Google Scholar 

  13. 13

    Posner, J., Russell, J.A. & Peterson, B.S. The circumplex model of affect: an integrative approach to affective neuroscience, cognitive development, and psychopathology. Dev. Psychopathol. 17, 715–734 (2005).

    PubMed  PubMed Central  Article  Google Scholar 

  14. 14

    Clark, D.L., Brown, E.C., Ramasubbu, R. & Kiss, Z.H.T. Intrinsic local beta oscillations in the subgenual cingulate relate to depressive symptoms in treatment-resistant depression. Biol. Psychiatry 80, e93–e94 (2016).

    PubMed  Article  Google Scholar 

  15. 15

    Leuchter, A.F., Cook, I.A., Hunter, A.M., Cai, C. & Horvath, S. Resting-state quantitative electroencephalography reveals increased neurophysiologic connectivity in depression. PLoS One 7, e32508 (2012).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  16. 16

    Neumann, W.J. et al. Different patterns of local field potentials from limbic DBS targets in patients with major depressive and obsessive compulsive disorder. Mol. Psychiatry 19, 1186–1192 (2014).

    PubMed  PubMed Central  Article  Google Scholar 

  17. 17

    Malone, D.A. Jr. et al. Deep brain stimulation of the ventral capsule/ventral striatum for treatment-resistant depression. Biol. Psychiatry 65, 267–275 (2009).

    PubMed  Article  Google Scholar 

  18. 18

    Mayberg, H.S. et al. Deep brain stimulation for treatment-resistant depression. Neuron 45, 651–660 (2005).

    CAS  PubMed  Article  Google Scholar 

  19. 19

    Schlaepfer, T.E. et al. Deep brain stimulation to reward circuitry alleviates anhedonia in refractory major depression. Neuropsychopharmacology 33, 368–377 (2008).

    PubMed  Article  Google Scholar 

  20. 20

    Clark, D.A. & Beck, A.T. Cognitive theory and therapy of anxiety and depression: convergence with neurobiological findings. Trends Cogn. Sci. 14, 418–424 (2010).

    PubMed  Article  Google Scholar 

  21. 21

    Grimm, S. et al. Imbalance between left and right dorsolateral prefrontal cortex in major depression is linked to negative emotional judgment: an fMRI study in severe major depressive disorder. Biol. Psychiatry 63, 369–376 (2008).

    PubMed  Article  Google Scholar 

  22. 22

    Lee, B.-T., Lee, B.C., Choi, I.G., Lyoo, I.K. & Ham, B.J. The neural substrates of affective processing toward positive and negative affective pictures in patients with major depressive disorder. Prog. Neuropsychopharmacol. Biol. Psychiatry 31, 1487–1492 (2007).

    PubMed  Article  Google Scholar 

  23. 23

    Bouchard, K.E., Mesgarani, N., Johnson, K. & Chang, E.F. Functional organization of human sensorimotor cortex for speech articulation. Nature 495, 327–332 (2013).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  24. 24

    Korzeniewska, A. et al. Ictal propagation of high frequency activity is recapitulated in interictal recordings: effective connectivity of epileptogenic networks recorded with intracranial EEG. Neuroimage 101, 96–113 (2014).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  25. 25

    Yaffe, R.B. et al. Reinstatement of distributed cortical oscillations occurs with precise spatiotemporal dynamics during successful memory retrieval. Proc. Natl. Acad. Sci. USA 111, 18727–18732 (2014).

    CAS  PubMed  Article  Google Scholar 

  26. 26

    Nahum, M. et al. Immediate Mood Scaler: tracking symptoms of depression and anxiety using a novel mobile mood scale. JMIR Mhealth Uhealth 5, e44 (2017).

    PubMed  PubMed Central  Article  Google Scholar 

  27. 27

    Friedman, J., Hastie, T. & Tibshirani, R. The Elements of Statistical Learning (Springer, New York, 2001).

  28. 28

    Van Overschee, P. & De Moor, B. Subspace Identification for Linear Systems: Theory, Implementation and Applications (Kluwer Academic, Norwell, Massachusetts, USA, 1996).

  29. 29

    Fitzgerald, P.B., Laird, A.R., Maller, J. & Daskalakis, Z.J. A meta-analytic study of changes in brain activation in depression. Hum. Brain Mapp. 29, 683–695 (2008).

    PubMed  PubMed Central  Article  Google Scholar 

  30. 30

    Schieber, M.H. Constraints on somatotopic organization in the primary motor cortex. J. Neurophysiol. 86, 2125–2143 (2001).

    CAS  PubMed  Article  Google Scholar 

  31. 31

    Andersen, R.A., Kellis, S., Klaes, C. & Aflalo, T. Toward more versatile and intuitive cortical brain-machine interfaces. Curr. Biol. 24, R885–R897 (2014).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  32. 32

    Brandman, D.M., Cash, S.S. & Hochberg, L.R. Review: human intracortical recording and neural decoding for brain-computer interfaces. IEEE Trans. Neural Syst. Rehabil. Eng. 25, 1687–1696 (2017).

    PubMed  PubMed Central  Article  Google Scholar 

  33. 33

    Hatsopoulos, N.G. & Suminski, A.J. Sensing with the motor cortex. Neuron 72, 477–487 (2011).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  34. 34

    Hotson, G. et al. Individual finger control of a modular prosthetic limb using high-density electrocorticography in a human subject. J. Neural Eng. 13, 026017–26017 (2016).

    PubMed  PubMed Central  Article  Google Scholar 

  35. 35

    Leuthardt, E.C., Schalk, G., Wolpaw, J.R., Ojemann, J.G. & Moran, D.W. A brain-computer interface using electrocorticographic signals in humans. J. Neural Eng. 1, 63–71 (2004).

    PubMed  Article  Google Scholar 

  36. 36

    Nicolelis, M.A. & Lebedev, M.A. Principles of neural ensemble physiology underlying the operation of brain-machine interfaces. Nat. Rev. Neurosci. 10, 530–540 (2009).

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  37. 37

    Sadtler, P.T. et al. Neural constraints on learning. Nature 512, 423–426 (2014).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  38. 38

    Sajda, P., Muller, K.-R. & Shenoy, K. Brain-computer interfaces. IEEE Signal Process. Mag. 25, 16–17 (2008).

    Article  Google Scholar 

  39. 39

    Schwartz, A.B., Cui, X.T., Weber, D.J. & Moran, D.W. Brain-controlled interfaces: movement restoration with neural prosthetics. Neuron 52, 205–220 (2006).

    CAS  PubMed  Article  Google Scholar 

  40. 40

    Shanechi, M.M. Brain-machine interface control algorithms. IEEE Trans. Neural Syst. Rehabil. Eng. 25, 1725–1734 (2017).

    PubMed  Article  Google Scholar 

  41. 41

    Shanechi, M.M. et al. Rapid control and feedback rates enhance neuroprosthetic control. Nat. Commun. 8, 13825 (2017).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  42. 42

    Shenoy, K.V. & Carmena, J.M. Combining decoder design and neural adaptation in brain-machine interfaces. Neuron 84, 665–680 (2014).

    CAS  PubMed  Article  Google Scholar 

  43. 43

    Thakor, N.V. Translating the brain-machine interface. Sci. Transl. Med. 5, 210ps17 (2013).

    PubMed  Article  Google Scholar 

  44. 44

    Wang, W. et al. An electrocorticographic brain interface in an individual with tetraplegia. PLoS One 8, e55344 (2013).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  45. 45

    Yanagisawa, T. et al. Electrocorticographic control of a prosthetic arm in paralyzed patients. Ann. Neurol. 71, 353–361 (2012).

    PubMed  Article  Google Scholar 

  46. 46

    Habel, U., Klein, M., Kellermann, T., Shah, N.J. & Schneider, F. Same or different? Neural correlates of happy and sad mood in healthy males. Neuroimage 26, 206–214 (2005).

    PubMed  Article  Google Scholar 

  47. 47

    Mak, A.K., Hu, Z.G., Zhang, J.X., Xiao, Z.W. & Lee, T.M. Neural correlates of regulation of positive and negative emotions: an fMRI study. Neurosci. Lett. 457, 101–106 (2009).

    CAS  PubMed  Article  Google Scholar 

  48. 48

    Sartorius, N., Ustün, T.B., Lecrubier, Y. & Wittchen, H.U. Depression comorbid with anxiety: results from the WHO study on psychological disorders in primary health care. Br. J. Psychiatry Suppl. 1996, 38–43 (1996).

    Article  Google Scholar 

  49. 49

    American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (DSM-5) (American Psychiatric Publishing, Arlington, Virginia, 2013).

  50. 50

    Russell, J.A. Core affect and the psychological construction of emotion. Psychol. Rev. 110, 145–172 (2003).

    PubMed  Article  Google Scholar 

  51. 51

    Kroenke, K., Spitzer, R.L. & Williams, J.B. The PHQ-9: validity of a brief depression severity measure. J. Gen. Intern. Med. 16, 606–613 (2001).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  52. 52

    Spitzer, R.L., Kroenke, K., Williams, J.B. & Löwe, B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch. Intern. Med. 166, 1092–1097 (2006).

    PubMed  Article  Google Scholar 

  53. 53

    Hermann, B.P., Seidenberg, M. & Bell, B. Psychiatric comorbidity in chronic epilepsy: identification, consequences, and treatment of major depression. Epilepsia 41 (Suppl. 2), S31–S41 (2000).

    PubMed  Article  Google Scholar 

  54. 54

    Harden, C.L. et al. A pilot study of mood in epilepsy patients treated with vagus nerve stimulation. Epilepsy Behav. 1, 93–99 (2000).

    PubMed  Article  Google Scholar 

  55. 55

    George, M.S. et al. Vagus nerve stimulation for the treatment of depression and other neuropsychiatric disorders. Expert Rev. Neurother. 7, 63–74 (2007).

    PubMed  Article  Google Scholar 

  56. 56

    Etkin, A. & Wager, T.D. Functional neuroimaging of anxiety: a meta-analysis of emotional processing in PTSD, social anxiety disorder, and specific phobia. Am. J. Psychiatry 164, 1476–1488 (2007).

    PubMed  PubMed Central  Article  Google Scholar 

  57. 57

    Goldstein, R.Z. & Volkow, N.D. Dysfunction of the prefrontal cortex in addiction: neuroimaging findings and clinical implications. Nat. Rev. Neurosci. 12, 652–669 (2011).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  58. 58

    Tracey, I. & Bushnell, M.C. How neuroimaging studies have challenged us to rethink: is chronic pain a disease? J. Pain 10, 1113–1120 (2009).

    PubMed  Article  Google Scholar 

  59. 59

    Yang, Y . & Shanechi, M.M. Generalized binary noise stimulation enables time-efficient identification of input-output brain network dynamics. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2016, 1766–1769 (2016).

    Google Scholar 

  60. 60

    Yang, Y ., Chang, E.F. & Shanechi, M.M. Dynamic tracking of non-stationarity in human ECoG activity. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2017, 1660–1663 (2017).

    Google Scholar 

  61. 61

    Fischl, B. FreeSurfer. Neuroimage 62, 774–781 (2012).

    PubMed  PubMed Central  Article  Google Scholar 

  62. 62

    Mazziotta, J. et al. A probabilistic atlas and reference system for the human brain: International Consortium for Brain Mapping (ICBM). Phil. Trans. R. Soc. Lond. B 356, 1293–1322 (2001).

    CAS  Article  Google Scholar 

  63. 63

    Yekutieli, D. & Benjamini, Y. Resampling-based false discovery rate controlling multiple test procedures for correlated test statistics. J. Stat. Plan. Inference 82, 171–196 (1999).

    Article  Google Scholar 

  64. 64

    Donoghue, J.P. Bridging the brain to the world: a perspective on neural interface systems. Neuron 60, 511–521 (2008).

    CAS  PubMed  Article  Google Scholar 

  65. 65

    Hsieh, H.-L. & Shanechi, M.M. Optimizing the learning rate for adaptive estimation of neural encoding models. PLoS Comput. Biol. 14, e1006168 (2018).

    PubMed  PubMed Central  Article  CAS  Google Scholar 

  66. 66

    Kao, J.C. et al. Single-trial dynamics of motor cortex and their applications to brain-machine interfaces. Nat. Commun. 6, 7759 (2015).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  67. 67

    Pistohl, T., Ball, T., Schulze-Bonhage, A., Aertsen, A. & Mehring, C. Prediction of arm movement trajectories from ECoG-recordings in humans. J. Neurosci. Methods 167, 105–114 (2008).

    PubMed  Article  Google Scholar 

  68. 68

    Shanechi, M.M., Orsborn, A.L. & Carmena, J.M. Robust brain-machine interface design using optimal feedback control modeling and adaptive point process filtering. PLOS Comput. Biol. 12, e1004730 (2016).

    PubMed  PubMed Central  Article  CAS  Google Scholar 

  69. 69

    So, K., Dangi, S., Orsborn, A.L., Gastpar, M.C. & Carmena, J.M. Subject-specific modulation of local field potential spectral power during brain-machine interface control in primates. J. Neural Eng. 11, 026002 (2014).

    PubMed  Article  Google Scholar 

  70. 70

    Stavisky, S.D., Kao, J.C., Nuyujukian, P., Ryu, S.I. & Shenoy, K.V. A high performing brain-machine interface driven by low-frequency local field potentials alone and together with spikes. J. Neural Eng. 12, 036009 (2015).

    PubMed  PubMed Central  Article  Google Scholar 

  71. 71

    Yang, Y. & Shanechi, M.M. An adaptive and generalizable closed-loop system for control of medically induced coma and other states of anesthesia. J. Neural Eng. 13, 066019 (2016).

    PubMed  Article  Google Scholar 

  72. 72

    Anderson, B.D. & Moore, J.B. Optimal Filtering (Dover, New York, 1979).

  73. 73

    Cook, R.D. & Weisberg, S. Residuals and Influence in Regression (Chapman and Hall, New York, 1982).

  74. 74

    Burgos-Robles, A. et al. Amygdala inputs to prefrontal cortex guide behavior amid conflicting cues of reward and punishment. Nat. Neurosci. 20, 824–835 (2017).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  75. 75

    Ebmeier, K.P., Donaghey, C. & Steele, J.D. Recent developments and current controversies in depression. Lancet 367, 153–167 (2006).

    PubMed  PubMed Central  Article  Google Scholar 

  76. 76

    Kim, S.-Y. et al. Diverging neural pathways assemble a behavioural state from separable features in anxiety. Nature 496, 219–223 (2013).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  77. 77

    Michopoulos, V. et al. Association of CRP genetic variation and CRP level with elevated PTSD symptoms and physiological responses in a civilian population with high levels of trauma. Am. J. Psychiatry 172, 353–362 (2015).

    PubMed  Article  Google Scholar 

  78. 78

    Mulders, P.C., van Eijndhoven, P.F., Schene, A.H., Beckmann, C.F. & Tendolkar, I. Resting-state functional connectivity in major depressive disorder: A review. Neurosci. Biobehav. Rev. 56, 330–344 (2015).

    PubMed  Article  Google Scholar 

  79. 79

    Namburi, P. et al. A circuit mechanism for differentiating positive and negative associations. Nature 520, 675–678 (2015).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  80. 80

    Tye, K.M. et al. Dopamine neurons modulate neural encoding and expression of depression-related behaviour. Nature 493, 537–541 (2013).

    CAS  PubMed  Article  Google Scholar 

Download references

Acknowledgements

This research was partially funded by the Defense Advanced Research Projects Agency (DARPA) under Cooperative Agreement Number W911NF-14-2-0043 (to M.M.S. and E.F.C.), issued by the Army Research Office contracting office in support of DARPA's SUBNETS program. The views, opinions and/or findings expressed are those of the author(s) and should not be interpreted as representing the official views or policies of the Department of Defense or the US Government.

Author information

Affiliations

Authors

Contributions

M.M.S. and E.F.C. supervised the project. M.M.S. conceived the dynamic modeling and decoding framework, and M.M.S., O.G.S. and Y.Y. developed it. E.F.C. and M.B.L. developed and implemented the continuous mood testing protocol and coordinated data collection for all the intracranial recording and imaging data. O.G.S. and Y.Y. implemented and performed the modeling and analyses. M.M.S. supervised all the modeling and analyses work. O.G.S., Y.Y. and M.M.S. wrote the manuscript with input from E.F.C. and H.E.D.

Corresponding authors

Correspondence to Edward F Chang or Maryam M Shanechi.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Integrated supplementary information

Supplementary Figure 1 Mood state ranges in each subject and validation of IMS against PHQ-9 and GAD-7.

(a) Range of mood state variations observed in each subject. The lower and upper bounds of the boxes represent the 25th and 75th percentiles of the IMS scores, the middle line represents the median and the whiskers represent the minimum and maximum scores for the subject. Number of samples in each subject is equal to the number of IMS points in that subject (Supplementary Table 3). (b) Correlation of PHQ-9 (red, number of samples equal to 110) and GAD-7 (blue, number of samples equal to 93) scores with the IMS score (adapted from Figure 2.D of Nahum et al.26*; the vertical axis is transferred to the equivalent IMS scoring that is used in this study, i.e., scoring each question from -3 to +3). This data shows the range of IMS values reported for various values of PHQ-9 and GAD-7 across this large population. The range of IMS observed in this study across our subjects is marked on the plots with the green shaded area. The vertical axes in a and b cover the total possible range of IMS (-72 to +72). The horizontal axis in b covers the total range of PHQ-951 (0 to 27) and GAD-752 (0 to 21).26. Nahum, M. et al. Immediate mood scaler: tracking symptoms of depression and anxiety using a novel mobile mood scale. JMIR mHealth and uHealth 5, e44 (2017).51. Kroenke, K., Spitzer, R. L. & Williams, J. B. The PHQ-9: validity of a brief depression severity measure. J. Gen. Intern. Med. 16, 606–613 (2001).52. Spitzer, R. L., Kroenke, K., Williams, J. B. & Löwe, B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch. Intern. Med. 166, 1092–1097 (2006).* Copyright ©Mor Nahum, Thomas M Van Vleet, Vikaas S Sohal, Julie J Mirzabekov, Vikram R Rao, Deanna L Wallace, Morgan B Lee, Heather Dawes, Alit Stark-Inbar, Joshua Thomas Jordan, Bruno Biagianti, Michael Merzenich, Edward F Chang. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org/2017/4/e44/), 12.04.2017. This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/).

Supplementary Figure 2 Details of model fitting, selection and evaluation.

(a) Progressive region selection. Progressive region selection starts with a network size of one (Net. Size = 1, i.e., one region), and progressively increases the network size until decoding becomes possible (with FDR control to correct for multiple comparisons). At each stage, model selection and model fitting are performed with leave-one-out cross-validation using the training IMS; then the corresponding decoder is tested on the test IMS. To assess significance, the same procedure is performed on sets of random IMS points to compute the random-test P value. (b) Model fitting for a candidate neural encoding model specified by a given network, a given number of PCs, and a given neural state dimension. PCA and LSSM are fitted purely based on neural features, unsupervised with respect to IMS points. Regression model is fitted using the training IMS with regularization. (c) Model selection. Model selection involves fitting several candidate neural encoding models by considering different networks, number of PCs and neural state dimensions. For each candidate neural encoding model, prediction error for training IMS and sensitivity to training IMS are computed and a neural encoding model with small values for both measures is selected. (d) Decoding. The selected neural encoding model is then used to build a decoder to predict the test IMS. Note that the test IMS is never seen in model fitting in b or in model selection in c, which together specify the decoder. The decoder extracts neural features from the selected network, linearly transforms them to PCs, extracts the neural state via Kalman filtering and applies a linear regression to them to predict mood state at the time of the test IMS. We evaluate the decoder only at the discrete time-points that the questionnaire was filled, i.e., that a true measured IMS point was observed.

Supplementary Figure 3 Cross-validated prediction of random IMS points.

(ah) Cross-validated predictions for an example (randomly selected) set of random IMS points is shown pooled across subjects in a and within individuals in bh. Figure convention is the same as in Fig. 2. Random-test P value is noted on each plot (Online Methods). Number of IMS samples in a is 87 and in bh is equal to the number of IMS points for each subject (Supplementary Table 3).

Supplementary Figure 4 Cross-validated prediction error for random and permuted sets of IMS points, compared with the true set of IMS points.

Cross-validated prediction error is quantified by the normalized root mean square error (NRMSE). (ah) Distribution of the prediction error for sets of random (dark blue) and permuted (light blue) IMS points are shown pooled across subjects in a and within individuals in bh. Cross-validated prediction error for the true IMS points (corresponding to the results shown in Fig. 2 and Table 1) is shown as a red vertical line in each plot. The random and permuted test P values are computed as the probability that NRMSE for random and permuted sets of IMS will be equal to or smaller than that of the true IMS points (area under the distribution tail to the left of the vertical line), respectively. (i, j) Bar plots for average cross-validated prediction errors are shown pooled across subjects in i and within individual subjects in j. In all cases, a two-sided t-test comparing true IMS NRMSE with random and permuted IMS average NRMSEs resulted in P < 10−207 (t > 39 and degrees of freedom > 998 for all t-tests), which is indicated by ***. Error bars represent s.e.m. Number of samples is equal to the number of random and permuted sets, i.e. 109QUOTE 109 in a and i and 1000 in bh and j.

Supplementary Figure 5 Decoders generalized from day to day.

Prediction of IMS points that were the only IMS point (i.e., report) obtained in a day was significant across subjects. The cross-validated IMS predictions in each subject are z-scored based on the mean and s.d. of all true IMS points in that subject. Then z-scored predictions of the IMS points that were the only IMS point in a day were pooled together across all four subjects that had such IMS points. The same procedure was applied to sets of random IMS points to find the random-test P value (Online Methods). Number of IMS samples is 14.

Supplementary Figure 6 Decoders could significantly predict the maximum and minimum IMS values in each subject, which were outside the range of IMS values used for decoder training.

The cross-validated IMS predictions from each subject are z-scored based on mean and s.d. of all true IMS points in that subject. Then the z-scores of the maximum and minimum IMS predictions in each subject were pooled across the population. The same procedure was applied to sets of random IMS points to find the random-test P value (Online Methods). Number of IMS samples is 14.

Supplementary Figure 7 IMS depression and anxiety subscales could also be separately decoded using the same networks selected for decoding the full IMS.

(ac) IMS depression and anxiety subscales consist of 7 and 5 of the total 24 questions, respectively26. Within cross-validation, the regression model was retrained to predict the IMS depression and anxiety subscales separately in each subject. The network, number of PCs and neural state dimension were fixed to be the same as those selected for decoding the full IMS. Despite the smaller number of questions and thus potentially a lower subscale measurement signal-to-noise ratio, the IMS depression and anxiety subscales could be significantly decoded across subjects using the same networks that were predictive of the full IMS. Cross-validated predictions were pooled across subjects by z-scoring using the true value of the scale in each subject. Cross-validated predictions versus the true value of the scales are shown for full IMS in a, for IMS depression subscale in b, and for IMS anxiety subscale in c. The random-test P value in each case is noted on the figure and is significant in all cases (Online Methods). Number of IMS and subscale samples in each case is 87. Performing a search of network, number of PCs and neural state dimensions for each IMS subscale separately led to similar results. Moreover, an additional correlation analysis showed that in subjects whose two subscales had different variations over time (all but EC108 whose two subscales had almost equal variance, i.e., 4.2% difference vs. more than 30% difference in the other six subjects), the decoded values for the two subscales also showed a similar difference in their variations. In these subjects, the ratio of the variances of the two subscales, and the ratio of the variances of the decoded values for the two subscales were significantly correlated (Pearson’s P = 0.023, number of samples is 6).26. Nahum, M. et al. Immediate mood scaler: tracking symptoms of depression and anxiety using a novel mobile mood scale. JMIR mHealth and uHealth 5, e44 (2017).

Supplementary Figure 8 Limbic regions alone were sufficient for decoding mood state in all subjects except EC137, whose decoding still required the limbic regions.

(ag) The best small mood-predictive network sufficient for decoding in each subject. In each panel, lateral (L), medial (M) and orbital (O) views of the brain are shown. The areas corresponding to key limbic regions based on labeling from the FreeSurfer software61 are shaded with color. Each recording channel is also marked with the color of the anatomical region to which it is assigned based on FreeSurfer labels and expert examination (with the latter having priority in case of any difference). All displayed cingulate coverage was in dorsal ACC, except for one intracranial electrode in EC87, which is marked as ventral ACC on the figure (Supplementary Table 2).61. Fischl, B. FreeSurfer. Neuroimage 62, 774–781 (2012).

Supplementary Figure 9 Extending the modeling and search to all regions robustly decoded mood state variations in six subjects.

(ae) Figure convention is the same as in Fig. 2. Random-test P < 0.05 for all after FDR correction (Online Methods). Number of samples in each subject is equal to the number of IMS points in that subject (Supplementary Table 3). Results for EC137 are shown in Fig. 2 as part of the main results.

Supplementary Figure 10 Extending the search to all regions consistently and robustly selected the same regions as those selected in a search within the limbic regions alone.

(ae) Figure convention is the same as in Supplementary Fig. 8. Note that in EC108, the same AMYG electrode as the limbic search was selected (Supplementary Fig. 8); this electrode had some channels (indicated in black) that were not verified as being exactly in AMYG and thus were not included in the limbic search but were included in the all-region search.

Supplementary Figure 11 Mood state decoding without limbic regions failed in most subjects.

(ab) The cross-validated normalized mean-absolute error is shown when using only limbic regions or using only other regions for decoding across the population in a and in each individual in b. This analysis was performed for the six subjects that had similar number of neural features inside (97 ± 54) and outside (96 ± 48) limbic regions (two-sided t-test P = 0.98, t = 0.032 and degrees of freedom = 5). Number of samples in a is 75 (total number of IMS points across the six subjects) and in b is equal to the number of IMS points for each subject (Supplementary Table 3). Bars represent mean and error bars represent s.e.m. In b, individual data points are overlaid on each bar plot as black dots (the vertical axis is broken to accommodate the visualization of outliers). Decoding with features outside limbic regions failed at the population level (a). In four of the six subjects, decoding was not significant without the limbic regions (b); only in EC108 and EC166, a mood-predictive network was identified outside the limbic regions in both cases consisting of inferior temporal cortex, which has been implicated in prior studies as mood-relevant81–83 (Supplementary Table 7). Significance caption: n.s. (not significant), random-test P > 0.05; *, random-test P < 0.05; **, random-test P < 0.005; ***, random-test P < 0.005.81. Morris, J. S. et al. A neuromodulatory role for the human amygdala in processing emotional facial expressions. Brain 121, 47–57 (1998).82. Sabatinelli, D., Bradley, M. M., Fitzsimmons, J. R. & Lang, P. J. Parallel amygdala and inferotemporal activation reflect emotional intensity and fear relevance. Neuroimage 24, 1265–1270 (2005).83. Sato, W., Kochiyama, T., Yoshikawa, S., Naito, E. & Matsumura, M. Enhanced neural activity in response to dynamic facial expressions of emotion: an fMRI study. Brain Res. Cogn. Brain Res. 20, 81–91 (2004).

Supplementary Figure 12 Decoding performance using only features from one frequency band compared to using features from all five frequency bands.

(a–b) The cross-validated normalized mean-absolute error is shown for predictions pooled across the population in a and for each individual in b. Within cross-validation, the selected network, the number of PCs, and the neural state dimension for each band were kept the same as those for all bands combined. Number of samples in a is 87 and in b is equal to the number of IMS points for each subject (Supplementary Table 3). Bars represent mean and error bars represent s.e.m. In b, individual data points are overlaid on each bar plot as black dots (the vertical axis is broken to accommodate the visualization of outliers). Asterisks indicate significantly predictive decoding using the corresponding band(s) (*, random-test P < 0.05; **, random-test P < 0.005; ***, random-test P < 0.0005). All frequency bands were predictive in one or more subjects even when used exclusively. Bands with significantly worse decoding error compared with all bands combined (one-sided Wilcoxon signed-rank P < 0.05) are marked with an upward red triangle. No band had significantly lower error than all bands combined. Also, on average, decoding error achieved when the decoder used all bands combined was significantly lower than when it used any single band exclusively as shown in a. Performing a search of the network, the number of PCs and the neural state dimension for each individual band separately led to similar results.

Supplementary Figure 13 Analytically estimated power spectral density (PSD) of the decoded mood state in each subject.

(af) The horizontal axis is the timescale (inverse of frequency) and the vertical axis is proportional to the log of PSD at the timescale (dB units). Area under the curve (in original power units) over a certain set of timescales is proportional to how prominent the decoded mood state variations are at those timescales. For example, 70% of the variations in the decoded mood state occurred at timescales of hours or slower (shown as the shaded area in each plot). This analysis was not applicable to EC150 as its neural encoding model was the special case of direct regression (i.e., an LSSM with all eigenvalues of the A matrix equal to 0). Timescale unit abbreviations: h: hours; m: minutes; s: seconds.

Supplementary Figure 14 Mood state decoders provided a slowly changing prediction of mood state around the measured IMS points.

Cross-validated prediction of mood state over time in a 20-minute window centered around each measured IMS point is shown in grey. The true value of the measured IMS point is marked with a black cross. The decoder used to obtain the grey trace in each subplot for prediction around each IMS point is trained only using the other IMS points in the subject (i.e., cross-validation). For 2 IMS points, decoding is shown for less than 10 minutes after the IMS point since the subject was going to sleep immediately after the completion of the IMS report rendering the neural features unreliable (shaded as blue). For comparison, traces of an overfit decoder trained with a degenerate version of our method and without cross-validation is shown in green. This overfit decoder was constructed by deliberately removing several components in our modeling framework, which are designed to prevent overfitting (i.e., progressive region selection, regularization in regression, and the sensitivity measure and inner-level cross-validation for model selection; Online Methods). Moreover, we fully removed the leave-one-out cross-validation and trained the overfit decoder on all IMS points. So to obtain the green trace, the IMS point in each subfigure is used in the training of the overfit decoder. In contrast to our decoder (grey trace), the overfit decoder results in mood traces (green) that often have large variations even within minutes. The s.d. of the overfit green trace during a 20-minute window around each IMS point was significantly larger than the s.d. of our cross-validated decoded trace across all subjects (one-sided Wilcoxon signed-rank P = 8.5×10−15, number of samples is 87).

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–14 (PDF 2270 kb)

Life Sciences Reporting Summary (PDF 131 kb)

Supplementary Tables and Notes

Supplementary Tables 1–8 and Supplementary Notes 1–10 (PDF 1370 kb)

Supplementary Code

Custom MATLAB code. (ZIP 2 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Sani, O., Yang, Y., Lee, M. et al. Mood variations decoded from multi-site intracranial human brain activity. Nat Biotechnol 36, 954–961 (2018). https://doi.org/10.1038/nbt.4200

Download citation

Further reading