Historical analysis of national subjective wellbeing using millions of digitized books

Article metrics

Abstract

In addition to improving quality of life, higher subjective wellbeing leads to fewer health problems and higher productivity, making subjective wellbeing a focal issue among researchers and governments. However, it is difficult to estimate how happy people were during previous centuries. Here we show that a method based on the quantitative analysis of natural language published over the past 200 years captures reliable patterns in historical subjective wellbeing. Using sentiment analysis on the basis of psychological valence norms, we compute a national valence index for the United Kingdom, the United States, Germany and Italy, indicating relative happiness in response to national and international wars and in comparison to historical trends in longevity and gross domestic product. We validate our method using Eurobarometer survey data from the 1970s and demonstrate robustness using words with stable historical meanings, diverse corpora (newspapers, magazines and books) and additional word norms. By providing a window on quantitative historical psychology, this approach could inform policy and economic history.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: Correlation of the NVI and aggregate life satisfaction data from the Eurobarometer survey.
Fig. 2: NVI through the period 1820–2009.

Data availability

The data necessary to reproduce the analyses presented in this article are provided at https://github.com/warwickpsych/NationalValenceIndex.

Code availability

The code necessary to reproduce the analyses presented in this article is provided at https://github.com/warwickpsych/NationalValenceIndex.

Change history

  • 14 November 2019

    An amendment to this paper has been published and can be accessed via a link at the top of the paper.

References

  1. 1.

    Di Tella, R., MacCulloch, R. J. & Oswald, A. J. Preferences over inflation and unemployment: evidence from surveys of happiness. Am. Econ. Rev. 91, 335–341 (2001).

  2. 2.

    Deaton, A. Income, health, and well-being around the world: evidence from the Gallup world poll. J. Econ. Perspect. 22, 53–72 (2008).

  3. 3.

    Stevenson, B. & Wolfers, J. Economic growth and subjective well-being: reassessing the Easterlin paradox. Brookings Pap. Econ. Ac. 39, 1–87 (2008).

  4. 4.

    Benjamin, D. J., Kimball, M. S., Heffetz, O. & Rees-Jones, A. What do you think would make you happier? What do you think you would choose? Am. Econ. Rev. 102, 2083–2110 (2012).

  5. 5.

    Proto, E. & Rustichini, A. A reassessment of the relationship between GDP and life satisfaction. PLoS One 8, e79358 (2013).

  6. 6.

    Bolt, J. & van Zanden, J. L. The Maddison Project: collaborative research on historical national accounts. Econ. Hist. Rev. 67, 627–651 (2014).

  7. 7.

    Broadberry, S., Campbell, B., Klein, A., Overton, M. & Van Leeuwen, B. British Economic Growth, 1270–1870: An Output-Based Approach School of Economics Discussion Paper http://hdl.handle.net/10871/13984 (University of Exeter, 2012).

  8. 8.

    Lin, Y. et al. Syntactic annotations for the Google Books Ngram Corpus. In Proc. ACL 2012 System Demonstrations 169–174 (Association for Computational Linguistics, 2012).

  9. 9.

    Michel, J.-B. et al. Quantitative analysis of culture using millions of digitized books. Science 331, 176–182 (2011).

  10. 10.

    Connor, B., Balasubramanyan, R., Routledge, B. R. & Smith, N. A. From tweets to polls: linking text sentiment to public opinion time series. International Conference on Web and Social Media, North America (ICWSM) 11, 122–129 (AAAI, 2010).

  11. 11.

    Bollen, J., Mao, H. & Zeng, X. Twitter mood predicts the stock market. J. Comput. Sci. 2, 1–8 (2011).

  12. 12.

    Golder, S. A. & Macy, M. W. Diurnal and seasonal mood vary with work, sleep, and daylength across diverse cultures. Science 333, 1878–1881 (2011).

  13. 13.

    Chmiel, A. et al. Collective emotions online and their influence on community life. PLoS One 6, e22207 (2011).

  14. 14.

    Eichstaedt, J. et al. Facebook language predicts depression in medical records. Proc. Natl Acad. Sci. USA 115, 11203–11208 (2018).

  15. 15.

    Dodds, P. S., Harris, K. D., Kloumann, I. M., Bliss, C. A. & Danforth, C. M. Temporal patterns of happiness and information in a global social network: hedonometrics and Twitter. PLoS One 6, e26752 (2011).

  16. 16.

    Thelwall, M., Buckley, K. & Paltoglou, G. Sentiment in Twitter events. J. Am. Soc. Inf. Sci. Technol. 62, 406–418 (2011).

  17. 17.

    Nguyen, T., Phung, D., Adams, B., Tran, T. & Venkatesh, S. Classification and pattern discovery of mood in weblogs. Adv. Knowl. Discov. Data Min. 6119, 283–290 (2010).

  18. 18.

    Feenstra, R. C., Inklaar, R. & Timmer, M. P. The next generation of the Penn World Table. Am. Econ. Rev. 105, 3150–3182 (2015).

  19. 19.

    van Zanden, J. L. et al. How Was Life? Global Well-Being Since 1820 (OECD Publishing, 2014).

  20. 20.

    Easterlin, R. A. Does economic growth improve the human lot? Some empirical evidence. Nations Househ. Econ. Growth 89, 89–125 (1974).

  21. 21.

    Easterlin, R. A., McVey, L. A., Switek, M., Sawangfa, O. & Zweig, J. S. The happiness–income paradox revisited. Proc. Natl Acad. Sci. USA 107, 22463–22468 (2010).

  22. 22.

    Kahneman, D. & Deaton, A. High income improves evaluation of life but not emotional well-being. Proc. Natl Acad. Sci. USA 107, 16489–16493 (2010).

  23. 23.

    Hills, T. T. The dark side of information proliferation. Perspect. Psychol. Sci. 14, 323–330 (2019).

  24. 24.

    Li, Y., Engelthaler, T., Siew, C. S. & Hills, T. T. The macroscope: a tool for examining the historical structure of language. Behav. Res. Methods 51, 1864–1877 (2019).

  25. 25.

    Hills, T. T. & Adelman, J. S. Recent evolution of learnability in American English from 1800 to 2000. Cognition 143, 87–92 (2015).

  26. 26.

    Jerven, M. An unlevel playing field: national income estimates and reciprocal comparison in global economic history. J. Glob. Hist. 7, 107–128 (2012).

  27. 27.

    Bradley, M. M. & Lang, P. J. Affective Norms for English Words (ANEW): Instruction Manual and Affective Ratings Technical Report C-1 (The Center for Research in Psychophysiology, Univ. Florida, 1999).

  28. 28.

    Nielsen, F. Å. A new ANEW: evaluation of a word list for sentiment analysis in microblogs. in Proceedings of the ESWC2011 Workshop on ‘Making Sense of Microposts’: Big things come in small packages 93–98 http://arxiv.org/abs/1103.2903 (2011).

  29. 29.

    Warriner, A. B., Kuperman, V. & Brysbaert, M. Norms of valence, arousal, and dominance for 13,915 English lemmas. Behav. Res. Methods 45, 1191–1207 (2013).

  30. 30.

    Schmidtke, D. S., Schröder, T., Jacobs, A. M. & Conrad, M. ANGST: affective norms for German sentiment terms, derived from the Affective Norms for English Words. Behav. Res. Methods 46, 1108–1118 (2014).

  31. 31.

    Montefinese, M., Ambrosini, E., Fairfield, B. & Mammarella, N. The adaptation of the Affective Norms for English Words (ANEW) for Italian. Behav. Res. Methods 46, 887–903 (2014).

  32. 32.

    Greenfield, P. M. The changing psychology of culture from 1800 through 2000. Psychol. Sci. 24, 1722–1731 (2013).

Download references

Acknowledgements

We thank colleagues for discussions on this research, especially S. Allen, S. Becker, S. Broadberry, N. Crafts, R. Duch, A. Oswald, L. Pascali, G. Ricco, D. Ronayne, J. Smith and T. Van Rens; and T. Engelthaler and L. Ying for research assistance. This work was supported by a Royal Society Wolfson Research Merit Award WM160074 (to T.T.H.), the Alan Turing Institute (to T.T.H. and C.I.S.), and The Center for Competitive Advantage in the Global Economy at the University of Warwick (to D.S. and E.P.). This research used cloud computing resources kindly provided through a Microsoft Azure for Research Award. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author information

T.T.H., E.P., D.S. and C.I.S. were involved in the study design, project planning, data analysis and writing the manuscript.

Correspondence to Thomas T. Hills.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information Primary Handling Editor: Stavroula Kousta.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

A correction to this article is available online at https://doi.org/10.1038/s41562-019-0781-5.

Supplementary information

Supplementary Information

Supplementary Notes, Supplementary Figs. 1–9, Supplementary Tables 1–15 and Supplementary References.

Reporting Summary

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hills, T.T., Proto, E., Sgroi, D. et al. Historical analysis of national subjective wellbeing using millions of digitized books. Nat Hum Behav 3, 1271–1275 (2019) doi:10.1038/s41562-019-0750-z

Download citation