In addition to improving quality of life, higher subjective wellbeing leads to fewer health problems and higher productivity, making subjective wellbeing a focal issue among researchers and governments. However, it is difficult to estimate how happy people were during previous centuries. Here we show that a method based on the quantitative analysis of natural language published over the past 200 years captures reliable patterns in historical subjective wellbeing. Using sentiment analysis on the basis of psychological valence norms, we compute a national valence index for the United Kingdom, the United States, Germany and Italy, indicating relative happiness in response to national and international wars and in comparison to historical trends in longevity and gross domestic product. We validate our method using Eurobarometer survey data from the 1970s and demonstrate robustness using words with stable historical meanings, diverse corpora (newspapers, magazines and books) and additional word norms. By providing a window on quantitative historical psychology, this approach could inform policy and economic history.
Subscribe to Journal
Get full journal access for 1 year
only $8.25 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
The data necessary to reproduce the analyses presented in this article are provided at https://github.com/warwickpsych/NationalValenceIndex.
The code necessary to reproduce the analyses presented in this article is provided at https://github.com/warwickpsych/NationalValenceIndex.
Di Tella, R., MacCulloch, R. J. & Oswald, A. J. Preferences over inflation and unemployment: evidence from surveys of happiness. Am. Econ. Rev. 91, 335–341 (2001).
Deaton, A. Income, health, and well-being around the world: evidence from the Gallup world poll. J. Econ. Perspect. 22, 53–72 (2008).
Stevenson, B. & Wolfers, J. Economic growth and subjective well-being: reassessing the Easterlin paradox. Brookings Pap. Econ. Ac. 39, 1–87 (2008).
Benjamin, D. J., Kimball, M. S., Heffetz, O. & Rees-Jones, A. What do you think would make you happier? What do you think you would choose? Am. Econ. Rev. 102, 2083–2110 (2012).
Proto, E. & Rustichini, A. A reassessment of the relationship between GDP and life satisfaction. PLoS One 8, e79358 (2013).
Bolt, J. & van Zanden, J. L. The Maddison Project: collaborative research on historical national accounts. Econ. Hist. Rev. 67, 627–651 (2014).
Broadberry, S., Campbell, B., Klein, A., Overton, M. & Van Leeuwen, B. British Economic Growth, 1270–1870: An Output-Based Approach School of Economics Discussion Paper http://hdl.handle.net/10871/13984 (University of Exeter, 2012).
Lin, Y. et al. Syntactic annotations for the Google Books Ngram Corpus. In Proc. ACL 2012 System Demonstrations 169–174 (Association for Computational Linguistics, 2012).
Michel, J.-B. et al. Quantitative analysis of culture using millions of digitized books. Science 331, 176–182 (2011).
Connor, B., Balasubramanyan, R., Routledge, B. R. & Smith, N. A. From tweets to polls: linking text sentiment to public opinion time series. International Conference on Web and Social Media, North America (ICWSM) 11, 122–129 (AAAI, 2010).
Bollen, J., Mao, H. & Zeng, X. Twitter mood predicts the stock market. J. Comput. Sci. 2, 1–8 (2011).
Golder, S. A. & Macy, M. W. Diurnal and seasonal mood vary with work, sleep, and daylength across diverse cultures. Science 333, 1878–1881 (2011).
Chmiel, A. et al. Collective emotions online and their influence on community life. PLoS One 6, e22207 (2011).
Eichstaedt, J. et al. Facebook language predicts depression in medical records. Proc. Natl Acad. Sci. USA 115, 11203–11208 (2018).
Dodds, P. S., Harris, K. D., Kloumann, I. M., Bliss, C. A. & Danforth, C. M. Temporal patterns of happiness and information in a global social network: hedonometrics and Twitter. PLoS One 6, e26752 (2011).
Thelwall, M., Buckley, K. & Paltoglou, G. Sentiment in Twitter events. J. Am. Soc. Inf. Sci. Technol. 62, 406–418 (2011).
Nguyen, T., Phung, D., Adams, B., Tran, T. & Venkatesh, S. Classification and pattern discovery of mood in weblogs. Adv. Knowl. Discov. Data Min. 6119, 283–290 (2010).
Feenstra, R. C., Inklaar, R. & Timmer, M. P. The next generation of the Penn World Table. Am. Econ. Rev. 105, 3150–3182 (2015).
van Zanden, J. L. et al. How Was Life? Global Well-Being Since 1820 (OECD Publishing, 2014).
Easterlin, R. A. Does economic growth improve the human lot? Some empirical evidence. Nations Househ. Econ. Growth 89, 89–125 (1974).
Easterlin, R. A., McVey, L. A., Switek, M., Sawangfa, O. & Zweig, J. S. The happiness–income paradox revisited. Proc. Natl Acad. Sci. USA 107, 22463–22468 (2010).
Kahneman, D. & Deaton, A. High income improves evaluation of life but not emotional well-being. Proc. Natl Acad. Sci. USA 107, 16489–16493 (2010).
Hills, T. T. The dark side of information proliferation. Perspect. Psychol. Sci. 14, 323–330 (2019).
Li, Y., Engelthaler, T., Siew, C. S. & Hills, T. T. The macroscope: a tool for examining the historical structure of language. Behav. Res. Methods 51, 1864–1877 (2019).
Hills, T. T. & Adelman, J. S. Recent evolution of learnability in American English from 1800 to 2000. Cognition 143, 87–92 (2015).
Jerven, M. An unlevel playing field: national income estimates and reciprocal comparison in global economic history. J. Glob. Hist. 7, 107–128 (2012).
Bradley, M. M. & Lang, P. J. Affective Norms for English Words (ANEW): Instruction Manual and Affective Ratings Technical Report C-1 (The Center for Research in Psychophysiology, Univ. Florida, 1999).
Nielsen, F. Å. A new ANEW: evaluation of a word list for sentiment analysis in microblogs. in Proceedings of the ESWC2011 Workshop on ‘Making Sense of Microposts’: Big things come in small packages 93–98 http://arxiv.org/abs/1103.2903 (2011).
Warriner, A. B., Kuperman, V. & Brysbaert, M. Norms of valence, arousal, and dominance for 13,915 English lemmas. Behav. Res. Methods 45, 1191–1207 (2013).
Schmidtke, D. S., Schröder, T., Jacobs, A. M. & Conrad, M. ANGST: affective norms for German sentiment terms, derived from the Affective Norms for English Words. Behav. Res. Methods 46, 1108–1118 (2014).
Montefinese, M., Ambrosini, E., Fairfield, B. & Mammarella, N. The adaptation of the Affective Norms for English Words (ANEW) for Italian. Behav. Res. Methods 46, 887–903 (2014).
Greenfield, P. M. The changing psychology of culture from 1800 through 2000. Psychol. Sci. 24, 1722–1731 (2013).
We thank colleagues for discussions on this research, especially S. Allen, S. Becker, S. Broadberry, N. Crafts, R. Duch, A. Oswald, L. Pascali, G. Ricco, D. Ronayne, J. Smith and T. Van Rens; and T. Engelthaler and L. Ying for research assistance. This work was supported by a Royal Society Wolfson Research Merit Award WM160074 (to T.T.H.), the Alan Turing Institute (to T.T.H. and C.I.S.), and The Center for Competitive Advantage in the Global Economy at the University of Warwick (to D.S. and E.P.). This research used cloud computing resources kindly provided through a Microsoft Azure for Research Award. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.
The authors declare no competing interests.
Peer review information Primary Handling Editor: Stavroula Kousta.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Hills, T.T., Proto, E., Sgroi, D. et al. Historical analysis of national subjective wellbeing using millions of digitized books. Nat Hum Behav 3, 1271–1275 (2019). https://doi.org/10.1038/s41562-019-0750-z
Converting the British Library’s Catalogue of British and Irish Newspapers into a Public Domain Dataset: Processes and Applications
Journal of Open Humanities Data (2021)
Journal of Public Economics (2021)
EPJ Data Science (2020)
Proceedings of the National Academy of Sciences (2020)
SSRN Electronic Journal (2019)