Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Mass-scale emotionality reveals human behaviour and marketplace success

Nature Human Behaviour volume 5, pages 1323–1329 (2021)Cite this article

Subjects

Abstract

Online reviews promise to provide people with immediate access to the wisdom of the crowds. Yet, half of all reviews on Amazon and Yelp provide the most positive rating possible, despite human behaviour being substantially more varied in nature. We term the challenge of discerning success within this sea of positive ratings the ‘positivity problem’. Positivity, however, is only one facet of individuals’ opinions. We propose that one solution to the positivity problem lies with the emotionality of people’s opinions. Using computational linguistics, we predict the box office revenue of nearly 2,400 movies, sales of 1.6 million books, new brand followers across two years of Super Bowl commercials, and real-world reservations at over 1,000 restaurants. Whereas star ratings are an unreliable predictor of success, emotionality from the very same reviews offers a consistent diagnostic signal. More emotional language was associated with more subsequent success.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Predicting movie revenue.
Fig. 2: Predicting restaurant table reservations.

Data availability

The data for Study 2 are available from Amazon (https://s3.amazonaws.com/amazon-reviews-pds/readme.html). The data from Studies 1, 3 and 4 are publicly hosted on www.metacritic.com (Study 1), www.twitter.com (Study 3), www.facebook.com (Study 3), www.opentable.com (Study 4) and www.yelp.com (Study 4). For purposes of verification and reproducibility, readers will be provided with the code and anonymized aggregated data results upon request. Although the data are publicly available, their use is governed by each site’s terms of use. Those interested in the original data should contact the site administrators for permission.

Code availability

The code for these analyses is available from the authors upon request.

References

  1. Asch, S. E. Studies of independence and conformity: I. A minority of one against a unanimous majority. Psychol. Monogr. Gen. Appl. 70, 1–70 (1956).

    Article  Google Scholar 

  2. Sherif, M. A study of some social factors in perception. Arch. Psychol. Columbia Univ. 187, 60 (1935).

    Google Scholar 

  3. Simonson, I. & Rosen, E. Absolute Value: What Really Influences Customers in the Age of (Nearly) Perfect Information (HarperBusiness, 2014).

  4. Smith, A. & Anderson, M. Online Shopping and E-Commerce (Pew Research Center, 2016); http://assets.pewresearch.org/wp-content/uploads/sites/14/2016/12/16113209/PI_2016.12.19_Online-Shopping_FINAL.pdf

  5. Hu, N., Zhang, J. & Pavlou, P. A. Overcoming the J-shaped distribution of product reviews. Commun. ACM 52, 144–147 (2009).

    Article  Google Scholar 

  6. Woolf, M. Playing with 80 million Amazon product review ratings using Apache Spark. minimaxir http://minimaxir.com/2017/01/amazon-spark/ (2017).

  7. McAuley, J., Pandey, R. & Leskovec, J. Inferring networks of substitutable and complementary products. in Proc. 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 785–794 (ACM, 2015).

  8. Yelp Factsheet (Yelp, 2017); https://www.yelp.com/factsheet

  9. Athey, S., Castillo, J. C. & Knoepfle, D. Service quality in the gig economy: empirical evidence about driving quality at Uber. White Paper. https://doi.org/10.2139/ssrn.3499781 (2019).

  10. Babić Rosario, A., Sotgiu, F., De Valck, K. & Bijmolt, T. H. A. The effect of electronic word of mouth on sales: a meta-analytic review of platform, product, and metric factors. J. Mark. Res. 53, 297–318 (2015).

    Article  Google Scholar 

  11. Floyd, K., Freling, R., Alhoqail, S., Cho, H. Y. & Freling, T. How online product reviews affect retail sales: a meta-analysis. J. Retail. 90, 217–232 (2014).

    Article  Google Scholar 

  12. You, Y., Vadakkepatt, G. G. & Joshi, A. M. A meta-analysis of electronic word-of-mouth elasticity. J. Mark. 79, 19–39 (2015).

    Article  Google Scholar 

  13. de Langhe, B., Fernbach, P. M. & Lichtenstein, D. R. Navigating by the stars: investigating the actual and perceived validity of online user ratings. J. Consum. Res. 42, 817–833 (2016).

    Article  Google Scholar 

  14. Holbrook, M. B. & Addis, M. Taste versus the market: an extension of research on the consumption of popular culture. J. Consum. Res. 34, 415–424 (2007).

    Article  Google Scholar 

  15. Fowler, G. A. When 4.3 stars is average: the Internet’s grade-inflation problem; Netflix is going with simpler thumbs-up or thumbs-down reviews, while online star ratings for many products have lost their meaning. Wall Street Journal https://www.wsj.com/articles/when-4-3-stars-is-average-the-internets-grade-inflation-problem-1491414200 (5 April, 2017).

  16. Pang, B., Lee, L. & Vaithyanathan, S. Thumbs up? Sentiment classification using machine learning techniques. in Proc. ACL-02 Conference on Empirical Methods in Natural Language Processing 10, 79–86 (Association for Computational Linguistics, 2002).

  17. Petty, R. E. & Krosnick, J. A. Attitude Strength: Antecedents and Consequences (Psychology Press, 1995).

  18. Warriner, A. B., Kuperman, V. & Brysbaert, M. Norms of valence, arousal, and dominance for 13,915 English lemmas. Behav. Res. Methods 45, 1191–1207 (2013).

    Article  Google Scholar 

  19. Wicker, A. W. Attitudes versus actions: the relationship of verbal and overt behavioral responses to attitude objects. J. Soc. Issues 25, 41–78 (1969).

    Article  Google Scholar 

  20. Visser, P. S., Bizer, G. Y. & Krosnick, J. A. in Advances in Experimental Social Psychology Vol. 38 (ed. Zanna, M. P.) 1–61 (Academic Press, 2006).

  21. Petty, R. E., Fabrigar, L. R. & Wegener, D. T. in Handbook of Affective Sciences (eds Davidson, R. J. et al.) 752–772 (Oxford Univ. Press, 2003).

  22. Zanna, M. P. & Rempel, J. K. in The Social Psychology of Knowledge (eds Bar-Tal, D. & Kruglanski, A. W.) 315–334 (Cambridge Univ. Press, 1988).

  23. Haddock, G., Zanna, M. P. & Esses, V. M. Assessing the structure of prejudicial attitudes: the case of attitudes toward homosexuals. J. Pers. Soc. Psychol. 65, 1105–1118 (1993).

    Article  Google Scholar 

  24. Maio, G. R. & Esses, V. M. The need for affect: individual differences in the motivation to approach or avoid emotions. J. Pers. 69, 583–614 (2001).

    Article  CAS  Google Scholar 

  25. Rocklage, M. D., Rucker, D. D. & Nordgren, L. F. The Evaluative Lexicon 2.0: the measurement of emotionality, extremity, and valence in language. Behav. Res. Methods 50, 1327–1344 (2018).

    Article  Google Scholar 

  26. Rocklage, M. D. & Fazio, R. H. The evaluative lexicon: adjective use as a means of assessing and distinguishing attitude valence, extremity, and emotionality. J. Exp. Soc. Psychol. 56, 214–227 (2015).

    Article  Google Scholar 

  27. Lavine, H., Thomsen, C. J., Zanna, M. P. & Borgida, E. On the primacy of affect in the determination of attitudes and behavior: the moderating role of affective-cognitive ambivalence. J. Exp. Soc. Psychol. 34, 398–421 (1998).

    Article  Google Scholar 

  28. Rocklage, M. D. & Fazio, R. H. Attitude accessibility as a function of emotionality. Pers. Soc. Psychol. Bull. 44, 508–520 (2018).

    Article  Google Scholar 

  29. Rocklage, M. D. & Fazio, R. H. On the dominance of attitude emotionality. Pers. Soc. Psychol. Bull. 42, 259–270 (2016).

    Article  Google Scholar 

  30. Rocklage, M. D. & Luttrell, A. Attitudes based on feelings: fixed or fleeting? Psychol. Sci. https://doi.org/10.1177/0956797620965532 (2021).

  31. Tooby, J. & Cosmides, L. The past explains the present. Ethol. Sociobiol. 11, 375–424 (1990).

    Article  Google Scholar 

  32. Ekman, P. E. & Davidson, R. J. The Nature of Emotion: Fundamental Questions (Oxford Univ. Press, 1994).

  33. Fazio, R. H. in Attitude Strength: Antecedents and Consequences (eds Petty, R. E. & Krosnick, J. A.) 247–282 (Lawrence Erlbaum Associates, 1995).

  34. Schwarz, N. in Handbook of Theories of Social Psychology (eds Van Lange, P. et al.) 289–308 (Sage, 2012).

  35. Fazio, R. H. Attitudes as object–evaluation associations of varying strength. Soc. Cogn. 25, 603–637 (2007).

    Article  Google Scholar 

  36. Frijda, N. H. & Mesquita, B. in Emotion and Culture: Empirical Studies of Mutual Influence (eds Kitayama, S. & Markus, H. R.) 51–87 (American Psychological Association, 1994).

  37. Keltner, D. & Haidt, J. Social functions of emotions at four levels of analysis. Cogn. Emot. 13, 505–521 (1999).

    Article  Google Scholar 

  38. Rocklage, M. D., Rucker, D. D. & Nordgren, L. F. Persuasion, emotion, and language: the intent to persuade transforms language via emotionality. Psychol. Sci. 29, 749–760 (2018).

    Article  Google Scholar 

  39. Van Kleef, G. A., De Dreu, C. K. W. & Manstead, A. S. R. The interpersonal effects of anger and happiness in negotiations. J. Pers. Soc. Psychol. 86, 57–76 (2004).

    Article  Google Scholar 

  40. Andrade, E. B. & Ho, T.-H. Gaming emotions in social interactions. J. Consum. Res. 36, 539–552 (2009).

    Article  Google Scholar 

  41. Lee, Y.-J., Hosanagar, K. & Tan, Y. Do I follow my friends or the crowd? Information cascades in online movie ratings. Manage. Sci. 61, 2241–2258 (2015).

    Article  Google Scholar 

  42. Schlosser, A. E. Posting versus lurking: communicating in a multiple audience context. J. Consum. Res. 32, 260–265 (2005).

    Article  Google Scholar 

  43. Moe, W. W. & Schweidel, D. A. Online product opinions: incidence, evaluation, and evolution. Mark. Sci. 31, 372–386 (2012).

    Article  Google Scholar 

  44. Russell, J. A. & Barrett, L. F. Core affect, prototypical emotional episodes, and other things called emotion: dissecting the elephant. J. Pers. Soc. Psychol. 76, 805–819 (1999).

    Article  CAS  Google Scholar 

  45. Ad Meter https://finance.yahoo.com/news/usa-today-commemorate-30th-ad-150000342.html (2018).

  46. Ad Meter 2017 FAQ (Ad Meter, 2017); http://admeter.usatoday.com/2017/01/17/ad-meter-2017-faq/

  47. Asur, S. & Huberman, B. A. Predicting the future with social media. in Proc. 2010 IEEE/ACM International Conference on Web Intelligence-Intelligent Agent Technology (WI-IAT) 1, 492–499 (IEEE Computer Society, 2010).

  48. O’Connor, B., Balasubramanyan, R., Routledge, B. & Smith, N. From tweets to polls: linking text sentiment to public opinion time series. in Proc. 4th AAAI Conference on Weblogs and Social Media 11, 122–129 (AAAI Press, 2010).

  49. Pham, M. T., Cohen, J. B., Pracejus, J. W. & Hughes, G. D. Affect monitoring and the primacy of feelings in judgment. J. Consum. Res. 28, 167–188 (2001).

    Article  Google Scholar 

  50. Roskos-Ewoldsen, D. R. & Fazio, R. H. On the orienting value of attitudes: attitude accessibility as a determinant of an object’s attraction of visual attention. J. Pers. Soc. Psychol. 63, 198–211 (1992).

    Article  CAS  Google Scholar 

  51. Berger, J. & Milkman, K. L. What makes online content viral? J. Mark. Res. 49, 192–205 (2012).

    Article  Google Scholar 

  52. Castelvecchi, D. Can we open the black box of AI? Nature 538, 20–23 (2016).

    Article  CAS  Google Scholar 

  53. Python Language Reference, version 2.7. http://www.python.org (Python Software Foundation, 2017).

  54. Amazon Customer Reviews Dataset (Amazon, 2020); https://s3.amazonaws.com/amazon-reviews-pds/readme.html

  55. Ni, J., Li, J. & McAuley, J. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Proc. 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) 188–197 (Association for Computational Linguistics, 2019).

  56. Filloon, W. In the battle for restaurant reservations, OpenTable is still way ahead. Eater https://www.eater.com/2018/9/24/17883688/opentable-resy-online-reservations-app-danny-meyer (2018).

Download references

Acknowledgements

We received no specific funding for this work. We thank Internet Video Archive LLC for their assistance in providing access to the movie data and metadata from Study 1.

Author information

Authors and Affiliations

  1. College of Management, University of Massachusetts, Boston, MA, USA

    Matthew D. Rocklage

  2. Kellogg School of Management, Northwestern University, Evanston, IL, USA

    Derek D. Rucker & Loran F. Nordgren

Authors
  1. Matthew D. Rocklage
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Derek D. Rucker
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Loran F. Nordgren
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

M.D.R., D.D.R. and L.F.N. conceptualized the work. M.D.R. obtained and analysed the data with collaboration from D.D.R. and L.F.N. M.D.R., D.D.R. and L.F.N. wrote the manuscript.

Corresponding author

Correspondence to Matthew D. Rocklage.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information Nature Human Behaviour thanks Jonah Berger, Saif Mohammad and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Methods, Supplementary Results and Supplementary Tables 1–10.

Reporting Summary

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rocklage, M.D., Rucker, D.D. & Nordgren, L.F. Mass-scale emotionality reveals human behaviour and marketplace success. Nat Hum Behav 5, 1323–1329 (2021). https://doi.org/10.1038/s41562-021-01098-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41562-021-01098-5

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing