Online reviews promise to provide people with immediate access to the wisdom of the crowds. Yet, half of all reviews on Amazon and Yelp provide the most positive rating possible, despite human behaviour being substantially more varied in nature. We term the challenge of discerning success within this sea of positive ratings the ‘positivity problem’. Positivity, however, is only one facet of individuals’ opinions. We propose that one solution to the positivity problem lies with the emotionality of people’s opinions. Using computational linguistics, we predict the box office revenue of nearly 2,400 movies, sales of 1.6 million books, new brand followers across two years of Super Bowl commercials, and real-world reservations at over 1,000 restaurants. Whereas star ratings are an unreliable predictor of success, emotionality from the very same reviews offers a consistent diagnostic signal. More emotional language was associated with more subsequent success.
Subscribe to Journal
Get full journal access for 1 year
only $8.25 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
The code for these analyses is available from the authors upon request.
Asch, S. E. Studies of independence and conformity: I. A minority of one against a unanimous majority. Psychol. Monogr. Gen. Appl. 70, 1–70 (1956).
Sherif, M. A study of some social factors in perception. Arch. Psychol. Columbia Univ. 187, 60 (1935).
Simonson, I. & Rosen, E. Absolute Value: What Really Influences Customers in the Age of (Nearly) Perfect Information (HarperBusiness, 2014).
Smith, A. & Anderson, M. Online Shopping and E-Commerce (Pew Research Center, 2016); http://assets.pewresearch.org/wp-content/uploads/sites/14/2016/12/16113209/PI_2016.12.19_Online-Shopping_FINAL.pdf
Hu, N., Zhang, J. & Pavlou, P. A. Overcoming the J-shaped distribution of product reviews. Commun. ACM 52, 144–147 (2009).
Woolf, M. Playing with 80 million Amazon product review ratings using Apache Spark. minimaxir http://minimaxir.com/2017/01/amazon-spark/ (2017).
McAuley, J., Pandey, R. & Leskovec, J. Inferring networks of substitutable and complementary products. in Proc. 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 785–794 (ACM, 2015).
Yelp Factsheet (Yelp, 2017); https://www.yelp.com/factsheet
Athey, S., Castillo, J. C. & Knoepfle, D. Service quality in the gig economy: empirical evidence about driving quality at Uber. White Paper. https://doi.org/10.2139/ssrn.3499781 (2019).
Babić Rosario, A., Sotgiu, F., De Valck, K. & Bijmolt, T. H. A. The effect of electronic word of mouth on sales: a meta-analytic review of platform, product, and metric factors. J. Mark. Res. 53, 297–318 (2015).
Floyd, K., Freling, R., Alhoqail, S., Cho, H. Y. & Freling, T. How online product reviews affect retail sales: a meta-analysis. J. Retail. 90, 217–232 (2014).
You, Y., Vadakkepatt, G. G. & Joshi, A. M. A meta-analysis of electronic word-of-mouth elasticity. J. Mark. 79, 19–39 (2015).
de Langhe, B., Fernbach, P. M. & Lichtenstein, D. R. Navigating by the stars: investigating the actual and perceived validity of online user ratings. J. Consum. Res. 42, 817–833 (2016).
Holbrook, M. B. & Addis, M. Taste versus the market: an extension of research on the consumption of popular culture. J. Consum. Res. 34, 415–424 (2007).
Fowler, G. A. When 4.3 stars is average: the Internet’s grade-inflation problem; Netflix is going with simpler thumbs-up or thumbs-down reviews, while online star ratings for many products have lost their meaning. Wall Street Journal https://www.wsj.com/articles/when-4-3-stars-is-average-the-internets-grade-inflation-problem-1491414200 (5 April, 2017).
Pang, B., Lee, L. & Vaithyanathan, S. Thumbs up? Sentiment classification using machine learning techniques. in Proc. ACL-02 Conference on Empirical Methods in Natural Language Processing 10, 79–86 (Association for Computational Linguistics, 2002).
Petty, R. E. & Krosnick, J. A. Attitude Strength: Antecedents and Consequences (Psychology Press, 1995).
Warriner, A. B., Kuperman, V. & Brysbaert, M. Norms of valence, arousal, and dominance for 13,915 English lemmas. Behav. Res. Methods 45, 1191–1207 (2013).
Wicker, A. W. Attitudes versus actions: the relationship of verbal and overt behavioral responses to attitude objects. J. Soc. Issues 25, 41–78 (1969).
Visser, P. S., Bizer, G. Y. & Krosnick, J. A. in Advances in Experimental Social Psychology Vol. 38 (ed. Zanna, M. P.) 1–61 (Academic Press, 2006).
Petty, R. E., Fabrigar, L. R. & Wegener, D. T. in Handbook of Affective Sciences (eds Davidson, R. J. et al.) 752–772 (Oxford Univ. Press, 2003).
Zanna, M. P. & Rempel, J. K. in The Social Psychology of Knowledge (eds Bar-Tal, D. & Kruglanski, A. W.) 315–334 (Cambridge Univ. Press, 1988).
Haddock, G., Zanna, M. P. & Esses, V. M. Assessing the structure of prejudicial attitudes: the case of attitudes toward homosexuals. J. Pers. Soc. Psychol. 65, 1105–1118 (1993).
Maio, G. R. & Esses, V. M. The need for affect: individual differences in the motivation to approach or avoid emotions. J. Pers. 69, 583–614 (2001).
Rocklage, M. D., Rucker, D. D. & Nordgren, L. F. The Evaluative Lexicon 2.0: the measurement of emotionality, extremity, and valence in language. Behav. Res. Methods 50, 1327–1344 (2018).
Rocklage, M. D. & Fazio, R. H. The evaluative lexicon: adjective use as a means of assessing and distinguishing attitude valence, extremity, and emotionality. J. Exp. Soc. Psychol. 56, 214–227 (2015).
Lavine, H., Thomsen, C. J., Zanna, M. P. & Borgida, E. On the primacy of affect in the determination of attitudes and behavior: the moderating role of affective-cognitive ambivalence. J. Exp. Soc. Psychol. 34, 398–421 (1998).
Rocklage, M. D. & Fazio, R. H. Attitude accessibility as a function of emotionality. Pers. Soc. Psychol. Bull. 44, 508–520 (2018).
Rocklage, M. D. & Fazio, R. H. On the dominance of attitude emotionality. Pers. Soc. Psychol. Bull. 42, 259–270 (2016).
Rocklage, M. D. & Luttrell, A. Attitudes based on feelings: fixed or fleeting? Psychol. Sci. https://doi.org/10.1177/0956797620965532 (2021).
Tooby, J. & Cosmides, L. The past explains the present. Ethol. Sociobiol. 11, 375–424 (1990).
Ekman, P. E. & Davidson, R. J. The Nature of Emotion: Fundamental Questions (Oxford Univ. Press, 1994).
Fazio, R. H. in Attitude Strength: Antecedents and Consequences (eds Petty, R. E. & Krosnick, J. A.) 247–282 (Lawrence Erlbaum Associates, 1995).
Schwarz, N. in Handbook of Theories of Social Psychology (eds Van Lange, P. et al.) 289–308 (Sage, 2012).
Fazio, R. H. Attitudes as object–evaluation associations of varying strength. Soc. Cogn. 25, 603–637 (2007).
Frijda, N. H. & Mesquita, B. in Emotion and Culture: Empirical Studies of Mutual Influence (eds Kitayama, S. & Markus, H. R.) 51–87 (American Psychological Association, 1994).
Keltner, D. & Haidt, J. Social functions of emotions at four levels of analysis. Cogn. Emot. 13, 505–521 (1999).
Rocklage, M. D., Rucker, D. D. & Nordgren, L. F. Persuasion, emotion, and language: the intent to persuade transforms language via emotionality. Psychol. Sci. 29, 749–760 (2018).
Van Kleef, G. A., De Dreu, C. K. W. & Manstead, A. S. R. The interpersonal effects of anger and happiness in negotiations. J. Pers. Soc. Psychol. 86, 57–76 (2004).
Andrade, E. B. & Ho, T.-H. Gaming emotions in social interactions. J. Consum. Res. 36, 539–552 (2009).
Lee, Y.-J., Hosanagar, K. & Tan, Y. Do I follow my friends or the crowd? Information cascades in online movie ratings. Manage. Sci. 61, 2241–2258 (2015).
Schlosser, A. E. Posting versus lurking: communicating in a multiple audience context. J. Consum. Res. 32, 260–265 (2005).
Moe, W. W. & Schweidel, D. A. Online product opinions: incidence, evaluation, and evolution. Mark. Sci. 31, 372–386 (2012).
Russell, J. A. & Barrett, L. F. Core affect, prototypical emotional episodes, and other things called emotion: dissecting the elephant. J. Pers. Soc. Psychol. 76, 805–819 (1999).
Ad Meter 2017 FAQ (Ad Meter, 2017); http://admeter.usatoday.com/2017/01/17/ad-meter-2017-faq/
Asur, S. & Huberman, B. A. Predicting the future with social media. in Proc. 2010 IEEE/ACM International Conference on Web Intelligence-Intelligent Agent Technology (WI-IAT) 1, 492–499 (IEEE Computer Society, 2010).
O’Connor, B., Balasubramanyan, R., Routledge, B. & Smith, N. From tweets to polls: linking text sentiment to public opinion time series. in Proc. 4th AAAI Conference on Weblogs and Social Media 11, 122–129 (AAAI Press, 2010).
Pham, M. T., Cohen, J. B., Pracejus, J. W. & Hughes, G. D. Affect monitoring and the primacy of feelings in judgment. J. Consum. Res. 28, 167–188 (2001).
Roskos-Ewoldsen, D. R. & Fazio, R. H. On the orienting value of attitudes: attitude accessibility as a determinant of an object’s attraction of visual attention. J. Pers. Soc. Psychol. 63, 198–211 (1992).
Berger, J. & Milkman, K. L. What makes online content viral? J. Mark. Res. 49, 192–205 (2012).
Castelvecchi, D. Can we open the black box of AI? Nature 538, 20–23 (2016).
Python Language Reference, version 2.7. http://www.python.org (Python Software Foundation, 2017).
Amazon Customer Reviews Dataset (Amazon, 2020); https://s3.amazonaws.com/amazon-reviews-pds/readme.html
Ni, J., Li, J. & McAuley, J. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Proc. 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) 188–197 (Association for Computational Linguistics, 2019).
Filloon, W. In the battle for restaurant reservations, OpenTable is still way ahead. Eater https://www.eater.com/2018/9/24/17883688/opentable-resy-online-reservations-app-danny-meyer (2018).
We received no specific funding for this work. We thank Internet Video Archive LLC for their assistance in providing access to the movie data and metadata from Study 1.
The authors declare no competing interests.
Peer review information Nature Human Behaviour thanks Jonah Berger, Saif Mohammad and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Rocklage, M.D., Rucker, D.D. & Nordgren, L.F. Mass-scale emotionality reveals human behaviour and marketplace success. Nat Hum Behav (2021). https://doi.org/10.1038/s41562-021-01098-5