Standardized classroom experiments provide evidence about how well scientific results reproduce when nearly identical methods are used. We use a sample of around 20,000 observations to test reproducibility of behaviour in trading and ultimatum bargaining. Double-auction results are highly reproducible and are close to equilibrium predictions about prices and quantities from economic theory. Our sample also shows robust correlations between individual surplus and trading order, and autocorrelation of successive price changes, which test different theories of price dynamics. In ultimatum bargaining, the large dataset provides sufficient power to identify that equal-split offers are accepted more often and more quickly than slightly unequal offers. Our results imply a general consistency of results across a variety of different countries and cultures in two of the most commonly used designs in experimental economics.
This is a preview of subscription content, access via your institution
Subscribe to Nature+
Get immediate online access to the entire Nature family of 50+ journals
Subscribe to Journal
Get full journal access for 1 year
only $9.92 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Get time limited or full article access on ReadCube.
All prices are NET prices.
Double auction and ultimatum game data can be found on the Open Science Framework https://osf.io/9mfws/.
Code for all analyses can be found on the Open Science Framework https://osf.io/9mfws/.
Open Science Collaboration. Estimating the reproducibility of psychological science. Science 349, aac4716 (2015).
Camerer, C. F. et al. Evaluating replicability of laboratory experiments in economics. Science 351, 1433–1436 (2016).
Camerer, C. F. et al. Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nat. Hum. Behav. 2, 637–644 (2018).
Zwaan, R. A., Etz, A., Lucas, R. E. & Donnellan, M. B. Making replication mainstream. Behav. Brain Sci. 41, e120 (2018).
Sanfey, A. G., Rilling, J. K., Aronson, J. A., Nystrom, L. E. & Cohen, J. D. The neural basis of economic decision-making in the ultimatum game. Science 300, 1755–1758 (2003).
Henrich, J. et al. Costly punishment across human societies. Science 312, 1767–1770 (2006).
Jensen, K., Call, J. & Tomasello, M. Chimpanzees are rational maximizers in an ultimatum game. Science 318, 107–109 (2007).
Hartley, C. & Fisher, S. Do children with autism spectrum disorder share fairly and reciprocally? J. Autism Dev. Disord. 48, 2714–2726 (2018).
O’Callaghan, C. et al. Fair play: social norm compliance failures in behavioural variant frontotemporal dementia. Brain 139, 204–216 (2015).
Morewedge, C. K., Krishnamurti, T. & Ariely, D. Focused on fairness: alcohol intoxication increases the costly rejection of inequitable rewards. J. Exp. Soc. Psychol. 50, 15–20 (2014).
Kirk, U., Downar, J. & Montague, P. R. Interoception drives increased rational decision-making in meditators playing the ultimatum game. Front. Neurosci. 5, 49 (2011).
Crockett, M. J., Clark, L., Tabibnia, G., Lieberman, M. D. & Robbins, T. W. Serotonin modulates behavioral reactions to unfairness. Science 320, 1739–1739 (2008).
Krajbich, I., Adolphs, R., Tranel, D., Denburg, N. L. & Camerer, C. F. Economic games quantify diminished sense of guilt in patients with damage to the prefrontal cortex. J. Neurosci. 29, 2188–2192 (2009).
Koenigs, M. & Tranel, D. Irrational economic decision-making after ventromedial prefrontal damage: evidence from the ultimatum game. J. Neurosci. 27, 951–956 (2007).
Dehaene, S. The Number Sense: How the Mind Creates Mathematics (Oxford Univ. Press, 2011).
Dehaene, S. & Mehler, J. Cross-linguistic regularities in the frequency of number words. Cognition 43, 1–29 (1992).
Frydman, C. & Jin, L. Efficient coding and risky choice. Preprint at SSRN https://doi.org/10.2139/ssrn.3270773 (2019).
Simon, H. A. Invariants of human behavior. Annu. Rev. Psychol. 41, 1–20 (1990).
Smith, V. L. The two faces of Adam Smith. South. Econ. J. 65, 2–19 (1998).
Smith, A. An Inquiry Into the Nature and Causes of the Wealth of Nations Vol. 1 (W. Strahan and T. Cadell, 1776).
Sobel, J. Generous actors, selfish actions: markets with other-regarding preferences. Int. Rev. Econ. 56, 3–16 (2009).
Dufwenberg, M., Heidhues, P., Kirchsteiger, G., Riedel, F. & Sobel, J. Other-regarding preferences in general equilibrium. Rev. Econ. Stud. 78, 613–639 (2011).
Chamberlin, E. H. An experimental imperfect market. J. Polit. Econ. 56, 95–108 (1948).
Smith, V. L. An experimental study of competitive market behavior. J. Polit. Econ. 70, 111–137 (1962).
Plott, C. R. & Smith, V. L. An experimental examination of two exchange institutions. Rev. Econ. Stud. 45, 133–153 (1978).
Smith, V. L. & Walker, J. M. Monetary rewards and decision cost in experimental economics. Econ. Inq. 31, 245–261 (1993).
Hertwig, R. & Ortmann, A. Experimental practices in economics: a methodological challenge for psychologists? Behav. Brain Sci. 24, 383–403 (2001).
Camerer, C. F. & Hogarth, R. M. The effects of financial incentives in experiments: a review and capital–labor–production framework. J. Risk. Uncertain. 19, 7–42 (1999).
Tompkinson, P. & Bethwaite, J. The ultimatum game: raising the stakes. J. Econ. Behav. Organ. 27, 439–451 (1995).
Güth, W., Schmittberger, R. & Schwarze, B. An experimental analysis of ultimatum bargaining. J. Econ. Behav. Organ. 3, 367–388 (1982).
Tisserand, J.-C. et al. Ultimatum game: a meta-analysis of the past three decades of experimental research. In Proceedings of International Academic Conferences, 0802032 (International Institute of Social and Economic Sciences, 2014).
Holt, C.A. in Handbook of Experimental Economics Vol. 1 (eds Kagel, J. & Roth, A. E.) 349–443 (Princeton Univ. Press, 1995).
Davis, D.D. & Holt, C.A. Experimental Economics (Princeton Univ. Press, 1993).
Kagel, J. & Roth, A.E. (eds) Handbook of Experimental Economics Vol. 1 (Princeton Univ. Press, 1995).
Kagel, J., Roth, A.E. (eds) Handbook of Experimental Economics Vol. 2 (Princeton Univ. Press, 2015).
Plott, C.R. & Smith, V.L. (eds) Handbook of Experimental Economics Results Vol. 1 (North-Holland, 2008).
Kagel, J.H. in Handbook of Experimental Economics Vol. 1 (eds Kagel, J. & Roth, A. E.) 501–585 (Princeton Univ. Press, 1995).
Cason, T.N. & Friedman, D. in The Double Auction Market: Institutions, Theories, and Evidence (eds Friedman, D. & Rust, J.) 253–283 (Addison-Wesley, 1993).
Smith, V. L. Economics in the laboratory. J. Econ. Perspect. 8, 113–131 (1994).
Gjerstad, S. The competitive market paradox. J. Econ. Dynam. Control 31, 1753–1780 (2007).
Cason, T. N. & Friedman, D. Price formation in double auction markets. J. Econ. Dynam. Control 20, 1307–1337 (1996).
Noussair, C. N., Plott, C. R. & Riezman, R. G. An experimental investigation of the patterns of international trade. Am. Econ. Rev. 85, 462–491 (1995).
Ketcham, J., Smith, V. L. & Williams, A. W. A comparison of posted-offer and double-auction pricing institutions. Rev. Econ. Stud. 51, 595–614 (1984).
Easley, D. & Ledyard, J.O. in The Double Auction Market: Institutions, Theories, and Evidence (eds Friedman, D. & Rust, J.) 63–97 (Addison-Wesley, 1993).
Wilson, R.B. in Arrow and the Ascent of Modern Economic Theory (ed. Feiwel, G. R.) 375–414 (Springer, 1987).
Friedman, D. A simple testable model of double auction markets. J. Econ. Behav. Organ. 15, 47–70 (1991).
Gode, D. K. & Sunder, S. Allocative efficiency of markets with zero-intelligence traders: market as a partial substitute for individual rationality. J. Polit. Econ. 101, 119–137 (1993).
Cliff, D. & Bruten, J. Less than human: simple adaptive trading agents for CDA markets. IFAC Proc. Vol. 31, 117–122 (1998).
Camerer, C.F. Behavioral Game Theory: Experiments in Strategic Interaction (Princeton Univ. Press, 2003).
Oosterbeek, H., Sloof, R. & van de Kuilen, G. Cultural differences in ultimatum game experiments: evidence from a meta-analysis. Exp. Econ. 7, 171–188 (2004).
Cooper, D. J. & Dutcher, E. G. The dynamics of responder behavior in ultimatum games: a meta-study. Exp. Econ. 14, 519–546 (2011).
Fehr, E. & Schmidt, K. M. A theory of fairness, competition, and cooperation. Q. J. Econ. 114, 817–868 (1999).
Bolton, G. E. & Ockenfels, A. Erc: a theory of equity, reciprocity, and competition. Am. Econ. Rev. 90, 166–193 (2000).
Blount, S. When social outcomes aren’t fair: the effect of causal attributions on preferences. Organ. Behav. Hum. Decis. Process. 63, 131–144 (1995).
Rabin, M. Incorporating fairness into game theory and economics. Am. Econ. Rev. 83, 1281–1302 (1993).
Falk, A. & Fischbacher, U. A theory of reciprocity. Games Econ. Behav. 54, 293–315 (2006).
Forsythe, R., Horowitz, J. L., Savin, N. E. & Sefton, M. Fairness in simple bargaining experiments. Games Econ. Behav. 6, 347–369 (1994).
Levine, D. K. Modeling altruism and spitefulness in experiments. Rev. Econ. Dynam. 1, 593–622 (1998).
Backus, M., Blake, T., Larsen, B. & Tadelis, S. Sequential bargaining in the field: evidence from millions of online bargaining interactions.Q. J. Econ. 135, 1319–1361 (2020).
Chabris, C. F., Morris, C. L., Taubinsky, D., Laibson, D. & Schuldt, J. P. The allocation of time in decision-making. J. Eur. Econ. Assoc. 7, 628–637 (2009).
Konovalov, A. & Krajbich, I. Revealed strength of preference: inference from response times. Judgm. Decis. Mak. 14, 381–394 (2019).
Krajbich, I., Oud, B. & Fehr, E. Benefits of neuroeconomic modeling: new policy interventions and predictors of preference. Am. Econ. Rev. 104, 501–506 (2014).
Andreoni, J. & Bernheim, B. D. Social image and the 50–50 norm: a theoretical and experimental analysis of audience effects. Econometrica 77, 1607–1636 (2009).
Bernheim, B. D. & Severinov, S. Bequests as signals: an explanation for the equal division puzzle. J. Polit. Econ. 111, 733–764 (2003).
Bauman, Y. & Rose, E. Selection or indoctrination: why do economics students donate less than the rest? J. Econ. Behav. Organ. 79, 318–327 (2011).
DerSimonian, R. & Laird, N. Meta-analysis in clinical trials. Control. Clin. Trials 7, 177–188 (1986).
Higgins, J. P. T., Thompson, S. G., Deeks, J. J. & Altman, D. G. Measuring inconsistency in meta-analyses. BMJ 327, 557 (2003).
Higgins, J. P. T. & Thompson, S. G. Quantifying heterogeneity in a meta-analysis. Stat. Med. 21, 1539–1558 (2002).
Roth, A. E., Prasnikar, V., Okuno-Fujiwara, M. & Zamir, S. Bargaining and market behavior in Jerusalem, Ljubljana, Pittsburgh, and Tokyo: an experimental study. Am. Econ. Rev. 81, 1068–1095 (1991).
Henrich, J., Heine, S. J. & Norenzayan, A. The weirdest people in the world? Behav. Brain Sci. 33, 61–83 (2010).
Ebersole, C. R. et al. Many labs 3: evaluating participant pool quality across the academic semester via replication. J. Exp. Soc. Psychol. 67, 68–82 (2016).
Klein, R. A. et al. Investigating variation in replicability: a many labs replication project. Soc. Psychol. 45, 142–152 (2014).
Klein, R. A. et al. Many labs 2: investigating variation in replicability across sample and setting. Adv. Methods Pract. Psychol. Sci. 1, 443–490 (2018).
Henrich, J. Does culture matter in economic behavior? Ultimatum game bargaining among the Machiguenga of the Peruvian Amazon. Am. Econ. Rev. 90, 973–979 (2000).
Henrich, J. et al. Markets, religion, community size, and the evolution of fairness and punishment. Science 327, 1480–1484 (2010).
Henrich, J. et al. Economic man in cross-cultural perspective: behavioral experiments in 15 small-scale societies. Behav. Brain Sci. 28, 795–815 (2005).
Camerer, C. F. & Thaler, R. H. Anomalies: ultimatums, dictators and manners. J. Econ. Perspect. 9, 209–219 (1995).
Smith, V. L. Experimental economics: induced value theory. Am. Econ. Rev. 66, 274–279 (1976).
T.I. was supported by Deutsche Forschungsgemeinschaft through CRC TRR 190. J.T.W. was supported by CRETA, National Taiwan University (NTU-107L900203 and MOST 107-3017-F-002-004) and the Ministry of Science and Technology of Taiwan (MOST 106-2628-H-002-001-MY4). C.F.C was supported by the Behavioral and Neuroeconomics Discovery Fund at Caltech (through the MacArthur Foundation Fellowship). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript. We thank T. Cason, J. Duffy, D. Friedman, M. Hsu, M. O. Jackson, J. O. Ledyard, C. R. Plott and audiences at Bay Area Behavioral and Experimental Economics Workshop, the ESA North American Conference, Barcelona GSE Summer Forum (External Validity, Generalizability and Replicability of Economic Experiments), Chapman University and Caltech (Applied Microeconomics Lunch) for helpful feedback. We also thank the MobLab engineering team for technical assistance.
The data used were shared from MobLab, a for-profit startup educational platform. The data availability is an in-kind contribution to all authors (although note that the data are available for purposes of analysis reproduction and extended analyses). P.L. was employed by MobLab from December 2017 to December 2018 and compensated more than US$10,000 during the last 3 years. A.L.B. was a visiting senior economist at MobLab from 22 January 2018 to 20 July 2018 and compensated more than US$10,000. T.I. has no competing interests. J.T.W. was a visiting senior economist at MobLab from 25 June 2018 to 7 September 2018 and is the John O. Ledyard Endowed Chair Professor of Experimental Economics at National Taiwan University, funded by contributions of more than US$10,000 to the university endowment fund from MobLab. S.W.W. holds an ownership stake in MobLab as a cofounder. She was compensated more than US$10,000 in 2018–2019. C.F.C. is a Scientific Advisor to MobLab, a position with no compensation but with a small ownership stake instead.
Peer review information Primary Handling Editor: Aisha Bradshaw.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Lin, PH., Brown, A.L., Imai, T. et al. Evidence of general economic principles of bargaining and trade from 2,000 classroom experiments. Nat Hum Behav 4, 917–927 (2020). https://doi.org/10.1038/s41562-020-0916-8
Review of Industrial Organization (2021)