Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Learning equilibria in symmetric auction games using artificial neural networks

Nature Machine Intelligence volume 3, pages 687–695 (2021)Cite this article

Subjects

Abstract

Auction theory is of central importance in the study of markets. Unfortunately, we do not know equilibrium bidding strategies for most auction games. For realistic markets with multiple items and value interdependencies, the Bayes Nash equilibria (BNEs) often turn out to be intractable systems of partial differential equations. Previous numerical techniques have relied either on calculating pointwise best responses in strategy space or iteratively solving restricted subgames. We present a learning method that represents strategies as neural networks and applies policy iteration on the basis of gradient dynamics in self-play to provably learn local equilibria. Our empirical results show that these approximated BNEs coincide with the global equilibria whenever available. The method follows the simultaneous gradient of the game and uses a smoothing technique to circumvent discontinuities in the ex post utility functions of auction games. Discontinuities arise at the bid value where an infinite small change would make the difference between winning and not winning. Convergence to local BNEs can be explained by the fact that bidders in most auction models are symmetric, which leads to potential games for which gradient dynamics converge.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1
Fig. 2: The empirical impact of risk-aversion on market efficiency.
Fig. 3: The effect of bidder correlation and risk attitudes on seller revenue.

Similar content being viewed by others

Data availability

All data analyses in this study are based exclusively on data generated by our custom simulation framework (see Code Availability). Raw simulation artefacts (all-iteration logs and trained models) will be made available by the corresponding author on request. Source data are provided with this paper.

Code availability

The source code of our simulation framework62, including instructions to reproduce all models and datasets referenced in this study, is freely available at https://github.com/heidekrueger/bnelearn, licensed under GNU-GPLv3.

References

  1. Brown, N. & Sandholm, T. Superhuman AI for multiplayer poker. Science 365, 885–890 (2019).

    Article  MathSciNet  Google Scholar 

  2. Daskalakis, C., Ilyas, A., Syrgkanis, V. & Zeng, H. Training gans with optimism. Preprint at https://arxiv.org/abs/1711.00141 (2017).

  3. Silver, D. et al. A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science 362, 1140–1144 (2018).

    Article  MathSciNet  Google Scholar 

  4. Daskalakis, C., Goldberg, P. & Papadimitriou, C. The complexity of computing a nash equilibrium. SIAM J. Comput. 39, 195–259 (2009).

    Article  MathSciNet  Google Scholar 

  5. Brown, G. W in Activity Analysis of Production and Allocation (ed. Koopmans, T. C.) 374–376 (Wiley, 1951).

    Google Scholar 

  6. Zinkevich, M. Online convex programming and generalized infinitesimal gradient ascent. In Proc. 20th International Conference on Machine Learning 928–936 (ICML, 2003).

  7. Bowling, M. Convergence and no-regret in multiagent learning. In Advances in Neural Information Processing Systems 209–216 (NIPS, 2005).

  8. Milgrom, P. R. & Weber, R. J. A theory of auctions and competitive bidding. Econometrica 50, 1089–1122 (1982).

  9. Klemperer, P. Auction theory: a guide to the literature. J. Econ. Surveys 13, 227–286 (1999).

    Article  Google Scholar 

  10. Vickrey, W. Counterspeculation, auctions, and competitive sealed tenders. J. Finance 16, 8–37 (1961).

    Article  MathSciNet  Google Scholar 

  11. Krishna, V. Auction Theory (Academic, 2009).

  12. Bergemann, D. & Morris, S. Robust implementation in direct mechanisms. Rev. Econ. Stud. 76, 1175–1204 (2009).

    Article  MathSciNet  Google Scholar 

  13. Campo, S., Perrigne, I. & Vuong, Q. Asymmetry in first-price auctions with affiliated private values. J. Appl. Econom. 18, 179–207 (2003).

    Article  Google Scholar 

  14. Janssen, M. C. Reflections on the 2020 Nobel Memorial Prize awarded to Paul Milgrom and Robert Wilson. Erasmus J. Philos. Econ. 13, 177–184 (2020).

    Google Scholar 

  15. Heinrich, J. & Silver, D. Deep reinforcement learning from self-play in imperfect-information games. Preprint at https://arxiv.org/abs/1603.01121 (2016).

  16. Lanctot, M. et al. A unified game-theoretic approach to multiagent reinforcement learning. In Proc. 31st International Conference on Neural Information Processing Systems (NIPS, 2017).

  17. Brown, N., Lerer, A., Gross, S. & Sandholm, T. Deep counterfactual regret minimization. In Proc. 36th International Conference on Machine Learning 793–802 (PMLR, 2019).

  18. Brown, N. & Sandholm, T. Superhuman AI for multiplayer poker. Science 365, 885–890 (2019).

    Article  MathSciNet  Google Scholar 

  19. Reeves, D. M. & Wellman, M. P. Computing equilibrium strategies in infinite games of incomplete information. Proc. 20th Conference on Uncertainty in Artificial Intelligence (UAI, 2004).

  20. Naroditskiy, V. & Greenwald, A. Using Iterated Best-response to Find Bayes-Nash Equilibria in Auctions 1894–1895 (AAAI, 2007).

  21. Rabinovich, Z., Naroditskiy, V., Gerding, E. H. & Jennings, N. R. Computing pure Bayesian-Nash equilibria in games with finite actions and continuous types. Artif. Intell. 195, 106–139 (2013).

    Article  MathSciNet  Google Scholar 

  22. Bosshard, V., Bünz, B., Lubin, B. & Seuken, S. Computing Bayes-Nash equilibria in combinatorial auctions with continuous value and action spaces. In Proc. 26th International Joint Conference on Artificial Intelligence 119–127 (IJCAI, 2017).

  23. Bosshard, V., Bünz, B., Lubin, B. & Seuken, S. Computing Bayes-Nash equilibria in combinatorial auctions with verification. J. Artif. Intell. Res. 69, 531–570 (2020).

    Article  MathSciNet  Google Scholar 

  24. Feng, Z., Guruganesh, G., Liaw, C., Mehta, A. & Sethi, A. Convergence analysis of no-regret bidding algorithms in repeated auctions. Preprint at https://arxiv.org/abs/2009.06136 (2020).

  25. Li, Z. & Wellman, M. P. Evolution strategies for approximate solution of Bayesian games. In Proc. AAAI Conference on Artificial Intelligence Vol. 35, 5531–5540 (AAAI, 2021).

  26. Cai, Y. & Papadimitriou, C. Simultaneous Bayesian auctions and computational complexity. In Proc. 15th ACM Conference on Economics and Computation 895–910 (ACM, 2014).

  27. Fudenberg, D. & Levine, D. K. Learning and equilibrium. Annu. Rev. Econ. 1, 385–420 (2009).

    Article  Google Scholar 

  28. Jafari, A., Greenwald, A., Gondek, D. & Ercal, G. On no-regret learning, fictitious play, and Nash equilibrium. Proc. 18th International Conference on Machine Learning 226–233 (ICML, 2001).

  29. Stoltz, G. & Lugosi, G. Learning correlated equilibria in games with compact sets of strategies. Games Econ. Behav. 59, 187–208 (2007).

    Article  MathSciNet  Google Scholar 

  30. Hartline, J., Syrgkanis, V. & Tardos, E. No-regret learning in Bayesian games. In Advances in Neural Information Processing Systems (eds Cortes, C. et al.) Vol. 28, 3061–3069 (NIPS, 2015); http://papers.nips.cc/paper/6016-no-regret-learning-in-bayesian-games.pdf

  31. Foster, D. J., Li, Z., Lykouris, T., Sridharan, K. & Tardos, E. Learning in games: robustness of fast convergence. In Advances in Neural Information Processing Systems 4734–4742 (NIPS, 2016).

  32. Viossat, Y. & Zapechelnyuk, A. No-regret dynamics and fictitious play. J. Econ. Theory 148, 825–842 (2013).

    Article  MathSciNet  Google Scholar 

  33. Mazumdar, E., Ratliff, L. J. & Sastry, S. S. On gradient-based learning in continuous games. SIMODS 2, 103–131 (2020).

    MathSciNet  Google Scholar 

  34. Dütting, P., Feng, Z., Narasimhan, H., Parkes, D. & Ravindranath, S. S. Optimal auctions through deep learning. In International Conference on Machine Learning 1706–1715 (PMLR, 2019).

  35. Feng, Z., Narasimhan, H. & Parkes, D. C. Deep learning for revenue-optimal auctions with budgets. In Proc. 17th International Conference on Autonomous Agents and Multiagent Systems 354–362 (AAMAS, 2018).

  36. Tacchetti, A., Strouse, D., Garnelo, M., Graepel, T. & Bachrach, Y. A neural architecture for designing truthful and efficient auctions. Preprint at https://arxiv.org/abs/1907.05181 (2019).

  37. Weissteiner, J. & Seuken, S. Deep learning-powered iterative combinatorial auctions. In Proc. AAAI Conference on Artificial Intelligence Vol. 34, 2284–2293 (AAAI, 2020).

  38. Morrill, D. et al. Hindsight and sequential rationality of correlated play. Preprint at https://arxiv.org/abs/2012.05874 (2020).

  39. Hartford, J. S. Deep Learning for Predicting Human Strategic Behavior. Ph.D. thesis, Univ. British Columbia (2016).

  40. Ghani, R. & Simmons, H. Predicting the end-price of online auctions. In Proc. International Workshop on Data Mining and Adaptive Modelling Methods for Economics and Management (CiteSeer, 2004).

  41. Zheng, S. et al. The AI economist: improving equality and productivity with AI-driven tax policies. Preprint at https://arxiv.org/abs/2004.13332 (2020).

  42. Goeree, J. K. & Lien, Y. On the impossibility of core-selecting auctions. Theoretical Econ. 11, 41–52 (2016).

    Article  MathSciNet  Google Scholar 

  43. Bichler, M. & Goeree, J. K. Handbook of Spectrum Auction Design (Cambridge Univ. Press, 2017).

  44. Debnath, L. et al. Introduction to Hilbert Spaces with Applications (Academic, 2005).

  45. Bichler, M., Guler, K. & Mayer, S. Split-award procurement auctions-can bayesian equilibrium strategies predict human bidding behavior in multi-object auctions? Prod. Oper. Manag. 24, 1012–1027 (2015).

    Article  Google Scholar 

  46. Ui, T. Bayesian nash equilibrium and variational inequalities. J. Math. Econ. 63, 139–146 (2016).

    Article  MathSciNet  Google Scholar 

  47. Hornik, K. Approximation capabilities of multilayer feedforward networks. Neural Networks 4, 251–257 (1991).

    Article  Google Scholar 

  48. Wierstra, D. et al. Natural evolution strategies. J. Mach. Learn. Res. 15, 949–980 (2014).

    MathSciNet  MATH  Google Scholar 

  49. Salimans, T., Ho, J., Chen, X., Sidor, S. & Sutskever, I. Evolution strategies as a scalable alternative to reinforcement learning. Preprint at https://arxiv.org/abs/1703.03864 (2017).

  50. Benaím, M., Hofbauer, J. & Sorin, S. Perturbations of set-valued dynamical systems, with applications to game theory. Dyn. Games Appl. 2, 195–205 (2012).

    Article  MathSciNet  Google Scholar 

  51. Letcher, A. et al. Differentiable game mechanics. J. Mach. Learn. Res. 20, 1–40 (2019).

    MathSciNet  Google Scholar 

  52. Monderer, D. & Shapley, L. S. Potential games. Games Econ. Behav. 14, 124–143 (1996).

    Article  MathSciNet  Google Scholar 

  53. Bünz, B., Lubin, B. & Seuken, S. Designing core-selecting payment rules: a computational search approach. In Proc. 2018 ACM Conference on Economics and Computation 109 (ACM, 2018).

  54. Ausubel, L. M. & Baranov, O. Core-selecting auctions with incomplete information. Int. J. Game Theory 49, 251–273 (2019).

  55. Guler, K., Bichler, M. & Petrakis, I. Ascending combinatorial auctions with risk averse bidders. Group Decis. Negot. 25, 609–639 (2016).

    Article  Google Scholar 

  56. Jehiel, P., Meyer-ter-Vehn, M., Moldovanu, B. & Zame, W. R. The limits of ex post implementation. Econometrica 74, 585–610 (2006).

    Article  MathSciNet  Google Scholar 

  57. Daskalakis, C., Skoulakis, S. & Zampetakis, M. The complexity of constrained min–max optimization. In Pro. 53rd Annual ACM SIGACT Symposium on Theory of Computing 1466–1478 (STOC, 2021).

  58. Vorobeychik, Y., Reeves, D. M. & Wellman, M. P. Constrained automated mechanism design for infinite games of incomplete information. In Proc. 23rd Conference on Uncertainty in Artificial Intelligence 400–407 (UAI, 2007).

  59. Viqueira, E. A., Cousins, C., Mohammad, Y. & Greenwald, A. Empirical mechanism design: designing mechanisms from data. In Proc. 35th Uncertainty in Artificial Intelligence Conference 1094–1104 (PMLR, 2020).

  60. Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. Preprint at https://arxiv.org/abs/1412.6980 (2015).

  61. Paszke, A. et al. Automatic differentiation in pytorch. In 31st Conference on Neural Information Processing Systems (NIPS, 2017).

  62. Heidekrüger, S., Kohring, N., Sutterer, S. & Bichler, M. bnelearn: A Framework for Equilibrium Learning in Sealed-Bid Auctions (Github, 2021); https://github.com/heidekrueger/bnelearn

Download references

Acknowledgements

We are grateful for funding by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation; grant no. BI 1057/1-9). We thank V. Bosshard, B. Lubin, P. Mertikopoulos, P. Milgrom, S. Seuken, T. Ui, F. Maldonado and participants of the NBER Market Design Workshop 2020 for valuable feedback on earlier versions.

Author information

Authors and Affiliations

  1. Department of Computer Science, Technical University of Munich, Garching, Germany

    Martin Bichler, Maximilian Fichtl, Stefan Heidekrüger, Nils Kohring & Paul Sutterer

Authors
  1. Martin Bichler
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maximilian Fichtl
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stefan Heidekrüger
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nils Kohring
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Paul Sutterer
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

M.B. conceived and supervised the project and contributed to the overall study design, theoretical analysis of NPGA and writing the manuscript. M.F. contributed to the theoretical analysis of NPGA. S.H. contributed to the design, implementation and optimization of the algorithm and simulation framework, the theoretical analysis, and to the writing of the manuscript. N.K. contributed to the optimization of the algorithm, the design, implementation and optimization of the simulation framework, the theoretical and empirical analysis, and the writing of the manuscript. P.S. contributed to design and implementation of the algorithm and simulation framework, and the empirical analysis.

Corresponding author

Correspondence to Martin Bichler.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information Nature Machine Intellligence thanks Pierre Baldi, Neil Newman and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Description and discussion of additional experiments (including Supplementary Figs. 1–4 and Tables 1–3); proof of auxiliary lemmata used in the proof of Proposition 1 of the main manuscript; mathematical derivation of conditional distributions required for evaluation in the absence of equilibria.

Supplementary Data Source Data Table 1

Last-iteration loss measurements of approximate equilibria found by NPGA for 10 runs each in LLG settings.

Source data

Source Data Fig. 2

Aggregate last-iteration efficiency measurements (mean and s.d. over 10 runs each) of learned NPGA policies in NVCG LLG for several risk parameters.

Source Data Fig. 3

Last-iteration seller revenue measurements of learned NPGA policies in NVCG LLG for several risk and correlation parameters.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bichler, M., Fichtl, M., Heidekrüger, S. et al. Learning equilibria in symmetric auction games using artificial neural networks. Nat Mach Intell 3, 687–695 (2021). https://doi.org/10.1038/s42256-021-00365-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s42256-021-00365-4

This article is cited by

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics