Artificial intelligence (AI) has achieved countless progress in single-player and competitive two-player games such as Chess, Poker, and Go, in which artificial agents model players with fully aligned goals, or where two agents are engaged in a competition. However, more complex games require negotiation to achieve human-level performance, as the goals of the participants may only partially align, thus creating a trade-off between cooperation and competition. An example of such a game is the seven-player strategy board game Diplomacy, which involves complex communication, negotiation, and alliance formation. Previous studies have shown that AI agents are able to play Diplomacy, but only the simplified ‘No-Press’ version of it, in which communication and negotiation are not allowed, thus excluding the most exciting and challenging aspect of the game.
In a recent work, Yoram Bachrach and colleagues introduce a deep reinforcement learning approach that enables artificial agents to communicate and to build alliances and joint plans, using Diplomacy as a suitable analog to realistic negotiation. Agents are trained to imitate gameplay extended by a communication protocol for negotiating a joint plan of action in the form of binding contracts. To identify mutually beneficial deals, potential future states are sampled using policy and value functions trained via reinforcement learning, followed by simulated predictions of what could occur in the next turn. The authors demonstrated that their agents were able to win a game up to 2.5 times more often when compared to agents that could not communicate with each other. Interestingly, moving towards human-level performance, the authors tested broken commitment scenarios between agents, as well as developed conditions for honest cooperation via sanctioning mechanisms for agents that violate agreements, in order to reduce their advantages and force optimization of their behavior. This work paves the way for the development of new frameworks for the collective action of artificial agents in complex environments, their truthful communication and effective cooperation, and dynamic coalition formation. Ultimately, this offers important insights not only for game strategy, but also for a variety of systems based on AI control, such as personal assistants, automatic tools for marketing and sales, and algorithmic trading and bidding.
This is a preview of subscription content, access via your institution