Bot can beat humans in multiplayer hidden-role games

Many gaming bots have been built to keep up with human players. Earlier this year, a team from Carnegie Mellon University developed the world's first bot that can beat professionals in multiplayer poker. DeepMind's AlphaGo made headlines in 2016 for besting a professional Go player. Several bots have also been built to beat professional chess players or join forces in cooperative games such as online capture the flag. In these games, however, the bot knows its opponents and teammates from the start.

At the Conference on Neural Information Processing Systems next month, the researchers will present DeepRole, the first gaming bot that can win online multiplayer games in which the participants' team allegiances are initially unclear. The bot is designed with novel "deductive reasoning" added into an AI algorithm commonly used for playing poker. This helps it reason about partially observable actions, to determine the probability that a given player is a teammate or opponent. In doing so, it quickly learns whom to ally with and which actions to take to ensure its team's victory.

The researchers pitted DeepRole against human players in more than 4,000 rounds of the online game "The Resistance: Avalon." In this game, players try to deduce their peers' secret roles as the game progresses, while simultaneously hiding their own roles. As both a teammate and an opponent, DeepRole consistently outperformed human players.

"If you replace a human teammate with a bot, you can expect a higher win rate for your team. Bots are better partners," says first author Jack Serrino '18, who majored in electrical engineering and computer science at MIT and is an avid online "Avalon" player.

The work is part of a broader project to better model how humans make socially informed decisions. Doing so could help build robots that better understand, learn from, and work with humans.

"Humans learn from and cooperate with others, and that enables us to achieve together things that none of us can achieve alone," says co-author Max Kleiman-Weiner, a postdoc in the Center for Brains, Minds and Machines and the Department of Brain and Cognitive Sciences at MIT, and at Harvard University. "Games like "Avalon' better mimic the dynamic social settings humans experience in everyday life. You have to figure out who's on your team and will work with you, whether it's your first day of kindergarten or another day in your office."

Joining Serrino and Kleiman-Weiner on the paper are David C. Parkes of Harvard and Joshua B. Tenenbaum, a professor of computational cognitive science and a member of MIT's Computer Science and Artificial Intelligence Laboratory and the Center for Brains, Minds and Machines.

Deductive bot

In "Avalon," three players are randomly and secretly assigned to a "resistance" team and two players to a "spy" team. Both spy players know all players' roles. During each round, one player proposes a subset of two or three players to execute a mission. All players simultaneously and publicly vote to approve or disapprove the subset. If a majority approve, the subset secretly determines whether the mission will succeed or fail. If two "succeeds" are chosen, the mission succeeds; if one "fail" is selected, the mission fails. Resistance players must always choose to succeed, but spy players may choose either outcome. The resistance team wins after three successful missions; the spy team wins after three failed missions.

Winning the game basically comes down to deducing who is resistance or spy, and voting for your collaborators. But that's actually more computationally complex than playing chess and poker. "It's a game of imperfect information," Kleiman-Weiner says. "You're not even sure who you're against when you start, so there's an additional discovery phase of finding whom to cooperate with."

DeepRole uses a game-planning algorithm called "counterfactual regret minimization" (CFR)-which learns to play a game by repeatedly playing against itself-augmented with deductive reasoning. At each point in a game, CFR looks ahead to create a decision "game tree" of lines and nodes describing the potential future actions of each player. Game trees represent all possible actions (lines) each player can take at each future decision point. In playing out potentially billions of game simulations, CFR notes which actions had increased or decreased its chances of winning, and iteratively revises its strategy to include more good decisions. Eventually, it plans an optimal strategy that, at worst, ties against any opponent.

CFR works well for games like poker, with public actions-such as betting money and folding a hand-but it struggles when actions are secret. The researchers' CFR combines public actions and consequences of private actions to determine if players are resistance or spy.

The bot is trained by playing against itself as both resistance and spy. When playing an online game, it uses its game tree to estimate what each player is going to do. The game tree represents a strategy that gives each player the highest likelihood to win as an assigned role. The tree's nodes contain "counterfactual values," which are basically estimates for a payoff that player receives if they play that given strategy.

At each mission, the bot looks at how each person played in comparison to the game tree. If, throughout the game, a player makes enough decisions that are inconsistent with the bot's expectations, then the player is probably playing as the other role. Eventually, the bot assigns a high probability for each player's role. These probabilities are used to update the bot's strategy to increase its chances of victory.

Simultaneously, it uses this same technique to estimate how a third-person observer might interpret its own actions. This helps it estimate how other players may react, helping it make more intelligent decisions. "If it's on a two-player mission that fails, the other players know one player is a spy. The bot probably won't propose the same team on future missions, since it knows the other players think it's bad," Serrino says.

Language: The next frontier

Interestingly, the bot did not need to communicate with other players, which is usually a key component of the game. "Avalon" enables players to chat on a text module during the game. "But it turns out our bot was able to work well with a team of other humans while only observing player actions," Kleiman-Weiner says. "This is interesting, because one might think games like this require complicated communication strategies."

Next, the researchers may enable the bot to communicate during games with simple text, such as saying a player is good or bad. That would involve assigning text to the correlated probability that a player is resistance or spy, which the bot already uses to make its decisions. Beyond that, a future bot might be equipped with more complex communication capabilities, enabling it to play language-heavy social-deduction games-such as a popular game "Werewolf" -which involve several minutes of arguing and persuading other players about who's on the good and bad teams.

Website search

Recently updated

Avira Total Security Suite Avira Total Security Suite Enjoy full antivirus security, privacy protection and system optimization thanks to this all-in-one suite that helps you safeguard your computer
RegRun Reanimator RegRun Reanimator Removing Trojans/Adware/Spyware and rootkits as well as other types of malware with the help of this comprehensive and useful application
IObit Malware Fighter IObit Malware Fighter Quickly remove existing ransomware infections as well as new infection attempts. Updated to work for all known variants of WannaCry

Software News

Dec 6
After more than two decades in operation, Craigslist is getting its own mobile apps.
Dec 6
With version 55 of its Android browser, Oslo, Norway-based Opera has a redesigned night mode that "reduces the light pollution caused by your phone to the bare minimum."
Dec 5
Tech watchers were discussing Mozilla this week over its new moves, new plans, new Firefox features.
Dec 4
Amazon is wasting no time in pursuing the opportunity to apply their voice technology expertise to the medical sector, which indeed has a need for voice to text transcriptions. With Amazon's service, the doctor can dictate ...
Dec 3
Every field in science and engineering employs highly specialized equipment: surface area analyzers, nanosizers, recording membrane osmometers. This equipment is often incredibly specific, designed for just a single function, ...
Nov 27
PlayStation VR is still going strong going in its fourth year. The peripheral has seen a fair amount of great games since then developers get a grasp of the maturing medium and create higher-quality experiences.

About us

Welcome to new crack resource CrackDownloadz.com! Our service can generate cracks, keygens and serials for your software to unlock it. CrackDownloadz provides a lot of popular cracks and keygens. No spyware and adware at all, just download new cracks, keygens and serials. If you have a software that needs a crack feel free to contact us.

Also you may contact us if you have software that needs to be removed from our website.