Each of these had the chance to change future behavior: once FTW had figured out the location of the two teams' bases, most of its memory recalls focused on those areas of the map. The value of killing your opponents came even later. Only once those ideas were in place did it figure out the value of picking up the flag. The agent first developed the concept of its own base, and it later figured out that there was an opposition base. "The internal representation of was found to encode a wide variety of knowledge about the game situation," they write. The researchers could track as FTW picked up game information. The goal was to get it to "acquire policies that are robust to the variability of maps, number of players, and choice of teammates and opponents, a challenge that generalizes that of ad hoc teamwork." The amount of effort required for this system to learn is pretty staggering the researchers refer to going through 45,000 games as "early in training." Distinctive behaviors were still being put in place by 200,000 games in. With the architecture in place, FTW was set to play itself on randomly generated maps in teams with one or more teammates. For this, the researchers chose a standard neural network trained through reinforcement learning. So if the outer layer has determined that defending the flag is the best option at the moment, the inner layer will implement that strategy by checking the visual input for opponents while keeping close to the flag. Advertisementīeneath that, there's a distinct layer that sets a "policy" based on the outer layer's decisions. After each round of training, the worst-performing systems were killed off their replacements were generated by introducing "mutations" into the best performing ones. To improve performance of this outer optimization, the Deep Mind team took an evolutionary approach called population-based training. You can think of it as creating sub-goals throughout the course of the game, directed in a way that maximizes the chances of an overall win. At the outer level, the system was focused on the end point of winning the game, and it learned overall strategies that helped reach that goal. How do you bridge that gap?įor their system, which they call FTW, the Deep Mind researchers built a two-level learning system. There's an enormous gap between what might be useful at a given moment, and the end-of-game score that the systems have to judge their performance against.
#QUAKE III ARENA GAME HOW TO#
This complexity makes for a severe challenge for systems that are meant to teach themselves how to play. Different strategies-explore, defend your flag, capture theirs, shoot your opponents-all potentially provide advantages, and players can switch among them at any point in the game. Those simple rules lead to complex play because maps can be generated procedurally, and each player is reacting to what they can see in real time, limited by their field of view and the map's features. You can also gain tactical advantage by "tagging" (read "shooting") your opponents, which, after a delay, sends them back to their spawn. You score points by capturing the opponent's flag. In capture-the-flag mode, both sides start in a spawn area and have a flag to defend. Quake III Arena, to an extent, gets rid of the grid. Not a lot of rulesĬhess' complexity is built from an apparently simple set of rules: an 8x8 grid of squares and pieces that can only move in very specific ways. Such questions, however, seem to be answered by a report in today's issue of Science, where Deep Mind reveals the development of an AI system that has taught itself to play Quake III Arena and can consistently beat human opponents in capture-the-flag games. It wasn't unreasonable to question whether the same approach would work for completely different classes of games. Further Reading Move over AlphaGo: AlphaZero taught itself to play three different gamesīut at least for tabletop games like those, the potential moves are discrete and don't require real-time decision-making.
0 kommentar(er)
