Each game genre plays differently and comes with its unique challenges. Studying and possibly overcoming these challenges tells us more about the design of machine learning agents under special conditions. In this project, we focus on developing Strategy Game AI. In strategy games, we are often tasked to control multiple units at the same time. This yields a combinatorial action space which grows exponentially with the number of units under our control. Even modern AI solutions cannot handle this explosion of the search space without special modifications. We at the TNT, study how machine learning-based agents can overcome this challenge and learn to play strategy games proficiently.
The original search space of strategy games is often too large to handle. Therefore, methods of abstraction are required to reduce the complexity of said search space and allow agents to quickly identify good actions. Mechanisms of abstraction include generating approximate homomorphisms of the original game, shaping rewards, filtering/clustering actions or states, and time-based abstractions such as the option learning framework. All those can drastically improve the learning speed of an AI agent. While strategy games provide us with interesting benchmark tasks to test these methods, abstraction is not limited to said domain and similar techniques can often be transferred to other machine-learning tasks.
