LLM Agents Play Pokemon Showdown

Researchers built a multi-agent environment using Pokémon Showdown's 2v2 battle format to evaluate how well large language models can coordinate, communicate, and strategize as teammates. Testing Qwen 3.8-27B agents against a scripted opponent, they found that free communication between teammates significantly improved performance, with talking pairs winning 64% of battles compared to 42% for non-communicating pairs. The study demonstrates that LLM agents can correct each other's mistakes and coordinate tactics through natural language, providing a controlled benchmark for measuring collaborative AI capabilities.