Latent Agents: A Post-Training Procedure for Internalized Multi-Agent Debate

Researchers developed a method to distill multi-agent debate—a technique that improves LLM reasoning—into a single model through fine-tuning, achieving comparable performance while using 93% fewer tokens. Analysis via activation steering revealed that internalized debate creates agent-specific subspaces in the model's activations, with practical applications including improved localization and control of harmful behaviors. The approach offers a more efficient alternative to computationally expensive multi-agent debate while providing insights into how reasoning capabilities can be mechanistically understood in distilled models.