An Empirical Study of Harness Design for Coding Agents

Researchers conducted an empirical study evaluating how different harness components—planning, action space, and context management—affect the performance of autonomous coding agents across multiple models on benchmark tasks. Key findings include that context management becomes more critical with limited context budgets, planning primarily reduces costs for stronger models rather than improving accuracy, and bash-capable models can operate efficiently without predefined tools, especially on command-line tasks. The study provides a modular framework for designing and evaluating coding agent harnesses based on model capabilities and resource constraints.