Migrating a production AI agent to GPT-5.6: 2.2x faster, 27% cheaper

Ploy migrated its AI agent from Claude Opus to OpenAI's new GPT-5.6 Sol, achieving 2.2x faster build times and 27% cost reduction while maintaining or improving quality on their high-stakes task of building marketing websites. The switch revealed that evaluation harnesses are often tuned to incumbent models, requiring fixes to tool schemas, caching, and reasoning replay to fairly compare different model providers. The key lesson is that differences between models often stem from provider-specific behaviors rather than raw capability, making careful trace analysis essential when evaluating alternatives.