Why harnesses matter for energy (and token) efficiency

Prompt caching efficiency in AI inference varies significantly by client harness implementation, with cache hit rates ranging from 92-97% across users, meaning some clients require 2.7 times more computational work than others for identical conversations. Cache misses—caused when harnesses modify previously sent prompts—roughly double a request's energy consumption, and this inefficiency is largely preventable through better harness design that avoids unnecessary prompt edits.