| |
Show HN: FrontierHarness Eval – 9 harness, same model, cost per pass varies 17x
Nine different code execution harnesses were evaluated on the same model, revealing a 17x variation in cost per task despite identical functionality, with costs ranging from $1.05 to $18.34 per pass. Codex achieved the highest pass rate at 66.7% for $3.47, while Exo Harness offered the lowest cost at $1.05 with a 53.3% pass rate, demonstrating significant trade-offs between performance and efficiency across implementations.
Read Full Article →
← More Tech news