Will It Mythos?

The author created a benchmark suite to test whether AI models other than Mythos can identify security bugs as effectively as Mythos claims to. Using nine confirmed bugs that Mythos previously found, the benchmark tests whether other leading models like Claude Opus can detect and describe these vulnerabilities when given only the code repository and file to review. While the author acknowledges limitations in this initial testing—including a small sample size and single run per bug—the benchmark aims to determine whether Mythos possesses uniquely superior bug-finding capabilities or if other models are equally capable.