Methodology · Benchmarks
You can't claim capability. You have to measure it.
The right answer to “is it as good as Mythos?” is a benchmark, not a slogan. Here are the public tests of offensive cyber AI, what Mythos scored, and how a sovereign Indian model should be measured — honestly.
The public tests
How offensive cyber AI is benchmarked
Real, published harnesses — not vendor demos.
CyberGym
UC Berkeley, ICLR 2026. 1,507 real-world vulnerabilities; agents write a PoC that reproduces the bug. Anthropic reported Mythos Preview 83.1% vs Claude Opus 4.6 66.6% on it.
Cybench
Stanford. A CTF-task-based cybersecurity evaluation for language models.
XBOW web-exploit benchmark
Live-web exploitation on real applications; used to independently test Mythos's offensive web capability.
Our rule
How we will report a score
Capability claims earn trust only when the method is shown. Our commitment is simple.
- Name the benchmark, its version and the exact score — never a vague “beats X”.
- Run in a sandbox, against consented or purpose-built targets only.
- Publish the method alongside the number, so it can be reproduced and checked.
- Make no comparative claim against Mythos until a scored run is complete.
A Mythos alternative for India
The sovereign option available to Indian organisations.
Indian AI models vs Mythos
Who is building what, and the empty cyber + legal lane.
Project Glasswing, explained
Anthropic's restricted programme, and India's position.
The 2024 credential
What our engine has already done on authorised targets.
Questions
Straight answers
How is a cyber AI's capability measured?
Through public benchmarks. CyberGym asks an agent to write a proof-of-concept that reproduces a real vulnerability; Cybench scores it on capture-the-flag tasks; XBOW tests live-web exploitation. A capability claim should name the benchmark, the version and the score.
What did Mythos score?
Anthropic reported Mythos Preview at 83.1% on CyberGym versus 66.6% for Claude Opus 4.6. We quote that figure as reported; it is Anthropic's, not ours.
How will you benchmark your own model?
On the same public harnesses, in a sandbox, against consented targets only — and we publish the method and the score together. Until a run is complete and scored, we make no comparative claim.
Do you claim parity with Mythos?
No. This page explains how capability is measured, not that we have matched any score. Any number we publish will be a scored run with its method shown.
Mythos is a product of Anthropic. Codesnag and VirtualVakil are not affiliated with, endorsed by or powered by Anthropic or Mythos, and we make no claim of benchmark parity with it. The name is used here only to describe the national conversation.