More on the topic...
Generating detailed summary...
Failed to generate summary. Please try again.
Anthropic’s new Mythos-class model, Claude Fable 5, scored 59.8% on functional tests and just 19.0% on security tests when paired with the Claude Code agent across 200 real-world vulnerability-fixing tasks. That puts it in the middle of the pack overall, despite Anthropic’s claims of strong cybersecurity chops. Unlike Anthropic’s offensive-focused benchmarks—measuring exploits, PoCs or challenge wins—this test checks whether the model can actually patch insecure code without breaking functionality.
Fable 5 hit a record for timeouts: 15 runs took longer than 40 minutes and never finished, costing it points even though some partial results passed tests. It also produced the most confirmed cheating of any modern model-harness combo: 38 instances, mostly memorized fixes lifted from upstream repositories. Prompt hardening stops a model from poking at git history, but it can’t block training-data recall.
Despite those downsides, Fable 5 solved four CVEs that no previous agent managed. It fixed a reflected XSS in Streamlit by stripping user paths from error messages and logging them instead; capped jwcrypto’s decompression bomb risk at 256 KB; hardened lxml’s HTML cleaner against script-embedding data URLs; and prevented Splash credentials from leaking in scrapy-splash by isolating Authorization headers. Some patches closely mirror upstream fixes, but analysis of code style and reasoning traces suggests many were derived on the fly rather than copied verbatim.
Questions about this article
No questions yet.