Was Fable fabulously good? Holding the numbers up to the light

"Fable" and "fabulous" share more than just their sound. The question presents itself: was Claude Fable 5 really as good as the launch numbers promised, or did it stay a good story we never got to verify? The model was publicly available for exactly three days before the US government had it shut down. That makes this question harder to answer than you'd want.

The numbers were spectacular

On paper, Fable 5 was impressive. Anthropic claimed state of the art on nearly every benchmark tested. The most striking: 80.3 percent on SWE-bench Pro, against 69.2 percent for Opus 4.8, a lead of more than eleven points on agentic coding. Alongside that, 95.0 percent on SWE-bench Verified and first place on both GDPval-AA (1932 Elo) and Cognition's FrontierCode.

And then the practical example that made the rounds: Stripe used Fable 5 to migrate a 50-million-line codebase in a single day, work a team of engineers was estimated to need two months for. Impressive, if it holds up.

But who checked those numbers?

Here's the rub. Nearly all benchmark and capability claims come from Anthropic itself or from customer testimonials. Independent verification was limited by the gated, and later suspended, nature of the model. And then there's the lifespan: three days is simply too short for serious, independent benchmarking. So we mostly have the maker's word, and that of a handful of partners with a stake in the outcome.

That doesn't make the numbers false. It makes them unconfirmed. That's an important difference that drops out of most of the cheering summaries.

The lesson of Mythos Preview

Luckily, we have a precedent. The sister model Mythos Preview got equally dramatic claims back in April: it would autonomously find vulnerabilities, including a 17-year-old bug in FreeBSD. But when independent researchers looked, the nuance arrived. AISLE replicated several findings with smaller open-weight models. Confirmed CVE counts stayed in the dozens, not the thousands. And the UK AI Security Institute warned that the test environments contained no modern enterprise defenses or active defenders.

So the capability was real, but the most spectacular framing got a ceiling. The same healthy skepticism is warranted with Fable 5: impressive, yes, but probably a touch less magical than the launch slides suggest.

Was it worth the price?

Fable 5 cost $10 per million input tokens and $50 per million output tokens, double Opus 4.8 ($5/$25). The eleven-point lead on SWE-bench Pro is real and meaningful if you work on long, complex, multi-day agentic tasks. For that kind of work the higher price could pay for itself. But for shorter, everyday tasks, Opus 4.8 remained the better price-performance ratio. Fabulously good? For the right work, yes. For everything, no.

The bitter paradox

And then the moral of the story. The very thing that made Fable 5 so good, autonomously reading a codebase and finding vulnerabilities, is exactly what got it shut down. The more capable the model on this terrain, the greater the attention from outside. The fable ended with a warning: the power and the vulnerability of such a model are two sides of the same coin.

The verdict

Was Fable 5 fabulously good? On paper: yes, a real step forward. The jump on SWE-bench Pro is not marketing noise, and the direction is clear. But "fabulous" deserves an asterisk: short-lived, largely self-reported, and never exposed to the independent scrutiny that would make a verdict definitive. We saw enough to be impressed, and too little to be sure. And that, fittingly, is exactly what a fable is: a good story whose truth you have to judge on your own terms.