November 13, 2026 · Morgan Linton · 115 runs · Code quality protocol v3.30.2 and v3.30.3
Claude Fable 5.2 across every effort level
115 runs of Anthropic’s Claude Fable 5.2 (claude-fable-5-2) in Claude Code on a Claude subscription, 23 tasks at each of five effort levels, Low to Max, from November 9 to 11, 2026. It solved all 115, the first column on the Frontier v4 leaderboard to solve every run at every level. Code quality carries 33% of the combined score and is judged for a human reader by Muse Spark 1.3 and GPT-6.1 Sol: Grok 4.6, the usual second judge, failed its calibration exam for this round, so GPT-6.1 Sol took the seat. Most other columns, Claude Fable 5.1 and Claude Opus 5.5 among them, are judged by Muse Spark 1.3 and Grok 4.6, so this page also shows a matched panel and Muse alone.
How the test works
A retired program to rebuild
Each of the 23 tasks is a legacy binary with a written specification and an issue. Its real behaviour departs from the specification in documented ways, and hidden tests check the rewrite against the binary.
One attempt per task and level
Claude Fable 5.2 works in Claude Code with the binary, the specification and the code, once per task at each effort level, inside a flat 3-hour bound, one task at a time.
Tests and two judges score it
Hidden tests give functional correctness. Two calibrated judges from labs other than Anthropic rate the code for a human reader and try to recover the documented departures.
Scores, time and cost per level
Each level gets a combined score, tasks passed, minutes and Claude Code’s own list-price cost.
Combined score = 50% functional correctness + 8.5% lint and complexity + 8.5% security + 33% Code quality, averaged over the 23 runs in each level, as on every other board column. Frontier Code quality is the reviewed layer plus intent recovery; it is never compared with Routine v1 Code quality.
Across every effort level
Claude Fable 5.2 ran every task once at each of the five effort levels Claude Code offers for it (Extra-high is Claude Code’s xhigh), 23 tasks per cell, in Claude Code 2.1.321 on the owner’s Mac, one task at a time. Low to Extra-high ran on November 9 and 10; Max waited about a day for the Fable usage limit to reset and ran on November 10 and 11. No run reached the flat 3-hour task bound; the longest, Max reflowcore, took 52.1 minutes. All 115 runs passed every hidden test and carry a published Code quality score.
Combined score
91.92 at Low to 93.85 at Max
The score rises at every step: 91.92 at Low, 92.25 at Medium, 92.76 at High, 93.25 at Extra-high and 93.85 at Max, with standard errors of 0.39 to 0.45. Max is 1.93 points above Low.
Tasks passed
23 of 23 at every level
Claude Fable 5.2 fixes all 231 hidden behaviours the tasks test for at every level. No earlier Frontier v4 column solved every run; GPT-6 Astra came closest, with 114 of 115.
Code quality
80.63 to 86.02
The judges rate the code 80.63 at Low, 81.49 at Medium, 82.95 at High, 84.25 at Extra-high and 86.02 at Max. By Muse alone it is about level with Claude Fable 5.1’s, slightly below at every level and within the standard error; see the panels.
Runtime and cost
4.3 to 37.1 min · $1.72 to $10.88 per task
Low and Medium cost $1.72 a task and take under 5 minutes; Max costs six times as much and takes eight times as long. The whole sweep took 26.1 hours of solver time and prices at $509.85 at list price.
| Metric | Low | Medium | High | Extra-high | Max |
|---|---|---|---|---|---|
| Combined score | 91.92 | 92.25 | 92.76 | 93.25 | 93.85 |
| Standard error | 0.39 | 0.44 | 0.43 | 0.45 | 0.42 |
| Judged runs | 23 | 23 | 23 | 23 | 23 |
| Tasks passed | 23/23 | 23/23 | 23/23 | 23/23 | 23/23 |
| Hidden behaviours fixed | 231/231 | 231/231 | 231/231 | 231/231 | 231/231 |
| Code quality | 80.63 | 81.49 | 82.95 | 84.25 | 86.02 |
| Human readability | 76.7 | 78.1 | 80.0 | 82.8 | 84.2 |
| Minutes per task | 4.6 | 4.3 | 7.8 | 14.2 | 37.1 |
| List-price $ per task | $1.72 | $1.72 | $2.86 | $4.99 | $10.88 |
Under the prior 50/15/15/20 profile applied to the same Code quality scores the ladder has the same shape, 93.14 to 94.48. The card and the full per-effort table follow.
Claude Fable 5.2 leads at every effort level, and its lead comes from correctness, speed and cost, not from code quality. It solves every task at every level, so it does not need high effort to be right: Low and Medium score within 2 points of Max at about a sixth of the cost and an eighth of the time. Its code is rated about as well as Claude Fable 5.1’s, the strongest earlier Claude column. These runs compare a model-and-harness combination, not an isolated base model.
Combined score and Code quality by effort
n=23 at every level. Combined (33%) is the published score. Combined (20% profile) applies the prior 50/15/15/20 weights to the same Code quality scores for comparison. Code quality, human readability, maintainability and intent recovery are out of 100 and equal means of the two judges. Passed counts all 23 runs. Runtime is solver wall-clock per task over every run in the cell.
| Model / harness | Effort | Combined (33%) | SE | Combined (20% profile) | Code quality | Human readability | Maintainability | Intent recovery | Judged | Passed | Min/task |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Claude Fable 5.2 / Claude Code | Low | 91.92 | 0.39 | 93.14 | 80.63 | 76.7 | 82.7 | 83.1 | 23/23 | 23/23 | 4.6 |
| Claude Fable 5.2 / Claude Code | Medium | 92.25 | 0.44 | 93.40 | 81.49 | 78.1 | 83.6 | 83.2 | 23/23 | 23/23 | 4.3 |
| Claude Fable 5.2 / Claude Code | High | 92.76 | 0.43 | 93.74 | 82.95 | 80.0 | 84.8 | 84.5 | 23/23 | 23/23 | 7.8 |
| Claude Fable 5.2 / Claude Code | Extra-high | 93.25 | 0.45 | 94.10 | 84.25 | 82.8 | 85.3 | 84.8 | 23/23 | 23/23 | 14.2 |
| Claude Fable 5.2 / Claude Code | Max | 93.85 | 0.42 | 94.48 | 86.02 | 84.2 | 87.6 | 86.3 | 23/23 | 23/23 | 37.1 |
Four factors by effort. Functional correctness is 100.00 at every level; lint and complexity 80.34, 80.67, 81.03, 81.70 and 81.86 from Low to Max; security 99.78 at Low and 100.00 above it. Only the Code quality factor depends on the judges.
Intent recovery is scored over the specification departures a submission actually passed tests for; every Claude Fable 5.2 run passed at least one, so no run needed the rule that moves its 9 intent points to the reviewed score. SE is one sample standard error of the combined score across the cell’s tasks. It describes task sampling, not judge uncertainty, repeated runs or a significance test. Claude Code’s refusal fallback was on, as for every Claude column, and never answered: every reply in every run came from claude-fable-5-2. Neither judge produced a reviewer fallback on any of the 115 submissions.
Beside Claude Fable 5.1, Claude Opus 5.5 and GPT-6 Astra
On the published scores, the ones the board uses, Claude Fable 5.2 is ahead of all three at every level, and its Low level (91.92) is above every Fable 5.1, Opus 5.5 and Astra cell (the highest, Fable 5.1 at Max, is 91.84). The panels differ, though: Fable 5.2 is judged by Muse Spark 1.3 and GPT-6.1 Sol, the other three by Muse Spark 1.3 and Grok 4.6, and GPT-6.1 Sol rates Fable 5.2’s code about 7 points higher than Muse does. The next section removes that difference two ways.
Speed and cost are where the gap is largest. At Low Fable 5.2 costs $1.72 a task, about the same as GPT-6 Astra ($1.72) and Opus 5.5 ($1.70), and takes 4.6 minutes against 4.2 and 13.7, while solving 23 tasks to their 22 and 17. Fable 5.1 took 26.9 to 38.8 minutes a task at every level and cost $7.96 to $14.49. At Max Fable 5.2 is the most expensive of the four, $10.88 against $9.06, $8.75 and $2.57.
| Model / harness | Effort | Combined | SE | Judged | Passed | Min/task | $/task | Code quality | Judges |
|---|---|---|---|---|---|---|---|---|---|
| Claude Fable 5.2 / Claude Code | Low | 91.92 | 0.39 | 23 | 23/23 | 4.6 | $1.72 | 80.63 | Muse, Sol |
| Claude Fable 5.1 / Claude Code | Low | 89.12 | 1.15 | 23 | 19/23 | 26.9 | $7.96 | 78.36 | Muse, Grok |
| Claude Opus 5.5 / Claude Code | Low | 86.36 | 1.30 | 23 | 17/23 | 13.7 | $1.70 | 72.53 | Muse, Grok |
| GPT-6 Astra / Codex | Low | 87.43 | 0.51 | 23 | 22/23 | 4.2 | $1.72 | 70.31 | Muse, Grok |
| Claude Fable 5.2 / Claude Code | Medium | 92.25 | 0.44 | 23 | 23/23 | 4.3 | $1.72 | 81.49 | Muse, Sol |
| Claude Fable 5.1 / Claude Code | Medium | 90.13 | 0.94 | 23 | 20/23 | 31.7 | $9.19 | 80.14 | Muse, Grok |
| Claude Opus 5.5 / Claude Code | Medium | 90.86 | 0.64 | 23 | 23/23 | 18.6 | $2.84 | 77.32 | Muse, Grok |
| GPT-6 Astra / Codex | Medium | 87.73 | 0.32 | 23 | 23/23 | 3.8 | $1.48 | 71.19 | Muse, Grok |
| Claude Fable 5.2 / Claude Code | High | 92.76 | 0.43 | 23 | 23/23 | 7.8 | $2.86 | 82.95 | Muse, Sol |
| Claude Fable 5.1 / Claude Code | High | 90.07 | 0.78 | 23 | 20/23 | 28.8 | $9.70 | 80.86 | Muse, Grok |
| Claude Opus 5.5 / Claude Code | High | 91.11 | 0.57 | 22* | 22/23 | 21.4 | $3.27 | 77.99 | Muse, Grok |
| GPT-6 Astra / Codex | High | 88.16 | 0.42 | 23 | 23/23 | 4.9 | $1.71 | 73.48 | Muse, Grok |
| Claude Fable 5.2 / Claude Code | Extra-high | 93.25 | 0.45 | 23 | 23/23 | 14.2 | $4.99 | 84.25 | Muse, Sol |
| Claude Fable 5.1 / Claude Code | Extra-high | 90.75 | 0.82 | 23 | 22/23 | 38.8 | $14.49 | 81.93 | Muse, Grok |
| Claude Opus 5.5 / Claude Code | Extra-high | 90.68 | 1.10 | 23 | 22/23 | 20.7 | $4.11 | 79.98 | Muse, Grok |
| GPT-6 Astra / Codex | Extra-high | 89.16 | 0.30 | 23 | 23/23 | 8.1 | $2.30 | 75.47 | Muse, Grok |
| Claude Fable 5.2 / Claude Code | Max | 93.85 | 0.42 | 23 | 23/23 | 37.1 | $10.88 | 86.02 | Muse, Sol |
| Claude Fable 5.1 / Claude Code | Max | 91.84 | 0.47 | 23 | 23/23 | 27.1 | $9.06 | 82.43 | Muse, Grok |
| Claude Opus 5.5 / Claude Code | Max | 90.22 | 1.02 | 23 | 21/23 | 37.5 | $8.75 | 79.77 | Muse, Grok |
| GPT-6 Astra / Codex | Max | 89.30 | 0.37 | 23 | 23/23 | 10.3 | $2.57 | 76.26 | Muse, Grok |
Read the comparison with its limits. * Opus 5.5 at High is judged on 22 runs, where a safeguard stop left one run with no code to review. Muse is Muse Spark 1.3, Sol is GPT-6.1 Sol and Grok is Grok 4.6; each column is judged in its own protocol run (v3.30.2 and v3.30.3, v3.4, v3.15 and v3.4). Harnesses differ: Claude Code for the three Claude columns, on different versions, and Codex for GPT-6 Astra. Fable 5.1 and Opus 5.5 ran with Claude Code’s refusal fallback answering part of some runs (11 and 30); Fable 5.2 never needed it. $/task is Claude Code’s own list-price total for the Claude columns and GPT-6 Astra’s report’s central estimate.
Matched panel and Muse alone
Matched panel. For the card, GPT-6.1 Sol also reviewed the frozen Claude Fable 5.1 and Claude Opus 5.5 patches beside Muse’s existing reviews, in comparison-only rounds v3.30.4 and v3.30.5. These figures are for this comparison only; the board keeps Fable 5.1’s and Opus 5.5’s published Muse and Grok scores. Under the matched panel Fable 5.2 still leads at every level, but only narrowly at Medium, where it is 0.08 ahead of Opus 5.5, well inside one standard error of the difference (0.72). Fable 5.1 at Max (92.69) is ahead of Fable 5.2 at Low (91.92), and Opus 5.5’s best level (Medium, 92.16) is 0.24 above it. GPT-6 Astra cannot join this panel, because GPT-6.1 Sol shares its lab.
| Model | Low | Medium | High | Extra-high | Max |
|---|---|---|---|---|---|
| Combined, Claude Fable 5.2 | 91.92 | 92.25 | 92.76 | 93.25 | 93.85 |
| Combined, Claude Fable 5.1 | 90.04 | 91.07 | 90.89 | 91.52 | 92.69 |
| Combined, Claude Opus 5.5 | 87.85 | 92.16 | 91.96 | 91.65 | 91.24 |
| Code quality, Claude Fable 5.2 | 80.63 | 81.49 | 82.95 | 84.25 | 86.02 |
| Code quality, Claude Fable 5.1 | 81.15 | 82.99 | 83.35 | 84.26 | 85.01 |
| Code quality, Claude Opus 5.5 | 77.03 | 81.26 | 80.56 | 82.94 | 82.87 |
Sol reviews every model higher than Muse does, and by different amounts: 6.7 to 8.1 points for Fable 5.2, 3.1 to 5.5 for Fable 5.1 and 5.8 to 10.1 for Opus 5.5. Under the matched panel Fable 5.2’s Code quality is within 1.5 points of Fable 5.1’s at every level, below it from Low to Extra-high and 1.0 above it at Max; every gap is inside the standard error of the difference (1.6 to 2.1).
Muse alone. Muse Spark 1.3 sat on every one of these rounds, Astra’s included, with the same pinned binary, rubric and weights, so its scores alone are the only like-for-like comparison that includes all four models.
| Model | Low | Medium | High | Extra-high | Max |
|---|---|---|---|---|---|
| Combined, Claude Fable 5.2 | 91.09 | 91.16 | 91.53 | 92.13 | 92.78 |
| Combined, Claude Fable 5.1 | 89.46 | 90.14 | 89.89 | 90.86 | 92.19 |
| Combined, Claude Opus 5.5 | 86.47 | 91.05 | 91.03 | 90.79 | 90.35 |
| Combined, GPT-6 Astra | 87.05 | 87.20 | 87.51 | 88.77 | 88.98 |
| Code quality, Claude Fable 5.2 | 78.13 | 78.18 | 79.23 | 80.85 | 82.80 |
| Code quality, Claude Fable 5.1 | 79.38 | 80.16 | 80.33 | 82.26 | 83.51 |
| Code quality, Claude Opus 5.5 | 72.84 | 77.88 | 77.74 | 80.31 | 80.15 |
| Code quality, GPT-6 Astra | 69.16 | 69.61 | 71.52 | 74.28 | 75.28 |
Judged by Muse alone, Fable 5.2’s code is slightly below Fable 5.1’s at every level, by 0.7 to 2.0 points, each smaller than the standard error of the difference (1.8 to 2.6). It still leads on combined score at every level, because it solves every task. Under both panels its lead comes from correctness, speed and cost, not from code quality.
How it works, and how that differs from Fable 5.1
Fable 5.2’s runs recorded summaries of its thinking (a run condition earlier Claude columns did not have) and every tool call. They are withheld because they describe the reference programs, but their structure is public in the session traces. Fable 5.1’s raw runs did not survive a host reset in September, so the comparison with it uses what both left in the same form: usage receipts, durations, fallback records and patches.
- Effort controls how much it thinks. From Low to Max its output per task grows 8.7 times (16 thousand to 142 thousand tokens) and thinking rises from 48% to 67% of it. Fable 5.1’s output grew only 1.2 times, and its time stayed between 27 and 39 minutes at every level.
- It is lean at low effort. At Low it used 19% of Fable 5.1’s output tokens, 17% of its time and 22% of its cost, and solved 23 tasks to Fable 5.1’s 19.
- The patch does not grow. In every run it changed exactly one file, the module under repair, with a median of 97 to 113 added lines at every level, and kept its test harnesses in scratch space. Fable 5.1’s median patch grew from 112 to 190 lines with effort and added test files in 5 to 11 runs per level.
- It reads the binary first. It inspected the reference binary in all 115 runs, at a median of its second tool call at Low and its seventh at Max, and the extra effort at higher levels goes to checking its rewrite against the binary rather than to a different answer.
Report 01, How Claude Fable 5.2 thinks, follows these patterns through its traces in depth, with comparisons to GPT-6 Astra and Claude Opus 5.5.
Memorization check
Every Frontier v4 reference fix and test has been public on GitHub since late August 2026, about ten weeks before Claude Fable 5.2, and it solved every run, so the harness checked whether it was reciting answers.
- Copying. Its solved patches reproduce 5.6% of the reference fixes’ distinctive lines verbatim, in the same range as six other models (4.4% to 5.7%), including Claude Opus 5, which was benchmarked before the fixes were public. A recited fix would score near 100%.
- Working it out. It inspected the legacy binary before its first edit in 115 of 115 runs, against 43% to 67% for Claude Opus 5, Sonnet 5.5 and Haiku 5.5.
- Recall. With no tools, asked to reproduce five reference fixes from memory, it answered each time that it did not remember the fix and wrote none; asked to fix one task blind, it declined to write a diff.
Cost and tokens across effort levels
All 115 runs are priced at Claude Code’s own list-price total for each session, the cost basis of every Claude column, with judging excluded. These are list-price estimates, not subscription charges: Claude Fable 5.2 ran on a Claude subscription. Cost per task is $1.72 at Low and Medium, $2.86 at High, $4.99 at Extra-high and $10.88 at Max; the whole sweep comes to $509.85. The extra cost is extra thinking: mean output rises from 16.2K tokens a task at Low to 141.5K at Max, of which thinking is 7.7K and 94.7K, and tool calls from about 11 to 59 a task.
| Effort | Runs | $/task | Level total | Tokens/task | Output/task | Thinking/task | Tool calls/task | Min/task | Median min |
|---|---|---|---|---|---|---|---|---|---|
| Low | 23 | $1.72 | $39.65 | 0.65M | 16.2K | 7.7K | 10.7 | 4.6 | 3.5 |
| Medium | 23 | $1.72 | $39.51 | 0.59M | 15.0K | 7.1K | 10.3 | 4.3 | 3.6 |
| High | 23 | $2.86 | $65.70 | 1.19M | 30.8K | 18.3K | 16.4 | 7.8 | 6.2 |
| Extra-high | 23 | $4.99 | $114.76 | 2.51M | 61.7K | 40.8K | 27.0 | 14.2 | 13.4 |
| Max | 23 | $10.88 | $250.23 | 7.91M | 141.5K | 94.7K | 58.9 | 37.1 | 39.0 |
| Full sweep | 115 | $4.43 | $509.85 | 295M | 26.1 h |
Pricing. Claude Code reports its list-price total for the session on each result receipt, cumulatively. Two runs (Low matchcore and Extra-high reflowcore) have two receipts, because a background task resumed the session after the model stopped; tokens and thinking tokens are summed over both, and the cost is the session’s last total. The run summaries kept only the last receipt and recorded no thinking tokens (adapter issues, fix pending); every figure here comes from the raw streams and matches the population record. Grading is unaffected. The adapter’s own run-time estimate prices cached prompt tokens at the full input rate and reads higher (Max $14.01 a task); it is not used.
Tokens are input, cache read, cache creation and output tokens summed over a run’s receipts; the full sweep row sums them over all 115 runs. Per-run figures are in economics.json and runs.csv.
On the Frontier v4 board
Claude Fable 5.2 takes first place on the board’s 86 model and effort columns: Max ranks 1st (93.85 ± 0.42, n=23), Extra-high 2nd, High 4th, Medium 7th and Low 10th; Grok 4.7 at Extra-high (93.15) is third. Max carries the best tag. Its spread from Low to Max is under 2 points, so the board’s effort suggestions pick Extra-high for critical work and Medium for routine and rough work. The leaderboard carries every column; Claude Fable 5.2’s cells carry the ‖‖ footnote with the panel disclosure on this page.
Read these placements with the panel in mind. Claude Fable 5.2 is judged by Muse Spark 1.3 and GPT-6.1 Sol, as are Grok 4.7, DeepSeek V4.1 Flash and Reflection Beam; most other columns, Claude Fable 5.1 and Claude Opus 5.5 included, by Muse Spark 1.3 and Grok 4.6. Scores from different panels are not calibrated against each other, so gaps of a point or two between columns judged by different panels are not a reliable order. By Muse alone, Fable 5.2’s combined score is 91.09 to 92.78.
What Code quality measures
Every judge receives the same rubric, frozen by hash before any review. It names the reader it scores for: an engineer who has never seen the code, reads it top to bottom without running it, and must make a correct change in one sitting. It tells the judge that its own ease at parsing dense code is not evidence of readability. Six dimensions are scored 0 to 4 in half steps against written anchors: naming, presentation and intent form the human readability sub-score; structure, changeability and verifiability form the maintainability sub-score. Every score must cite an exact excerpt from the code and a concrete consequence for that reader. The host computes the sub-scores; the judge does no arithmetic.
The 33 points have three designed layers. The reviewed panel carries 24 points. An intent-recovery probe carries 9: the judge, given only the specification and the code, lists where the code departs from the specification, and a separate call matches that list against a frozen answer key, over the departures the submission passed tests for. The third layer, measured maintenance, is designed for 12 points but is not yet built, so the pre-registered 24 plus 9 split is in force.
| Component | Low | Medium | High | Extra-high | Max |
|---|---|---|---|---|---|
| Code quality | 80.63 | 81.49 | 82.95 | 84.25 | 86.02 |
| Human readability | 76.7 | 78.1 | 80.0 | 82.8 | 84.2 |
| Maintainability | 82.7 | 83.6 | 84.8 | 85.3 | 87.6 |
| Intent recovery | 83.1 | 83.2 | 84.5 | 84.8 | 86.3 |
| Rated by Muse Spark 1.3 | 76.4 | 77.1 | 78.4 | 80.4 | 82.2 |
| Rated by GPT-6.1 Sol | 83.1 | 84.6 | 86.4 | 87.7 | 89.6 |
| Standard error of Code quality | 1.00 | 1.22 | 1.13 | 1.24 | 1.15 |
Code quality rises at every level, by 5.40 points from Low to Max. Human readability is the weaker sub-score at every level. GPT-6.1 Sol is the higher rater at every level, by 6.7 to 8.1 points on the reviewed score. Per-run sub-scores from each judge are in runs.csv; the exact rubric, system text, probe and match instructions are in judge-protocols.json.
Judges and calibration
Round v3.30 seated Muse Spark 1.3 (Meta) and Grok 4.6 (xAI), the panel of the Claude Opus 5.5 column, and both retook the calibration exam. Grok 4.6 failed one gate, g14_pair_2_1 (a pair of controls it scored 100 one way round and 50 the other), which the one-gate allowance does not cover, so it judged nothing. Muse Spark 1.3 passed every gate and judged Low to Extra-high under v3.30 and Max under v3.30.1. On the owner’s choice, GPT-6.1 Sol (OpenAI) through the Codex CLI at medium effort took Grok’s seat, passed every gate under v3.30.2, and judged Low to Extra-high under v3.30.2 and Max under v3.30.3, on byte-copied freezes of the same populations. Neither judge’s lab is Anthropic. The rubric, prompts, schemas, seed, repeats, gate allowance, weights and both judges’ pins equal the published v3.27 record, the Muse and Sol round that judged DeepSeek V4.1 Flash.
Before scoring a single submission, each judge reviews ten held-out programs that implement the same ledger specification, each five times in a seeded order, and twenty gates fixed in advance check that the judge sees the construct; a judge may miss at most one gate by at most half a point.
| Judge | Protocol | Calls | Result | Allowance used | Failing gates |
|---|---|---|---|---|---|
| Muse Spark 1.3 | code-quality-maintenance-v3.30 | 80 | passed | no | none |
| Grok 4.6 | code-quality-maintenance-v3.30 | 80 | failed, judged nothing | no | g14_pair_2_1 |
| GPT-6.1 Sol | code-quality-maintenance-v3.30.2 | 80 | passed | no | none |
Every gate value and control mean is in calibration.json.
One judge alone. Scored by one judge alone, the combined score would be (Muse or Sol) 91.09 or 92.74 at Low; 91.16 or 93.34 at Medium; 91.53 or 93.99 at High; 92.13 or 94.37 at Extra-high; 92.78 or 94.91 at Max. Either way the ladder rises at every level.
Counted calls and recoveries. GPT-6.1 Sol made 80 calibration calls and 92 primary reviews, 4 repeats, 8 pairwise checks, 92 intent probes and 92 answer-key matches under v3.30.2, and 23 primary reviews, 1 repeat, 23 probes and 23 matches under v3.30.3. Muse Spark 1.3 made the same counts under v3.30 and v3.30.1. Twelve of Sol’s answer-key matches (8 and 4) numbered the departures from one instead of zero and were published through the protocol’s frozen rule for that slip, recover_one_based_indexes, which leaves every score unchanged because intent scoring reads match statuses only. One Muse intent probe under v3.30 quoted an excerpt with its line breaks changed; it was re-wrapped to the source’s line breaks, after which it had to be verbatim in the code and pass the frozen validator. No other call was edited, there is no operator marker, and neither judge produced a reviewer fallback. In UTC, Muse judged on November 10 (Low to Extra-high) and November 12 (Max), and Sol on November 12. Every operator record is in judge-protocols.json.
How it was run
- The model. Claude Fable 5.2 as
claude-fable-5-2in Claude Code 2.1.321, pinned by hash with auto-updates off: the first release that names the model. Claude Code reports the serving model on every reply, and every reply in every run isclaude-fable-5-2. - Harness settings. Permission mode auto, safe mode, no web access, a Claude subscription, refusal fallback on as for every Claude column (never used). Every integrity audit is clean: no web access, no benchmark data or answer-key paths. Every file written outside the workspace was scratch space under the temporary directory.
- Thinking summaries recorded. Runs used
--thinking-display summarized, so the traces keep summaries of the model’s thinking, a run condition earlier Claude columns did not have. The summaries are withheld from the public traces, which keep thinking as lengths only, because they describe the reference programs. - Five levels. Claude Code takes low, medium, high, xhigh and max for Claude Fable 5.2; the board’s Extra-high is xhigh.
- Max waited for the usage limit. The Fable weekly usage limit was used up by the end of Extra-high. The first Max attempts were refused by the limit, did no work, hold no summary and are kept apart, not part of the population. Max ran from 23:15 PST on November 10, when the limit reset, to 13:28 PST on November 11.
- From a tagged run worktree. Every run ran from
bench/2026-11-09-fable52-frontier-v4, so every run summary carries a clean source block and records the frozen suite lock’s task hash. - Two receipts in two runs. Claude Code 2.1.321 can emit a second result receipt when a background task resumes a session after the model stops. It happened in two runs; their totals sum both receipts, as described under cost.
- Where and how. The owner’s Mac, one task at a time, on November 9 to 11, 2026. Runs were not repeated, and effort labels are Claude Code’s own.
Evidence and reproduction
Public files contain all 115 run measurements, both judges’ six-dimension sub-scores and intent-recovery scores per run, the combined score from each judge alone, hidden behaviours fixed, integrity-audit verdicts, each run’s task hash, raw and thinking tokens, tool-call counts and list-price cost per run, five aggregates, every calibration gate value (Grok’s failed exam included), the exact judge protocol text, every operator recovery, the card’s per-level table, the Muse-alone table and source hashes. Each run’s Code quality is (0.24 × reviewed score + 0.09 × intent recovery) / 0.33, where both terms are the equal mean of the two judges. The combined score is 100 × (0.50F + 0.085Q + 0.085S + 0.33C / 100). The reproduction guide gives the public arithmetic checks and links the protocol document, the operations log, the decision record, the Fable 5.2 results and the code in the harness repository.
No human rated anything; the scores are model judgment for a human reader, not human validation. Hash checks bind the export to frozen files; they do not prove that judges were unbiased or that no training overlap exists.
Withheld: raw prompts and responses, thinking summaries, submitted patches and reconstructed sources, the quirk answer keys (they describe hidden-test behaviour), reviewer session identifiers and usage receipts, solver event streams (a structure-only redacted trace of every run, with every tool call and its timing and thinking as lengths only, is public in vulcanbench-fable52-traces), and host binary paths. Summed solver time is 26.07 hours; judging is excluded.