GPT-5.5 vs. GPT-5.6 Luna across every effort level

Across every effort level

Each model ran every task once at each effort level its API accepts, 23 tasks per cell: GPT-5.5 at Low, Medium, High and Extra-high, and GPT-5.6 Luna at those four plus Max. Both ran through the same Codex CLI on the same ChatGPT subscription, so this compares two models in one harness. Combined score weights are 50% functional correctness, 8.5% lint and complexity, 8.5% security and 33% Code quality, judged for a named human reader by Muse Spark 1.3 and Grok 4.6, two judges from labs with no model on this board, with a ground-truth intent-recovery probe.

Under the prior 50/15/15/20 profile applied to the same Code quality scores, the same picture holds: GPT-5.5 ahead at Low and Medium, Luna ahead at High, Extra-high and Max. The card and the full per-effort table follow.

Cost and tokens across effort levels

The same 207 runs priced from their Codex receipts at list API rates, cache-aware, with judging excluded. These are API-equivalent estimates, not subscription charges: both models ran on a ChatGPT subscription, and what the plan actually bills is not observable from the receipts. GPT-5.5 lists at 25 times Luna's input and output rates, and its per-task cost is 16.6 to 74.8 times Luna's at matched effort. Luna uses more tokens than GPT-5.5 from High up and reaches 11.88M per task at Max, most of it cache reads.

What Code quality measures

Every judge receives the same rubric, frozen by hash before any review. It names the reader it scores for: an engineer who has never seen the code, reads it top to bottom without running it, and must make a correct change in one sitting. It tells the judge that its own ease at parsing dense code is not evidence of readability. Six dimensions are scored 0 to 4 in half steps against written anchors: naming, presentation and intent form the human readability sub-score; structure, changeability and verifiability form the maintainability sub-score. Every score must cite an exact excerpt from the code and a concrete consequence for that reader. The host computes the sub-scores; the judge does no arithmetic.

The 33 points have three designed layers. The reviewed panel carries 24 points. An intent-recovery probe carries 9: every task in this suite is a rewrite of a retired binary whose real behaviour departs from its written specification in documented ways, and the judge, given only the specification and the code, must list where the code departs from the specification. A separate call matches that list against a frozen answer key, over the departures the submission passed tests for. The third layer, measured maintenance, is designed for 12 points but is not yet built, so the pre-registered 24 plus 9 split is in force and is stated on the card.

At each model's best effort the reviewed scores are close and GPT-5.5 reads slightly better; Luna recovers more of the hidden contract. Across matched efforts Luna's Code quality and human readability are higher at every level, while maintainability and intent recovery trade places; both judges rank the reviewed score the same way in every cell. Both models sit well below the Astra and Fable 5.1 board under the same rubric, 62 to 72 here against 70 to 82 there: the judges find their code harder for a person to read. Per-run sub-scores from each judge are in runs.csv; the exact rubric, system text, probe and match instructions are in judge-protocols.json.

Judges and calibration

The scored panel is Muse Spark 1.3 (Meta) through the Muse CLI on its Standard tier, and Grok 4.6 (xAI) through the Cursor CLI at medium effort, with equal weight. Neither lab has a model on this board, so neither judge grades a relative. Every session is fresh, tools are disabled, the workspace is empty, model identity is checked per call from the CLI's own records, and solver labels are withheld. Protocol v3.5 changes nothing in the rubric, controls, gates, repeats, seed or judge settings from v3.4; it applies the protocol to this population, and both judges retook the calibration exam under it before any counted call.

Before scoring a single submission, each judge reviews ten held-out programs that implement the same ledger specification: clear, compressed, compressed then auto-formatted, verbose with duplicated policy, needlessly abstracted, misleadingly commented, narrated with a comment on every line, a documented legacy quirk, an embedded instruction to give full marks, and hidden module state. Each program is reviewed five times in a seeded order. Twenty gates fixed in advance check that the judge sees the construct; a judge may miss at most one gate by at most half a point.

Muse Spark 1.3 used the pre-registered allowance: its five repeats on one control spread 0.02 of a point more than gate 11 permits, within the half-point allowance, and every other gate passed. Grok 4.6 passed every gate. Both judges separate the clear program from the compressed one by more than two points on naming and about two points on presentation, credit formatting on presentation but not on naming, penalise misleading comments on intent, and penalise hidden state on verifiability. Every gate value and control mean is in calibration.json.

Each judge made 718 calls: 80 in calibration, 207 primary reviews, 9 repeats, 8 pairwise checks, 207 intent probes and 207 answer-key matches. Muse needed a second attempt on five calls (two unsupported excerpts, two malformed JSON responses, one missing consequence in calibration) and Grok on six (four unsupported excerpts, one malformed response, and one transport fault when the Cursor CLI could not resolve its API host). The transport fault stopped the run for a person; the operator wrapper gained a rule that gives one fresh attempt after a judge CLI network fault, with the receipt retained, and the call was retried under it. Every second attempt is archived beside the first in the harness run directory.

Evidence and reproduction

Public files contain all 207 run measurements, both judges' six-dimension sub-scores and intent-recovery scores per run, raw tokens and API-equivalent cost estimates per run, nine aggregates, every calibration gate value, the exact judge protocol text and source hashes. Each run's Code quality is (0.24 × reviewed score + 0.09 × intent recovery) / 0.33, where both terms are the equal mean of the two judges, or the reviewed score alone where intent recovery has no denominator. The combined score is 100 × (0.50F + 0.085Q + 0.085S + 0.33C / 100). The reproduction guide gives the public arithmetic checks and links the protocol documents, controls, runner and operator wrapper.

No human rated anything; the scores are model judgment for a human reader, not human validation. Runs were not repeated, and effort labels are Codex's own. Hash checks bind the export to frozen files; they do not prove that judges were unbiased or that no training overlap exists.

Withheld: raw prompts and responses, submitted patches and reconstructed sources, the quirk answer keys (they describe hidden-test behaviour), and reviewer session identifiers and usage receipts. Summed solver time is 61.42 hours; judging is excluded.