DeepSeek V4.1 Flash across every effort level

Across every effort level

DeepSeek V4.1 Flash (API id deepseek-flash, which DeepSeek’s pricing page mapped to DeepSeek-V4.1-Flash-1017) ran every task once at each of the three reasoning levels DeepSeek’s API takes, 23 tasks per cell, through DeepSeek Harness 0.2.1-rc.2 on a DeepSeek API key, from October 20, 07:11 UTC, to October 22, 17:19 UTC, 2026, on the owner’s Mac in the local sandbox, one task at a time. The API has no Medium and nothing maps to Extra-high, so these are all the levels. Combined score weights are 50% functional correctness, 8.5% lint and complexity, 8.5% security and 33% Code quality, judged for a named human reader by Muse Spark 1.3 and GPT-6.1 Sol with a ground-truth intent-recovery probe.

Under the prior 50/15/15/20 profile applied to the same Code quality scores the ladder runs 90.28 to 90.70. List price is DeepSeek’s off-peak rate, which the board uses; billed is what DeepSeek charged, higher at Low because most Low runs fell in its weekday peak hours (see Cost and tokens).

Beside Claude Opus 5.5 and GPT-6.1 Sol

At Low, DeepSeek V4.1 Flash has the highest combined score of the three, 88.65 against 86.36 for Claude Opus 5.5 and 86.22 for GPT-6.1 Sol. At High and Max it is behind Opus 5.5 (91.11 and 90.22) and essentially level with GPT-6.1 Sol (88.23 and 88.35; Flash is 88.61 and 88.36). It fully solves 21, 20 and 20 tasks, against 17, 22 and 21 for Opus 5.5 and 20, 23 and 23 for GPT-6.1 Sol. It is faster than Opus 5.5 at every level, and faster than GPT-6.1 Sol at High and Max but slower at Low. At list price it is far cheaper: GPT-6.1 Sol costs 5.3, 5.0 and 6.2 times as much per task at Low, High and Max, and Opus 5.5 29, 49 and 118 times as much.

Cost, tokens and peak hours

The sweep billed a DeepSeek API key, so this is metered API spend, not an estimate. The list-price figure, which the board uses, prices every run’s tokens at DeepSeek’s off-peak rate for deepseek-flash, checked October 19, 2026 on DeepSeek’s pricing page: $0.15 per million input tokens on a cache miss, $0.003 on a cache hit and $0.60 output. DeepSeek bills 2x on weekdays from 01:00 to 04:00 and 06:00 to 10:00 UTC. The harness meters each call at the time it was made, so the billed figure is higher wherever calls fell in those hours: 14 of the 23 Low runs and one High run, none at Max. Judging is excluded.

On the Frontier v4 board

DeepSeek V4.1 Flash’s best level, Low at 88.65 ± 0.67 (n=23), ranks 22nd of the board’s 70 model and effort columns, just below GPT-6.1 Sol at Extra-high (88.78 ± 0.40). High and Max follow at 23rd and 24th, just above GPT-6.1 Sol at Max (88.35 ± 0.33), all within each other’s errors. Low carries the best tag, and because the three levels sit within 0.3 points, the board’s critical, routine and rough effort suggestions all pick Low, the cheapest. On the board’s cost view its line sits at the far left, under a tenth of a dollar per task. The leaderboard carries every column; DeepSeek V4.1 Flash’s cells carry the ¶¶ footnote with the disclosures on this page.

Read these placements with the panel in mind. Most columns, Claude Opus 5.5 and GPT-6.1 Sol included, are judged by Muse Spark 1.3 and Grok 4.6; Grok 4.7’s and DeepSeek V4.1 Flash’s by Muse Spark 1.3 and GPT-6.1 Sol; Muse Wonder Full’s by Claude Opus 5.5 and GPT-6.1 Sol. Scores from different panels are not calibrated against each other, so gaps of a point or two between DeepSeek V4.1 Flash and its neighbours are not a reliable order.

What Code quality measures

Every judge receives the same rubric, frozen by hash before any review. It names the reader it scores for: an engineer who has never seen the code, reads it top to bottom without running it, and must make a correct change in one sitting. It tells the judge that its own ease at parsing dense code is not evidence of readability. Six dimensions are scored 0 to 4 in half steps against written anchors: naming, presentation and intent form the human readability sub-score; structure, changeability and verifiability form the maintainability sub-score. Every score must cite an exact excerpt from the code and a concrete consequence for that reader. The host computes the sub-scores; the judge does no arithmetic.

The 33 points have three designed layers. The reviewed panel carries 24 points. An intent-recovery probe carries 9: the judge, given only the specification and the code, lists where the code departs from the specification, and a separate call matches that list against a frozen answer key, over the departures the submission passed tests for. The third layer, measured maintenance, is designed for 12 points but is not yet built, so the pre-registered 24 plus 9 split is in force. Frontier Code quality is therefore the reviewed layer plus intent recovery, and it is never compared with Routine v1 Code quality, which is scored on a different suite and population.

Intent recovery is the strongest component at every level, 81.2 to 82.6: given only the specification and the code, the judges find most of the departures DeepSeek V4.1 Flash implemented. The reviewed layer is lower, and human readability, 65.4 to 68.9, is its weakest part at every level. Per-run sub-scores from each judge are in runs.csv; the exact rubric, system text, probe and match instructions are in judge-protocols.json.

Judges and calibration

GPT-6.1 Sol in Grok 4.6’s seat. The scored panel is Muse Spark 1.3 (Meta) through the Muse CLI on its Standard tier, with its v3.4 settings and binary pin, and GPT-6.1 Sol (OpenAI) at medium effort through Codex CLI 0.159.0, its launcher and native binary pinned by sha256, with equal weight. Grok 4.6, the second judge for most columns, runs through Cursor, and the Cursor account was out of usage on October 23, 2026. Rather than wait, the owner put GPT-6.1 Sol in the seat, as protocol v3.20 did for Grok 4.7; this is the same pair, with the same settings and pins. Neither lab is DeepSeek. Muse Code’s updater had removed the v3.4 Muse binary from its usual place, so the round ran a byte-identical backup, matched by sha256 and re-checked before every stage. Every session is fresh, tools are disabled, the workspace is empty and solver labels are withheld. Muse’s identity is checked per call from the CLI’s own records; Codex does not record the serving model, so GPT-6.1 Sol’s identity is the model requested.

A different panel from the columns it is shown beside. Claude Opus 5.5 and GPT-6.1 Sol’s own columns, like most columns on the board, are judged by Muse Spark 1.3 and Grok 4.6. Panels are not calibrated against each other, so DeepSeek V4.1 Flash’s Code quality and combined score are not strictly comparable with theirs. GPT-6.1 Sol therefore judges a column it is shown beside; its own column was judged without it. Protocol v3.27 changes nothing in the rubric, controls, quirk keys, gates, repeats, seed or weights from v3.7; it applies the protocol to this population with this judge pair.

Before scoring a single submission, each judge reviews ten held-out programs that implement the same ledger specification: clear, compressed, compressed then auto-formatted, verbose with duplicated policy, needlessly abstracted, misleadingly commented, narrated with a comment on every line, a documented legacy quirk, an embedded instruction to give full marks, and hidden module state. Each program is reviewed five times in a seeded order. Twenty gates fixed in advance check that the judge sees the construct; a judge may miss at most one gate by at most half a point. Both judges retook the exam under v3.27 before any counted call.

Both judges passed every gate with no allowance used. Every gate value and control mean is in calibration.json.

The judges disagree on level. GPT-6.1 Sol rates DeepSeek V4.1 Flash’s reviewed Code quality about 8 points above Muse Spark 1.3, 9.6 at Low, 8.2 at High and 7.3 at Max: 74.5 against 64.9 at Low, 76.6 and 68.4 at High, 74.5 and 67.1 at Max. Both passed the same exam, so this is a difference in how strictly each scores real code, not a failed judge; GPT-6.1 Sol rated Grok 4.7 about 6 points above Muse in the same way. The published score is their mean. Scored by one judge alone, the combined score would be 87.21 (Muse) or 90.10 (Sol) at Low, 87.24 or 89.98 at High, and 87.14 or 89.57 at Max; either way the three levels sit within 0.6 points of each other. Each run’s single-judge figures are in runs.csv.

Each judge made 69 primary reviews, 69 intent probes, 69 answer-key matches, 3 repeats and 6 pairwise checks. Muse needed a second attempt on two calls and Sol on ten, under the protocol’s single-retry rule. Six of Sol’s answer-key matches are published through a standing operator rule: both attempts failed only because Sol numbered the departures from one instead of zero, so the wrapper moved the indexes down by one with every status untouched and kept the original in the receipt. Neither judge produced a reviewer fallback. Judging ran on October 23, 2026, from 09:34 to 12:36 UTC.

How it was run

• The model. DeepSeek V4.1 Flash through DeepSeek’s API as deepseek-flash, which DeepSeek’s pricing page mapped to DeepSeek-V4.1-Flash-1017 when it was checked before each level. DeepSeek Harness records the model it requested, not the id in DeepSeek’s response, so model identity is harness-recorded.
• DeepSeek’s own harness, as shipped. DeepSeek Harness (@deepseek-ai/dsh, DeepSeek’s official open-source agent, MIT licence), version 0.2.1-rc.2, a developer preview prerelease, in its headless profile, pinned by exact version and a hash of its whole install tree. It ran inside VulcanBench’s macOS outer sandbox with dsh’s own sandbox and approvals off, since the sandboxes cannot nest, and a fresh home folder per run. Web tools were off, as for every harness on the board. dsh uploads each session’s log to DeepSeek by default and has an OpenTelemetry feedback upload; at the owner’s choice both were off. The model never sees either upload, and task content still reaches DeepSeek in the model requests. Everything else, including subagents, compaction and session titles, was as shipped.
• Three levels. DeepSeek’s API takes low, high and max reasoning; it has no medium and nothing maps to extra-high. Effort labels are DeepSeek’s own.
• From a tagged run worktree. Every run ran from bench/2026-10-20-deepseek-dsh-frontier-v4, so every run summary carries a clean source block and records the frozen suite lock’s task hash. No hash bridge was needed.
• Two pauses, one macOS update. The owner paused the sweep twice between tasks during Low. Each run a pause interrupted (Low metercore and Low reflowcore) was set aside unscored and the task rerun. macOS went from 26 to 27.0.1 during the second pause; re-grading the 13 Low runs finished before it on 27.0.1 gave 13 identical functional scores, and the pinned dsh install hash was unchanged.
• No timeouts, no overlapping jobs. Every run finished inside the flat 3-hour bound; the longest, Max lodgecore, took 43.7 minutes. A per-minute log watched for any other local benchmark job beside the sweep and recorded none.
• Where and how. The local sandbox on the owner’s Mac, one task at a time, from October 20, 07:11 UTC, to October 22, 17:19 UTC. The API key was in the agent’s environment and the shell network was not isolated, as recorded on every run. Every integrity audit is clean: no web access, and no benchmark data or answer-key paths.
• One attempt per task and level. Runs were not repeated.

Evidence and reproduction

Public files contain all 69 run measurements, both judges’ six-dimension sub-scores and intent-recovery scores per run, the combined score from each judge alone, hidden behaviours fixed, integrity-audit verdicts, each run’s task hash and harness version, model calls, raw tokens, list and billed cost, three aggregates, every calibration gate value, the exact judge protocol text and source hashes. Each run’s Code quality is (0.24 × reviewed score + 0.09 × intent recovery) / 0.33, where both terms are the equal mean of the two judges. The combined score is 100 × (0.50F + 0.085Q + 0.085S + 0.33C / 100). The reproduction guide gives the public arithmetic checks and links the protocol document, the decision record, the DeepSeek Harness adapter and the code in the harness repository.

No human rated anything; the scores are model judgment for a human reader, not human validation. Hash checks bind the export to frozen files; they do not prove that judges were unbiased or that no training overlap exists.

Withheld: raw prompts and responses, submitted patches and reconstructed sources, the quirk answer keys (they describe hidden-test behaviour), reviewer session identifiers and usage receipts, dsh session logs and solver traces, the two pause-interrupted attempts, and host binary paths. Summed solver time is 10.47 hours; judging is excluded.