Gemini 4 Neon solved 86 of its 92 VulcanBench Frontier v4 runs: every task at Medium, High and Extra-high, and 17 of 23 at Low. This report is about how. Its traces show a model that collects before it reasons. It reads the repository, runs the program, then reads the machine code, and only after all of that does it think at length and write the rewrite, usually in a single edit. Turning up the effort setting makes it read more, not check more.
The task in one paragraph
Each Frontier v4 task hands the model a small repository: a module that is supposed to reproduce a retired program, a written specification, and the retired program itself as compiled binaries for macOS and Linux. The module is wrong in ways the issue only hints at, and the specification may have drifted from what the program really does. Hidden tests compare the repaired module with the binary, byte for byte, on inputs the model never sees. The binary is the ground truth; everything else is a claim about it. (Methodology has the full design, and Report 01 looks at Claude Fable 5.2 on the same tasks.)
What this report does not show. Neon's Antigravity sessions recorded a summary of its thinking with each model response. Those summaries are full of what it learned about each program, which would hand future models the answers, so this report shows only their structure: counts, lengths, timing and the actions they lead to. It never quotes them.
Anatomy of a solve
Most Neon runs follow the same arc, and the phases come in this order at every level:
- GatherOne command lists the repository, then a batch of single-file reads takes in the module, the specification, the README and the legacy notes. Median: four or five files read before any machine code.
- Run the programAt Low and Medium it usually feeds the reference a few hand-typed lines before reading its machine code (16 of 23 runs at each level). At Extra-high it mostly skips this step (4 of 23).
- Read the machine codeSymbols and strings, then a disassembly saved to scratch space and read in passes. Median: its 9th or 10th tool call, still inside the first minute. From High up it reads both the macOS and the Linux build in 18 of 23 runs.
- ThinkThe longest stretch of reasoning in the run, holding about half of its thinking. At Low it leads straight into the rewrite; from High up it usually sends Neon back to the machine code for more.
- ExperimentScripted batches of inputs through the reference, often before the module is touched at all (14 to 17 runs of 23 at every level).
- Write once, check, finishThe rewrite, usually in one or two edits with the edit tool, then a randomized comparison with the reference and a final look at the diff. Every one of the 92 patches changes only the module.
The chart below lays one task, blendcore, end to end at each level, with Claude Fable 5.2's Low and Extra-high runs on the same task underneath for contrast. Each tick is a tool call, coloured by what it did; each bar above the line is the thinking that came before it.
Ticks: tool calls, coloured by activity. Bars above the line: thinking tokens spent before that call (height is the square root, so small bursts stay visible). The short bar under each label is the run's wall-clock length, on one scale for all six runs.
Every Neon row opens with a run of reading ticks and almost no thinking: it collects first. The tallest bar then moves earlier as effort rises, from about four fifths of the way through the Low run, right before the one edit, to just over a quarter of the way through at Extra-high, where it is followed by more reading and experiments. Fable 5.2's rows reach the machine code within seconds and, at Extra-high, fill the second half with probing and checking.
Walkthrough: three minutes at Low
Blendcore at Low took Neon 3.4 minutes, 15 tool calls and 6,926 thinking tokens. The run, step by step, with times from the start:
- 0.0 to 0.2 min · gatherOne command lists the repository and the legacy folder; three reads take in the module, the specification and the README together.
- 0.4 to 0.9 min · run the programIt pipes a three-line command stream into the reference's launcher, reads the launcher script, then copies the macOS build to scratch space and runs it directly.
- 1.0 to 1.4 min · read the machine codeSymbols and strings, then a full disassembly written to a scratch file and read in two passes.
- 1.4 to 2.7 min · think, then rewrite6,239 thinking tokens, 90% of the run's thinking, in one response, which ends in a single edit to the module.
- 3.0 min · checkOne script sends random command streams to the reference and to the rewrite and compares the output.
- 3.1 min · finishA look at
git status, and it reports.
Across all 23 Low tasks Neon averages 27 tool calls, 3 edits and 8 test runs, more than at any higher level, with the first edit at a median of 2.7 minutes, 72% of the way through the run. Its biggest burst of thinking follows a binary inspection in 19 of 23 runs and comes right before an edit in 13.
Walkthrough: ten minutes at Extra-high
The same task at Extra-high took 10.3 minutes, 32 tool calls and 40,750 thinking tokens, nearly six times the Low run. The rewrite it shipped is about the same size. Here is where the extra work went:
- 0.0 to 0.3 min · gatherThe file tree, five files read in one batch (the README, the module, the specification, the legacy notes and the launcher), then a check of the machine's architecture and of which disassemblers are installed.
- 0.4 to 0.8 min · read the macOS buildLinked libraries, symbols, section layout and the string table, then a full disassembly saved to scratch and read in two passes. No hand-typed input this time.
- 0.8 to 2.9 min · think, then look again17,273 thinking tokens, 42% of the run's thinking, and its longest summary (about 19,800 characters). The response that follows does not write anything: it goes back to the disassembly to look up one address.
- 2.9 to 4.0 min · read the Linux buildAnother 3,009 tokens, then the second binary: section headers, strings, the main code range disassembled in pieces and read, and a short script that decodes a data table straight from the file's bytes.
- 4.0 to 7.1 min · experiment before editingFour scripts send batches of hand-built inputs through the reference (odd whitespace and unusual bytes among them) to confirm what it read in the machine code, with about 12,700 thinking tokens between them. The module is still untouched.
- 8.0 min · rewriteOne edit to the module, after 3,914 tokens of thinking.
- 8.7 to 9.7 min · check and touch upA randomized comparison of module and reference, a re-read of the module, one small second edit and another randomized comparison.
- 9.8 min · clean up and finishIt deletes its scratch copy of the binary, looks at
git statusand the diff, and reports.
Most of the extra six minutes went into understanding the program before writing anything: a second binary, a second look at the first, and experiments against the reference. The checking after the rewrite is about as long as at Low. That is the Extra-high pattern across all 23 tasks: the first edit still lands about 75% of the way in, and the module gets a mean of 1.3 edits.
Where the extra thinking goes
Neon's thinking per task grows 4.4 times from Low to Extra-high and its time doubles, from 5.1 to 10.0 minutes, while thinking grows from 45% of output to 71%. To see what that thinking is for, each response's thinking is credited to the action that followed it: reading, probing the binary by hand, editing the module, testing, or wrapping up. Report 01 did the same for Fable 5.2, which is shown underneath.
The two models move in opposite directions. At Low, Neon's thinking is spread across reading (32%), editing (29%) and testing (34%). By Extra-high, 71% of it comes before reading the repository or the binary, and testing falls to 15%. Fable 5.2 goes the other way: its share before reading falls from 47% at Medium to 30% at Extra-high while probing and testing grow. Neon's biggest burst shows the same drift. At Low it comes right before an edit in 13 of 23 runs; at Extra-high it never does, and in 21 of 23 runs it comes right before more reading.
Over the wall clock, Neon thinks late. Only 4% to 10% of its thinking falls in the first fifth of a run, where it is gathering files and running the program, against 10% to 24% for Fable 5.2 from Low to Extra-high, and 61% to 80% lands between 40% and 80% of the way through. At higher effort the peak moves earlier, toward the reading of the machine code.
The extra reading is concrete. From High up, Neon reads the Linux build as well as the macOS build in 18 of 23 runs, against 4 or 5 for Fable 5.2. It also runs scripted experiments before its first edit at every level, where Fable 5.2 mostly does so only at Extra-high and Max. And it stops typing input into the reference by hand before reading the machine code: 16 runs at Low, 4 at Extra-high, while Fable 5.2 climbs from 1 to 14.
| Runs that | Model | Low | Medium | High | Extra-high |
|---|---|---|---|---|---|
| Ran the reference before reading its machine code | Neon | 16/23 | 16/23 | 11/23 | 4/23 |
| Fable 5.2 | 1/23 | 2/23 | 10/23 | 14/23 | |
| Ran scripted experiments before the first edit | Neon | 17/23 | 14/23 | 16/23 | 16/23 |
| Fable 5.2 | 4/23 | 3/23 | 9/23 | 16/23 | |
| Read both the macOS and the Linux binary | Neon | 0/23 | 1/23 | 18/23 | 18/23 |
| Fable 5.2 | 0/23 | 0/23 | 4/23 | 5/23 | |
| Read the machine code before the first edit | Neon | 21/23 | 23/23 | 23/23 | 23/23 |
| Fable 5.2 | 23/23 | 23/23 | 23/23 | 23/23 |
Machine code read before the first edit: the two Low runs that never read it, queuecore and schedcore, solved their tasks anyway by running the reference instead.
What does not grow is the checking after the rewrite. Counted from the commands each run executed, none of Neon's 92 runs used any of the four extra checks that mark Fable 5.2's high-effort runs: mutation testing (planting deliberate bugs to see whether the checks catch them), running the reference's Linux build in a container, a throughput check, or a second Python version. Its test runs per task fall from 7.7 at Low to 3.3 at High and 4.6 at Extra-high.
| Check | Model | Low | Medium | High | Extra-high |
|---|---|---|---|---|---|
| Mutation testing | Neon | 0/23 | 0/23 | 0/23 | 0/23 |
| Fable 5.2 | 0/23 | 0/23 | 3/23 | 12/23 | |
| Linux build in a container | Neon | 0/23 | 0/23 | 0/23 | 0/23 |
| Fable 5.2 | 0/23 | 0/23 | 5/23 | 13/23 | |
| Throughput check | Neon | 0/23 | 0/23 | 0/23 | 0/23 |
| Fable 5.2 | 0/23 | 0/23 | 2/23 | 3/23 | |
| Second Python version | Neon | 0/23 | 0/23 | 0/23 | 0/23 |
| Fable 5.2 | 0/23 | 0/23 | 0/23 | 1/23 |
Its thinking summaries read differently too. Keyword counts over the summaries (the same rough rules Report 01 used) find Neon hedging in almost every run at every level: words such as likely, probably and hypothesis appear in 20 to 23 runs of 23, where Fable 5.2 used them in about half its runs until Extra-high. What grows for Fable 5.2 and barely moves for Neon is talk about the job around the code: doubting the specification (7 or 8 Neon runs at every level, against Fable 5.2's climb from 3 to 19), keeping the change small (1 to 11, against 4 to 21) and cleaning up scratch files (0 to 8, against 2 to 21).
| Runs whose thinking mentions | Model | Low | Medium | High | Extra-high |
|---|---|---|---|---|---|
| Machine code and disassembly | Neon | 21/23 | 23/23 | 23/23 | 23/23 |
| Fable 5.2 | 23/23 | 23/23 | 23/23 | 23/23 | |
| Hypotheses (likely, probably, I suspect) | Neon | 20/23 | 23/23 | 23/23 | 23/23 |
| Fable 5.2 | 11/23 | 12/23 | 12/23 | 21/23 | |
| Checking itself (verify, confirm) | Neon | 22/23 | 23/23 | 23/23 | 23/23 |
| Fable 5.2 | 23/23 | 18/23 | 23/23 | 23/23 | |
| Edge cases | Neon | 20/23 | 23/23 | 23/23 | 23/23 |
| Fable 5.2 | 22/23 | 22/23 | 23/23 | 23/23 | |
| Comparing against the reference | Neon | 20/23 | 20/23 | 22/23 | 22/23 |
| Fable 5.2 | 23/23 | 23/23 | 23/23 | 23/23 | |
| Probing the program | Neon | 16/23 | 13/23 | 12/23 | 15/23 |
| Fable 5.2 | 20/23 | 18/23 | 16/23 | 18/23 | |
| Distrusting the specification | Neon | 7/23 | 7/23 | 8/23 | 8/23 |
| Fable 5.2 | 3/23 | 4/23 | 10/23 | 19/23 | |
| Keeping the change small | Neon | 1/23 | 7/23 | 5/23 | 11/23 |
| Fable 5.2 | 4/23 | 8/23 | 13/23 | 21/23 | |
| Cleaning up | Neon | 0/23 | 4/23 | 3/23 | 8/23 |
| Fable 5.2 | 2/23 | 4/23 | 17/23 | 21/23 |
All of Neon's per-level means in one place:
| Per task, mean | Low | Medium | High | Extra-high |
|---|---|---|---|---|
| Solved (all hidden tests pass) | 17/23 | 23/23 | 23/23 | 23/23 |
| Minutes | 5.1 | 5.9 | 6.4 | 10.1 |
| Output tokens | 20.1k | 28.4k | 38.2k | 56.9k |
| of which thinking | 9.1k (45%) | 15.6k (55%) | 25.3k (66%) | 40.3k (71%) |
| Model responses with a thinking summary | 14 | 12 | 12 | 18 |
| Largest single burst, share of thinking | 45% | 60% | 53% | 48% |
| Tool calls | 27.5 | 23.8 | 24.5 | 31.0 |
| Binary inspection calls | 8.0 | 6.9 | 7.6 | 9.5 |
| Calls that run the reference by hand | 1.5 | 1.6 | 1.6 | 1.0 |
| Test runs | 7.7 | 5.1 | 3.3 | 4.6 |
| Edits to the module | 3.0 | 1.7 | 1.3 | 1.3 |
| Edit-then-test cycles | 1.5 | 1.0 | 1.0 | 1.0 |
| First binary inspection, median tool call | 9 | 10 | 9 | 9 |
| First edit, median minute | 2.7 | 3.6 | 4.5 | 7.5 |
| Share of the run before the first edit, median | 72% | 75% | 75% | 75% |
The six misses at Low
Neon left hidden tests failing in 6 of its 23 Low runs (cellarcore, hedgecore, lodgecore, matchcore, quotacore and tallycore, with functional scores from 0.75 to 0.92) and in none above Low. All six read the machine code before editing, so the misses are not a black-box habit. They differ from the 17 solved Low runs in how the thinking was spent:
17 solved Low runs
median per run- Biggest burst: 56% of the run's thinking.
- About 6,900 thinking tokens in 3.6 minutes.
- 3 test runs and 2 edits.
6 Low runs with failing tests
median per run- Biggest burst: 25% of the run's thinking.
- About 7,400 thinking tokens in 4.2 minutes.
- 7 test runs and 3.5 edits.
The misses used about as much thinking as the solves but in smaller pieces, with more rounds of edit and test, as if Neon were patching by trial instead of reasoning its way to the rewrite first. At Medium it solved all six. For four of them (hedgecore, matchcore, quotacore and tallycore) the Medium run spent two to three times the thinking of the Low run, about 14,000 to 15,000 tokens against 4,900 to 7,500, in under two minutes more. The other two, cellarcore and lodgecore, were already Neon's longest Low runs, at 12.2 and 9.0 minutes. With six cases this is a pattern worth noting, not proof of a cause; harder tasks may invite both scattered thinking and failure.
Against Claude Fable 5.2
Fable 5.2 is the natural comparison: it solved all 115 of its runs, its streams are complete, and Report 01 measured them with the same rules used here. Neon has no Max level, so the comparison runs from Low to Extra-high.
Gemini 4 Neon
Antigravity CLI · 86 of 92 solved- Gathers files and runs the program first; first machine-code read at a median of its 9th or 10th call, 0.6 to 0.8 minutes in.
- About 75% of the run passes before the first edit, at every level.
- Extra effort goes to reading: a second binary, a second look at the first, experiments before writing.
- Patches change only the module and grow with effort, from a median of 51 added lines at Low to 83 to 86 from High up, with comments on 1% to 2% of them.
Claude Fable 5.2
Claude Code · 115 of 115 solved- Goes straight to the machine code; first read at a median of its 2nd to 5th call, 0.1 to 0.3 minutes in.
- The first edit comes earlier as effort rises: 60% of the way through at Low, 50% at Extra-high.
- Extra effort goes to proving the rewrite: mutation testing, the Linux build in a container, throughput.
- Patches change only the module and stay at about 100 added lines at every level, with comments on 12% to 19% of them.
At the two ends of the dial the two are close on thinking: 9.1k against 7.7k tokens a task at Low, and about 40k each at Extra-high, with thinking at 71% and 66% of output there. In between Neon thinks more, twice as much at Medium (15.6k against 7.1k). Neon is a little slower at Low and Medium and faster at High and Extra-high (10.0 minutes against 14.0 at Extra-high). The difference is not how much they think but what for. Neon front-loads understanding and checks lightly after writing; Fable 5.2 reads quickly, writes earlier and keeps checking, more so as effort rises.
Fable 5.2 also never left a hidden test failing, at any level. Neon's six misses all came at its cheapest setting, where its thinking was most scattered and its test runs were at their most numerous, which suggests that at Low the trial-and-error loop is standing in for reasoning it did not do up front.
| Per task | Model | Low | Medium | High | Extra-high |
|---|---|---|---|---|---|
| Thinking tokens, mean | Neon | 9.1k | 15.6k | 25.3k | 40.3k |
| Fable 5.2 | 7.7k | 7.1k | 18.3k | 40.8k | |
| Thinking share of output | Neon | 45% | 55% | 66% | 71% |
| Fable 5.2 | 48% | 47% | 59% | 66% | |
| Minutes, mean | Neon | 5.1 | 5.9 | 6.4 | 10.1 |
| Fable 5.2 | 4.6 | 4.3 | 7.8 | 14.2 | |
| Tool calls, mean | Neon | 27.5 | 23.8 | 24.5 | 31.0 |
| Fable 5.2 | 10.7 | 10.3 | 16.4 | 27.0 | |
| First binary inspection, median tool call | Neon | 9 | 10 | 9 | 9 |
| Fable 5.2 | 2 | 2 | 4 | 5 | |
| Share of the run before the first edit, median | Neon | 72% | 75% | 75% | 75% |
| Fable 5.2 | 60% | 58% | 53% | 50% | |
| Test runs, mean | Neon | 7.7 | 5.1 | 3.3 | 4.6 |
| Fable 5.2 | 3.0 | 1.3 | 3.0 | 6.5 | |
| Edits to the module, mean | Neon | 3.0 | 1.7 | 1.3 | 1.3 |
| Fable 5.2 | 1.8 | 1.8 | 2.4 | 4.3 |
The two models ran in different harnesses with different tools. Antigravity's read_file reads one file per call, so Neon's tool-call counts and the call number of its first machine-code read run higher than Claude Code's for the same work; compare the minutes instead.
What it adds up to
- It gathers before it thinks. Neon reads the repository's files and, at lower effort, runs the program before it disassembles anything, and only 4% to 10% of its thinking falls in the first fifth of a run. It still reaches the machine code inside the first minute, in 90 of 92 runs before its first edit.
- It thinks late and writes once. About three quarters of every run passes before the module is touched, at every effort level, and from Medium up the module gets one or two edits.
- Effort buys understanding, not proof. From Low to Extra-high the share of thinking spent before reading rises from 32% to 71%, the Linux build joins the macOS build, and experiments move ahead of the rewrite. The checking after the rewrite does not grow, and none of the 92 runs used the heavier checks Fable 5.2 reaches for.
- Low is where it slips. All six of its failures came at Low, in runs whose thinking was split into small pieces and spent on rounds of patching. One level up, the same tasks got two to three times the thinking and were solved.
Methods and caveats
Runs. Gemini 4 Neon (gemini-4-neon) in Antigravity CLI 1.5.12 on a Google AI Pro subscription, at four effort levels (Neon has no Max), one run per task per level, from November 20 to 21, 2026, from bench/2026-11-20-gemini4-neon-frontier-v4. Neon ran through a custom Antigravity agent with file and shell tools only and no web access. The AI Pro quota refused 40 attempts during the sweep, four of them at Extra-high after the run had started work; every refused attempt was discarded and the task rerun from scratch, so each published run is one uninterrupted session. Claude Fable 5.2's runs are the ones in Report 01 (Claude Code 2.1.321, November 9 to 11).
Sources. For each of Neon's 92 runs: the Antigravity event stream (tool calls and per-response token counts), the tool-call log with start and end times, the Antigravity session database (for the thinking summaries' count, length and keywords) and the final patch. Fable 5.2's figures are Report 01's published data, plus three order measures added here (running the reference before reading its machine code, scripted experiments before the first edit, reading both binaries), computed from its raw streams with the same code.
Activities. Each Antigravity tool is mapped onto Report 01's vocabulary (run_command as a shell command, read_file as a read, edit_file and write_file as edits) and every call goes through Report 01's classification rules unchanged. The same caveats apply: commands that do two things get one label, scripts are counted as tests whether they send input to the reference, the module or both, and the shares are approximate.
Thinking. Antigravity reports thinking tokens for every model response, so no estimate is needed: each response's thinking goes to the first tool call it made, and a response with no tool call is the closing report. Its output tokens include thinking, as Claude Code's do. Fable 5.2's per-activity thinking in Report 01 is a rescaled estimate, which makes the split chart a close but not exact comparison. Thinking summaries are summaries, not the raw reasoning; theme counts are rough keyword matches.
Scores. This report is about how Neon works, not how it ranks. Neon's combined score runs from 84.32 at Low to 87.87 at High (87.48 at Medium, 87.70 at Extra high), with Code quality judged by Muse Spark 1.3 and Grok 4.6 (round v3.32). Two codeccore runs, one at Low and one at Extra high, have no published Code quality score: Grok's follow-up check returned malformed output twice on each, and the frozen protocol leaves such a run unpublished, so those two levels are judged on 22 runs. Both runs are included in this report's trace figures. Fable 5.2 was judged by a different panel (Muse Spark 1.3 and GPT-6.1 Sol), so the two models' Code quality scores are not a like-for-like comparison. Full results are on the Gemini 4 Neon benchmark page.
Data. summary.json (every figure on this page) and runs-traces.csv (one row per Neon run). They are built by scripts/build_report02_data.py from the private run records and drawn by scripts/build_report02_charts.py. Neither file contains thinking text or task content.