October 2026 AI Papers: Reading the Headline Numbers

The most trustworthy number from October's AI papers is also one of the smallest.

It is the 39% from Scaling Discovery Requires Forgetting, and it beats GitSwarm's 79.4% on that score for one plain reason. The Berkeley and Stanford team compared their method against a single agent on the same budget. GitSwarm compared hundreds of agents against one.

Our October roundup says what seven papers found, and the agent setup guide turns four of them into instructions-file lines. Neither stops on the numbers. The pages ranking for this month's papers don't either: two daily digests from lilys.ai, a few hundred words each in themed buckets, and a yearly arXiv index that is mostly titles.

So this page does one thing. It asks what each headline number we've published is a number of. Every figure is the authors' own, from the arXiv abstract or DAIR's weekly summary as reported in our roundup. The arithmetic in between is mine.

The short answer

Four questions to ask of any AI paper's number

They work on vendor launch posts too, which borrow the same habits.

1. What stayed the same between the two columns? A result only isolates the method if everything else was held fixed. Usually that means the model, the budget, or the test set.
2. Percent or percentage points? Going from 4% to 8% is 4 points and also a 100% gain. Papers pick the framing that reads larger, and so do the people summarizing them.
3. Does "up to" hide a range? It means the best case. Ask what the worst case was.
4. Who ran the comparison? In every paper on this page, the comparisons are the authors' own. That is normal. It still means the baseline was configured by the people who wanted it to lose.

Our guide to AI tools for scientific research lists the slower checks, like released code and reproduction by another group. Those take months. These take a minute.

GitSwarm: 79.4% against 65.1%

That is a gap of 14.3 points, and on its face it is the most impressive result of the month. Read the two columns again, though. One is hundreds of asynchronous Codex workers (GPT-5.5, per DAIR) sharing a Git repository. The other is one Codex agent working alone. The summary we worked from does not say the two sides got the same compute, and when one side is hundreds of agents it would be strange if they had. So the 14.3 points measure GitSwarm plus a great deal more inference, and the paper's own name for the idea, compounding inference, more or less admits it.

The 30 out of 30 on IMOProofBench-Advanced is a ceiling. It cannot show how much headroom was left.

My favourite GitSwarm figure is the 70.1%, and it needs the most care. Its denominator is failed research commits, and what it counts is how many were cited by later accepted work. Citation is reuse. A later agent might cite a dead end to avoid it, which is useful, or to repeat it, which is not, and the number cannot tell those apart.

The forgetting paper: up to 39% over one agent

This is the number I would trust most this month, and I would still shrink it in my head.

The team gave every arm the same budget, then compared periodic wipe-and-restart against one long-running agent, parallel agents, and parallel agents trading candidates. That is the comparison you want. The gains were up to 15% over parallel agents and up to 39% over a single agent across four open-ended tasks, from GPU kernel optimization to model training. "Up to" is doing work in both figures. Four tasks produced those maxima, and the summaries we have do not give the low end. Expect less than 39% from your own restart habit. Keep it anyway; it's free.

LongHarness: 15.8% to 60.2%, and the Claude Code comparison

The exact-match jump on OfficeQA Pro V1 is 44.4 points. As a ratio the new score is nearly four times the old one, and on V2, where the base score went from 4.4% to 61.1%, it is close to fourteen times. Both framings are true. The points are the honest one to repeat, because a tiny starting score inflates any ratio. A score of 4.4% means the base model was answering about one question in twenty-three. Almost anything structured would multiply that.

Then the line most people will quote: LongHarness beat Claude Code by 12.0 points on V1 and Codex by 18.9 points on V2. The backbone was held fixed. That means the Claude Code in this comparison was running on the same model as LongHarness, and the Claude Code you can buy runs only on Claude models, per our Claude Code vs Codex comparison. The paper measured harness against harness. That is a real finding about how Claude Code keeps track of evidence, and it fits the weakness our NotebookLM vs Perplexity Spaces vs Claude Projects test found once a project passes 200,000 tokens. Your subscription was never in the test.

The quantum circuit paper: 7 of 8

The discovered circuits cut both Toffoli gates and qubits against the state of the art in seven of eight tasks.

It is tempting to file the eighth as a loss. Our sources do not describe it as one. In that task the search found a new reversible architecture for prime-field modular inversion, which made a classical algorithm called divsteps practical. That is a discovery, and the gate-count scoreboard has no column for it. Counting it as a miss undersells the paper. Counting it as an eighth win oversells it.

LoGRA: two memory figures that look like one

NVIDIA reports 45.7% less training memory per GPU at 7B parameters. Separately, at 1.5B parameters, LoGRA used 7.2 GiB against 13.2 GiB for rank-256 LoRA. That works out to about 45.5%, close enough to pass for the same measurement. They are at different model sizes, and only the 1.5B figure names its baseline in what we reported. If you fine-tune, quote the one that matches the model size you actually train.

Looking Ahead has no number

Our sources describe Tencent ARC Lab's result as better physical plausibility on a lower training budget, and that is the whole claim as we have it.

What a careful reader takes from October

Rank the month by how well each number survives the four questions and the order flips. The forgetting paper goes first, because it held the budget fixed and its gains are modest enough to be believable. LongHarness is a strong result about harness design that keeps getting quoted as a result about products. GitSwarm's headline is the biggest and the hardest to compare with anything, since nothing we know of matched its compute on the other side. That ordering is a judgment, and I am putting my name on it.

For when outsiders get to test these numbers, our watchlist page has the dates.

Frequently Asked Questions

Which notable AI research paper of October 2026 has the most reliable headline number?

Scaling Discovery Requires Forgetting, from UC Berkeley, Stanford and Bespoke Labs. It compared methods on the same compute budget and reports up to 39% over a single agent and up to 15% over parallel agents. "Up to" marks the best of four tasks, so expect less in practice.

Is GitSwarm's 79.4% on ProgramBench a fair comparison?

It compares hundreds of agents sharing a Git repository against one Codex agent scoring 65.1%. The summary we used does not say compute was matched, so the 14.3-point gap reflects the method and the extra inference together.

Did LongHarness beat the Claude Code you can buy?

No such test was run. LongHarness beat Claude Code by 12.0 points on OfficeQA Pro V1 with both running on the same backbone model, not on Claude. It compares harnesses, and the figures are the authors' own in a preprint.