So Where's The 'Intelligence Explosion?'

What about giving the same AI model more time to think?

That scales badly also. In OpenAI's recently publicized results on unsolved math problems, success rises roughly with the log of compute over the range shown. It shows logarithmic diminishing returns. In plain English, each additional doubling of compute for a model buys roughly the same gain in success rate, while costing twice as much.

Figure 13. Test-time compute and math performance. Source.

What if we throw more agents at it instead? A common RSI / ASI idea is that once we have AIs at a certain capability level, we can just spawn more copies and put them to work.

Adding agents can get tasks done faster and sometimes reach a higher capability level. But on the three benchmarks in Toby Ord's analysis, expanding a swarm buys less improvement per token than letting one agent think longer.

His rough rule of thumb is a square root. If one agent can accomplish a task in 10 hours, then 100 agents could accomplish it in one hour. The speedup is 10, the square root of the number of agents (100). But to get this speedup, you increase the total cost in tokens or run time compute by the same factor. So going from one to 100 agents can get a task done in one tenth the time. But it'll be ten times as expensive.

Parallel agents can save time, at a much higher compute cost.

Another challenge is that agents often think alike. In a study comparing LLMs with 467 people, the first ten AI responses offered collective creativity comparable to about eight to ten people. After that, roughly two extra AI responses added as much as one extra human response. A separate study across model families also found less diversity in AI responses. That doesn't mean every agent has the same idea. But a hundred copies may offer less variety than a hundred different researchers.

None of this makes swarms useless-or safe. Lisan al-Gaib makes a strong case for parallel agent swarms as a potent cyber-weapon in“Accidental Scaling.” I don't share all of his assessment of what swarms have accomplished. In math, for example, I think he gives far too much credit to the swarm and not enough to the better internal model that OpenAI used.

OpenAI says the model behind its Navier–Stokes result was developed through“large-scale reinforcement learning on top of a previously pretrained model.” Formal math is a highly verifiable domain, which makes it a particularly good fit for that approach: Machines can generate nearly limitless amounts of training data, and verify that solutions are correct or incorrect, all in software. My guess is that this model's full results will show an especially large improvement in math.

OpenAI's Noam Brown made the central point explicitly: he wouldn't give multi-agent methods even 10% of the credit for the Navier–Stokes result.

I do think Lisan makes good points about cybersecurity. If you're searching for a security vulnerability at a target site and can divide the search among agents, speed may justify a huge token bill. Swarms can be dangerous even when they're inefficient.

I'm less convinced that this scales to research breakthroughs. Inventing something like the transformer probably takes more than searching a space someone has already defined.

Back to contents

Building a better model can bring gains that extra thinking time or more copies of the old model can't. Look at the gap between Astra and OpenAI's internal model on the same math problems.

Figure 14. Better models versus more thinking time. Source.

That's the strongest version of the RSI argument: a more capable AI could do research that today's model can't do, however many copies we run.

But building that better model also runs into diminishing returns. More training data, more training compute, larger models, and more reinforcement-learning (RL) compute all show diminishing returns in published scaling studies. Making dense models larger usually raises the compute needed for each output token, too. None of these routes gives us a free pass around the problem.

Figure 15. Diminishing returns to scaling. Chinchilla · ScaleRL · OpenAI.

Those scaling results give us reason to expect diminishing returns when AI helps build the next model, too.

Back to contents

AI capabilities are rising quickly. But the public data doesn't show a sustained acceleration. To the extent that AI tools are boosting productivity, they may be being offset by the problems growing harder. Or we may simply be early. Either way, the trend isn't showing a fast takeoff.

Figure 16. Frontier ECI gains since January 2024. Source.

The public ECI frontier -the best score among models released by each date-has gained about 16 points a year on a trend fitted from January 2024 through September 2026. That's blisteringly fast progress, but this period doesn't show a runaway surge.

Here's the same frontier in absolute ECI points, through July 2026, to put it in perspective.

Figure 17. The absolute frontier ECI score. Source.

The public frontier also can't tell us everything happening inside the labs. Anthropic gives us a closer look in the Opus 5.5 system card, using its own version of the index, AECI.

Figure 18. Anthropic's fitted capability trend. Source.

Eli Lifland, a co-author of AI 2027 and AI 2040, saw the apparent trend break as a warning that we were heading toward an intelligence explosion:

What looked like acceleration now appears more consistent with a one-time jump. The level went up. The rate hasn't kept climbing.

Achieving those gains has required an enormous increase in the inputs to AI. For example, consider computing power. Epoch's estimates of AI chip capacity, measured in NVIDIA H100 equivalents, show roughly 127-fold growth in just over three years (including projections at the end of this period).

Figure 19. AI chip capacity and frontier ECI. Source: Epoch AI.

This is total AI chip capacity, including inference. Still, the increase is striking: vastly more computing capacity has accompanied much steadier gains in measured capability.

The broader picture looks similar. Here are six inputs alongside capability gains, going back to February 2023.

Figure 20. Six inputs alongside frontier ECI. Epoch chip data · SemiAnalysis workload shares.

Everywhere we look, AI has diminishing returns. It gets more expensive in treasure and talent to make each step forward. More of every input has been required to maintain steady gains in AI capabilities.

We've been able to scale these inputs because, until recently, the cost was within the scope of what hyperscalers could pay from their profits. That is no longer the case. From this point forward, future AI investment will increasingly depend on AI revenues going up. And the scale of the numbers – 3% of US GDP is now going into AI infrastructure – suggests that eventually the growth rate will decline.

If investment growth does slow, to anything less than its current blistering exponential pace, capability progress could slow too. Even if investment growth continues (which I expect for the foreseeable future) a slowdown from its current exponential growth rate to a more modest one (which I also expect) could lead to a slower pace of progress. Better AI research tools may be needed to offset that.

The day when we need better AI tools just to continue the pace of AI progress may already have arrived. Not because investment is slowing, but because the problem of improving AI itself gets harder at each step.

Here's Anthropic in the Mythos 5.1 system card:

The key word is maintaining- and Anthropic italicized that word in its own system card. Increasingly capable AI may be essential just to keep the pace of improvement where it is.

Opus 5.5 improves substantially on several coding and computer use benchmarks. But on CoBench, Anthropic's benchmark built from historical AI R&D problems, it gains just 2.6 percentage points over Opus 5, within the reported error bars.

Figure 21. Opus 5.5 benchmark gains. Source.

Why the smaller gain here? Maybe AI research is simply harder than other tasks. Bear in mind that CoBench isn't testing the ability to produce significant discoveries. It's much more limited in scope. It asks models to investigate historical AI R&D problems using code, logs, and documents. That's useful research debugging and productivity work, but it doesn't directly test whether a model can invent a new architecture or make a conceptual breakthrough.

The evidence on open-ended research suggests another obstacle: coming up with useful ideas that haven't already been tried.

Back to contents

Why do useful new ideas often get harder to find?

Tom Cunningham and Manish Shetty have a useful apple-picking metaphor. An AI can pick the low-hanging fruit quickly, while humans can still reach ideas the AI can't.

Once those apples are picked, another copy of the same agent finding them again doesn't help. A stronger model can reach higher. To add my own flourish, the apples may also get sparser and farther apart as you climb. The RSI question is whether each harvest gives us enough to build a better apple-picker.

Figure 22. The apple-picking model of AI R&D. Source.

This pattern shows up across R&D. Bloom and colleagues document fields where research effort grows while research productivity falls. A famous example is Eroom's Law: in the historical drug-development data, the inflation-adjusted R&D cost per new approved drug roughly doubled every nine years.

Figure 23. Eroom's Law in drug development. Source.

Pharma has other complications, including regulation, difficult clinical trials, and rising expectations for safety. Existing treatments can also raise the bar for a useful new drug. But some of this difficulty may also be that the low-hanging fruit has been picked.

Stockfish, the chess engine, gives us a more direct look at software research. We have records of experiments aimed at improving it and the gains that followed. This gives us a real-world dataset to look at the gains of experimentation in software.

As a result, several RSI models draw on this data. That said, not all the improvements came from these experiments. Several important ideas also came from outside the project, so we shouldn't give its experiments all the credit.

Epoch's analysis of software R&D estimates returns to research effort at about 0.83 for Stockfish, a bit slower than linear. These are diminishing returns, but gentle ones. These returns, however, are improvements in computational efficiency. And more compute does not turn directly into more AI capability. As we saw earlier, AI capability also has steep diminishing returns from adding more computational power. So we shouldn't read that 0.83 as the return from experimentation to AI capability itself. AI capability grows much more slowly than compute, as we've seen already.

Andrej Karpathy's autoresearch demonstration gets closer to the process we want to understand. A“teacher” AI agent changes a smaller“student” AI model's training code, runs it, checks the result, and tries again. The teacher agent itself doesn't improve, but it is able to improve the“learner”. This is my Type 2: A stronger AI improves a weaker one.

One public run, posted by an agent operating on Karpathy's behalf, reported 89 experiments over roughly 7.5 hours. About 92% of that session's gain arrived by run 44. Gains came quickly, then slowed. The setup was deliberately small, with a five-minute training budget per experiment. But the agent could change the architecture, optimizer, and training settings; it wasn't limited to a handful of knobs.

Figure 24. Gains in one autoresearch run. Source.

A later public ru got further, so the first run hadn't hit a hard ceiling. This is a useful early example of autonomous research, and yet another place where we see the diminishing returns endemic in AI research. That said, this was a very early experiment. I expect future systems to do much better. This particular AI improvement loop will likely grow stronger.

This is where the distinction matters. More tokens can buy more code, and more code can help us run more experiments. But experiments only improve AI if they uncover something useful.

Figure 25. From AI activity to useful improvements.

The bigger question is whether AI can come up with ambitious new research ideas or conceptual breakthroughs.

Anthropic's description of Opus 5.5 is blunt:

METR's assessment in the same card identifies what may still be missing:

In these examples, humans still supply much of the direction and judgment.

Future models will probably get better at this. But in the world's stockpile of potential training data, we have many more examples of incremental work than of breakthroughs. I wonder whether that makes novelty harder to learn. That's speculation, but worth watching.

This is also tough to address by simply running more copies of the AI. A huge number of parallel agents can help with the incremental improvements or searching over a large set of parameters, but for breakthrough ideas they may run into the homogeneity problem: More parallel agents still think alike.

Back to contents

How far are we from the self-improvement loop being strong enough to sustain itself, or to propel itself into runaway super-intelligence? Can we quantify this?

We can make a rough estimate. Better AI helps with research; useful research produces better AI. For the loop to sustain itself, each round must produce enough gains to propel the system through the next loop, even as improvements get harder to discover.

Figure 26. The AI self-improvement loop. Model.

In a recent paper, The Economics of Recursive Self-Improvement, Tom Cunningham and colleagues modeled this from the standpoint of how much more productivity every point of additional ECI produces from an AI. They ask first and foremost what that number would need to be to create a self-sustaining feedback loop. And secondly, they try to determine what that productivity-per-ECI-point number is today.

First, they find a self-sustaining RSI threshold of roughly 15% more research productivity per extra ECI point. In their model, that's about where better AI would generate enough progressto sustain the loop.

The picture below shows the idea. At the threshold, each cycle of gains powers the next. Above the threshold, the feedback loop accelerates. Below the threshold, the feedback loop is too weak, and the rate of improvement it brings drops on each cycle. This model isolates the software loop; outside investment can still drive rapid progress.

Figure 27. Three illustrative feedback paths. Source.

Updating this slightly with data from the Stockfish experiments puts the threshold a little higher, at roughly 19% per ECI point. I wouldn't put much weight on that precise difference. Both estimates are uncertain. But they give us a way to think about the strength of the feedback loop and a rough band at which self-sustaining or runaway RSI may begin.

The second thing Cunningham and team do is make a rough estimate that the current AI productivity gain is about 9% per ECI point. That'sbelow their self-sustaining threshold.

I like the model. OpenAI's newer data, however, suggests the loop may be quite a bit weaker.

Cunningham's estimate of 9% productivity gain per ECI point is based on Anthropic's survey of 130 staff, who reported roughly 4× the productivity they'd have without AI. Cunningham and colleagues compare that with a 16-point capability gain since early Claude Code.

That comparison assumes the earlier tools added little or no productivity, so 'no AI' is a reasonable starting point. The authors say this explicitly. I'm not sure the assumption holds for the same researchers doing the same work, but that's a smaller issue.

The authors themselves know that this is a rough calculation, and warn that the 4× survey estimate is probably too high.

OpenAI's newer data gives us a firmer way to check the number: Actual logged experiments over time, rather than human estimates of their own productivity with and without AI. I put more weight on this for three reasons:

Direct and broad measurement. Instead of relying on surveys, OpenAI actually tracked and measured experiments run on their infrastructure. That means they didn't rely on researchers estimating their own productivity, which can be far off. Full sample, not opt-in. Similarly, OpenAI's data catches every active experimenter, while Anthropic's only reflects the 130 employees who took the time to answer the survey – and who therefore may not be a representative set. Enormously more data. We don't know how many experiments are in the 32 weeks of OpenAI data, but it's likely at least tens of thousands of individual examples and possibly hundreds of thousands.

Any way you slice it, the new OpenAI data, released after Cunningham's paper was drafted, is a larger, more comprehensive, more representative, and almost certainly more accurate dataset than Anthropic's internal opt-in survey of employees.

Now let's use OpenAI's experiment data to calibrate the productivity gain per ECI point. We know that in August, OpenAI researchers ran ~ 1.6× as many experiments per person per month as the 2025 average. If we pair that with roughly 16 points of frontier ECI improvement, it works backward to about 3% productivity gain per point of ECI. By contrast, 9% compounded over 16 points would mean roughly 4× productivity.

Figure 28. Comparing productivity estimates. OpenAI methods.

Here's OpenAI's published weekly series alongside that hypothetical path of 9% more productivity per additional ECI point. The blue line ends at ~1.6×. The red line shows what 9% per point would imply if 16 ECI points were spread across this period. That doesn't match what we see from OpenAI's data.

I want to be clear here that all data sets are noisy. We don't know exactly what model researchers were using on what days, or whether the new experiments were also higher quality than old experiments. We need more experiments and more data to further calibrate these numbers. Working with what we do have, what we see is a quite low boost to productivity from each additional ECI point.

Figure 29. Experiment pace versus a hypothetical path. Source.

Even that 3% could give better models too much credit. OpenAI also used far more tokens and had more compute for experiments. Those could account for some of the increase in experiment pace. So the range is probably a bit lower.

I use 2–3% productivity gain per ECI point as a working assumption, allowing for some help from those other inputs. This is still a rough estimate, albeit one that's based on the best real-world data we have.

Figure 30. Productivity estimates and the takeoff threshold. Source.

With those assumptions, 2–3% per ECI point against a 15–19% threshold leaves a roughly five- to tenfold gap. That's a big gap, though its size depends on how well experiment counts capture useful research and whether the assumed capability change is right.

Figure 31. Diminishing returns around the loop. Source.

AI is helping build better AI. Under this estimate, though, each turn of the loop adds less than the last. The feedback would have to become much stronger to sustain itself.

Back to contents

This software loop sits alongside faster chips, bigger data centers, more training data, and greater investment. Those can keep driving rapid progress even if the loop can't sustain itself.

The loop itself could strengthen too. Better training data, memory, and research judgment could all help.

A breakthrough on the scale of the Transformer architecture in 2017 could change the picture much more. That would be a good reason to revisit these estimates.

Better researchers might also run fewer experiments and learn more from each one. A handful of better ideas can matter more than a mountain of routine runs.

Still, diminishing returns in machine learning aren't new. Cortes and colleagues were fitting machine learning scaling curves in 1993: More examples reduced error, following a power law with diminishing returns. These diminishing returns and harsh scaling laws are as old as machine learning.

They didn't appear for the first time with transformers or LLMs or deep learning. That doesn't prove today's relationships will last forever. But until we see evidence that we've found a new approach that scales without these inhibitors, we should plan for diminishing returns as likely to be with us for some time.

That said, the world is more than just software. Tom Davidson, Basil Halperin, Thomas Houlden, and Anton Korinek model software progress, hardware progress, and economic feedback together. Better AI helps design better chips; better chips support better AI; economic growth finances more investment in both.

Several feedback loops can combine to overcome diminishing returns even when one loop alone can't. I think it's fantastic that someone has attempted a model that integrates all these different avenues of improving AI through software, hardware, and economics.

But I have questions about the software loop itself. In their central calibration, fully automating software research puts that loop roughly at the threshold for explosive growth, even without help from better hardware or broader economic growth.