Godlike AI Is Coming, Whether We Pace The Frontier Or Not
It usually happens the same way: an Intelligence Explosion, of the type envisioned by I.J. Good - the smarter AI gets, the faster it's able to make itself even smarter, meaning that AI becomes godlike nearly instantaneously once it reaches a certain threshold of capability.
Almost every discussion about AI safety that I hear in San Francisco or read in the tech blogosphere centers around this sort of event.
The discussions are all about whether recursive self-improvement (RSI) is near, and whether we should“pace the frontier” - i.e., deliberately slow down or pause AI progress - to stop RSI from creating an intelligence explosion.
As I've listened to these arguments, I've grown increasingly skeptical that this is the right discussion we should be having. But before I explain why, let me quickly summarize the debate itself, to give an idea of what the arguments look like.
The intelligence explosion debate, in briefIn fact, I've published some pieces of that discussion on this blog. Back in September, Ramez Naam wrote a guest post arguing that an intelligence explosion is not imminent.
His basic arguments were:
-
OpenAI data show that AI is still only able to automate short chunks of the AI research process.
AI research is subject to diminishing returns, so that each increase in AI capability should buy less of a speedup in the pace of automated research.
Diminishing returns also mean that accelerating AI progress will require exponentially more of other resources, such as compute and money.
There's no sign yet of an acceleration in the rate of AI progress.
In a follow-up post, Ramez did a deeper dive on the topic of agent swarms, arguing that they're not as effective as many people believe.
Theo Jaffee echoed these arguments, noting that OpenAI's data shows that AI hasn't even been able to fully automate smaller research tasks yet:
Luis Garicano and Alex Imas also wrote a post downplaying the likelihood of an intelligence explosion.
Their arguments are more economic in nature. They show that although AI progress has accelerated, it also shows diminishing returns in terms of research spending, meaning that every unit of progress has come with a higher price tag:
They also estimate an economic model of RSI,[2] and conclude that ideas in AI research are probably getting harder to find, reducing the likelihood of an intelligence explosion.
On the other side of the debate, we have the recent METR report about Anthropic's internal progress in automating AI research. The report uses Anthropic's own internal measure of AI capability, AECI, rather than the better-known public ECI.
It also relies on nonpublic data, so deciding how much of the report to believe requires a lot of guesswork and assumptions. But the basic story METR tells is that Claude is already speeding up Anthropic's research process by a lot:
Dwarkesh Patel, meanwhile, argues that AI research isn't that different from math, and that the same factors that made AI suddenly go from useless to godlike at math will happen soon with AI's own self-improvement:
He writes:
So that's the debate. Basically, it depends on whether ideas get harder to find faster than AI gets better at finding ideas, and only time will tell which one wins.
But as I said, while this debate is intellectually interesting, I'm not sure how much it matters.
How useful is this debate?Whether an intelligence explosion happens is obviously very important. A sudden, unfathomable leap in AI capabilities to godlike levels would be, to put it mildly, a very big deal. But that doesn't mean an intelligence explosion is the thing we should be thinking and arguing about.
First of all, if an intelligence explosion is possible, it's going to happen, and there's probably just not much we can do to affect how it all shakes out.
Legal Disclaimer:
MENAFN provides the
information “as is” without warranty of any kind. We do not accept any
responsibility or liability for the accuracy, content, images, videos,
licenses, completeness, legality, or reliability of the information
contained in this article. If you have any complaints or copyright issues
related to this article, kindly contact the provider above.

Comments
No comment