GPT-5: Has AI Just Plateaued?

The main technical innovation behind GPT-5 seems to be the introduction of a“router”. This decides which model of GPT to delegate to when asked a question, essentially asking itself how much effort to invest in computing its answers (then improving over time by learning from feedback about its previous choices).

The options for delegation include the previous leading models of GPT and also a new“deeper reasoning” model called GPT-5 Thinking. It's not clear what this new model actually is. OpenAI isn't saying it is underpinned by any new algorithms or trained on any new data (since all available data was pretty much being used already).

One might therefore speculate that this model is really just another way of controlling existing models with repeated queries and pushing them to work harder until it produces better results.

It was back in 2017 when researchers at Google found out that a new type of AI architecture was capable of capturing tremendously complex patterns within long sequences of words that underpin the structure of human language.

By training these so-called large language models (LLMs) on large amounts of text, they could respond to prompts from a user by mapping a sequence of words to their most likely continuation in accordance with the patterns present in the dataset. This approach to mimicking human intelligence became better and better as LLMs were trained on larger and larger amounts of data – leading to systems like ChatGPT.

Ultimately, these models just encode a humongous table of stimuli and responses. A user prompt is the stimulus, and the model might just as well look it up in a table to determine the best response. Considering how simple this idea seems, it's astounding that LLMs have eclipsed the capabilities of many other AI systems – if not in terms of accuracy and reliability, certainly in terms of flexibility and usability.

The jury may still be out on whether these systems could ever be capable of true reasoning, or understanding the world in ways similar to ours, or keeping track of their experiences to refine their behaviour correctly – all arguably necessary ingredients of AGI.

In the meantime, an industry of AI software companies has sprung up that focuses on“taming” general purpose LLMs to be more reliable and predictable for specific use cases.

Having studied how to write the most effective prompts, their software might prompt a model multiple times, or use numerous LLMs, adjusting the instructions until it gets the desired result. In some cases, they might“fine-tune” an LLM with small-scale add-ons to make them more effective.

OpenAI's new router is in the same vein, except it's built into GPT-5. If this move succeeds, the engineers of companies further down the AI supply chain will be needed less and less. GPT-5 would also be cheaper to users than its LLM competitors because it would be more useful without these embellishments.

At the same time, this may well be an admission that we have reached a point where LLMs cannot be improved much further to deliver on the promise of AGI. If so, it will vindicate those scientists and industry experts who have been arguing for a while that it won't be possible to overcome the current limitations in AI without moving beyond LLM architectures.

Sign up for one of our free newsletters

• The Daily Report Start your day right with Asia Times' top stories
• AT Weekly Report A weekly roundup of Asia Times' most-read stories

Old wine into new models?

OpenAI's new emphasis on routing also harks back to the“meta reasoning” that gained prominence in AI in the 1990s, based on the idea of“reasoning about reasoning.” Imagine, for example, you were trying to calculate an optimal travel route on a complex map.

Heading off in the right direction is easy, but every time you consider another 100 alternatives for the remainder of the route, you will likely only get an improvement of 5% on your previous best option. At every point of the journey, the question is how much more thinking it's worth doing.

This kind of reasoning is important for dealing with complex tasks by breaking them down into smaller problems that can be solved with more specialized components. This was the predominant paradigm in AI until the focus shifted to general-purpose LLMs.

No more gold rush? Photo: JarTee via The Conversation

It is possible that the release of GPT-5 marks a shift in the evolution of AI which, even if it is not a return to this approach, might usher in the end of creating ever more complicated models whose thought processes are impossible for anyone to understand.

Whether that could put us on a path toward AGI is hard to say. But it might create an opportunity to move towards creating AIs we can control using rigorous engineering methods.

And it might help us remember that the original vision of AI was not only to replicate human intelligence, but also to better understand it.

Michael Rovatsos is professor of artificial intelligence, University of Edinburgh

This article is republished from The Conversation under a Creative Commons license. Read the original article .

Sign up here to comment on Asia Times stories Or Sign in to an existing accoun

Thank you for registering!

An account was already registered with this email. Please check your inbox for an authentication link.

• Click to share on X (Opens in new window) X
• Click to share on LinkedIn (Opens in new window) LinkedIn
• Click to share on Facebook (Opens in new window) Facebook
• Click to share on WhatsApp (Opens in new window) WhatsApp
• Click to share on Reddit (Opens in new window) Reddit
• Click to email a link to a friend (Opens in new window) Email
• Click to print (Opens in new window) Print