'Godfather Of AI' Now Fears It's Unsafe And Has A Plan To Fix It

According to a preprint paper released online earlier this year, Scientist AI will differ from current AI systems in two key ways.

First, it can assess and communicate its confidence level in its answers, helping to reduce the problem of AI giving overly confident and incorrect responses. Second, it can explain its reasoning to humans, allowing its conclusions to be evaluated and tested for accuracy.

Interestingly, older AI systems had this feature . But in the rush for speed and new approaches, many modern AI models can't explain their decisions. Their developers have sacrificed explainability for speed.

Bengio also intends“Scientist AI” to act as a guardrail against unsafe AI. It could monitor other, less reliable and harmful AI systems - essentially fighting fire with fire.

This may be the only viable solution to improve AI safety. Humans cannot properly monitor systems such as ChatGPT, which handle over a billion queries daily. Only another AI can manage this scale.

Using an AI system against other AI systems is not just a sci-fi concept – it's a common practice in research to compare and test different levels of intelligence in AI systems .

Large language models and machine learning are just small parts of today's AI landscape.

Another key addition Bengio's team is adding to Scientist AI is the“world model,” which brings certainty and explainability. Just as humans make decisions based on their understanding of the world, AI needs a similar model to function effectively.

The absence of a world model in current AI models is clear.

One well-known example is the“hand problem”: most of today's AI models can imitate the appearance of hands but cannot replicate natural hand movements, because they lack an understanding of the physics - a world model - behind them.

Another example is how models such as ChatGPT struggle with chess, failing to win and even making illegal moves .

This is despite simpler AI systems, which do contain a model of the“world” of chess, beating even the best human players . These issues stem from the lack of a foundational world model in these systems, which are not inherently designed to model the dynamics of the real world .

Bengio is on the right track, aiming to build safer, more trustworthy AI by combining large language models with other AI technologies.

However, his journey isn't going to be easy. LawZero's US$30 million in funding is small compared to efforts such as the US$500 billion project announced by US President Donald Trump earlier this year to accelerate the development of AI.

Sign up for one of our free newsletters

• The Daily Report Start your day right with Asia Times' top stories
• AT Weekly Report A weekly roundup of Asia Times' most-read stories

Making LawZero's task harder is the fact that Scientist AI – like any other AI project – needs huge amounts of data to be powerful, and most data are controlled by major tech companies .

There's also an outstanding question. Even if Bengio can build an AI system that does everything he says it can, how is it going to be able to control other systems that might be causing harm?

Still, this project, with talented researchers behind it, could spark a movement toward a future where AI truly helps humans thrive. If successful, it could set new expectations for safe AI, motivating researchers, developers, and policymakers to prioritize safety.

Perhaps if we had taken similar action when social media first emerged, we would have a safer online environment for young people's mental health. And maybe, if Scientist AI had already been in place, it could have prevented people with harmful intentions from accessing dangerous information with the help of AI systems.

Armin Chitizadeh is lecturer, School of Computer Science, University of Sydney

This article is republished from The Conversation under a Creative Commons license. Read the original article .

Sign up here to comment on Asia Times stories Or Sign in to an existing accoun

Thank you for registering!

An account was already registered with this email. Please check your inbox for an authentication link.

• Click to share on X (Opens in new window) X
• Click to share on LinkedIn (Opens in new window) LinkedIn
• Click to share on Facebook (Opens in new window) Facebook
• Click to share on WhatsApp (Opens in new window) WhatsApp
• Click to share on Reddit (Opens in new window) Reddit
• Click to email a link to a friend (Opens in new window) Email
• Click to print (Opens in new window) Print