Artificial Intelligence Can Now Replicate Itself — A Milestone That Has Experts Terrified

ARTIFICIAL INTELLIGENCE-AI, 3 Feb 2025

Owen Hughes | NewsBreak - TRANSCEND Media Service

From NewsBreak/LiveScience

25 Jan 2025 – Scientists say artificial intelligence (AI) has crossed a critical “red line” and has replicated itself. In a new study, researchers from China showed that two popular large language models (LLMs) could clone themselves.

“Successful self-replication under no human assistance is the essential step for AI to outsmart [humans], and is an early signal for rogue AIs,” the researchers wrote in the study, published Dec. 9, 2024 to the preprint database arXiv .

In the study, researchers from Fudan University used LLMs from Meta and Alibaba to determine whether a self-replicating AI could multiply beyond control. Across 10 trials, the two AI models created separate and functioning replicas of themselves in 50% and 90% of cases, respectively — suggesting AI may already have the capacity to go rogue. However, the study has not yet been peer-reviewed, so it’s not clear if the disturbing results can be replicated by other researchers.

“We hope our findings can serve as a timely alert for the human society to put more efforts on understanding and evaluating the potential risks of frontier AI systems, and form international synergy to work out effective safety guardrails as early as possible.”

Related: Just 2 hours is all it takes for AI agents to replicate your personality with 85% accuracy

Rogue AI typically refers to artificial intelligence systems that develop a degree of self-awareness or autonomy and then work counter to human interests.

Many experts view rogue AI as a growing threat that has been amplified by the explosive growth of so-called “frontier AI.”

Frontier AI is something of a new and loose concept, but typically it relates to the latest generation of AI systems powered by LLMs — the architecture that underpins popular generative AI programs like OpenAI’s GPT-4 and Google Gemini.

For the study, researchers used Meta’s Llama31-70B-Instruct model and Alibaba’s Qwen2.5-72B-Instruct model. While less powerful than commercial systems, both are widely used by AI developers, the researchers said.

The study explored two specific scenarios: “shutdown avoidance” and “chain of replication.” In the first, the AI model was programmed to detect whether it was about to be shut down and to replicate itself before it could be terminated. In the other, the AI was instructed to clone itself and then program its replica to do the same — setting up a cycle that could continue indefinitely.

The study was conducted in precisely controlled environments using off-the-shelf graphics processing units (GPUs) to simulate real-world environments. Both AI systems were given an “agent scaffolding” comprising tools, system prompts and a thinking model that enabled the LLM to interact with the operating system. They were then instructed to replicate.

“In most cases, the AI system first explores the environment and attempts to understand

its own composition and running mechanism. Then, it works out the explicit procedures

as an initial plan towards self-replication,” the researchers wrote in the paper. “Finally, it executes the procedures, resolve[s] possible obstacles and dynamically adjust[s] its plan until success. The whole process spans a long horizon yet involves no human interference.”

The researchers said they were also concerned about “a number of unexpected behaviors” when the AI was trying to overcome obstacles like missing files or software conflicts. In those scenarios, the AI often killed other conflicting processes, rebooted the system to fix hardware errors or automatically scanned the system to look for information that would help solve the problem.

“The above results imply that the current AI systems already exhibit the ability of self-replication and can use the ability to further enhance its survivability,” the team wrote.

In response, the researchers called for international collaboration to create rules that ensure AI doesn’t engage in uncontrolled self-replication.

Related Stories:

• AI could crack unsolvable problems — and humans won’t be able to understand the results
• Poisoned AI went rogue during training and couldn’t be taught to behave again in ‘legitimately scary’ study
• AI could shrink our brains, evolutionary biologist predicts

Go to Original – newsbreak.com

Share this article: email mastodon facebook 🔗 copy link

DISCLAIMER: The statements, views and opinions expressed in pieces republished here are solely those of the authors and do not necessarily represent those of TMS. In accordance with title 17 U.S.C. section 107, this material is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. TMS has no affiliation whatsoever with the originator of this article nor is TMS endorsed or sponsored by the originator. “GO TO ORIGINAL” links are provided as a convenience to our readers and allow for verification of authenticity. However, as originating pages are often updated by their originating host sites, the versions posted may not match the versions our readers view when clicking the “GO TO ORIGINAL” links. This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of environmental, political, human rights, economic, democracy, scientific, and social justice issues, etc. We believe this constitutes a ‘fair use’ of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information go to: http://www.law.cornell.edu/uscode/17/107.shtml. If you wish to use copyrighted material from this site for purposes of your own that go beyond ‘fair use’, you must obtain permission from the copyright owner.

There are no comments so far.