'You would be able to say to it: 'Make a better version of yourself.' And it just goes off and does that completely autonomously': Anthropic Co-Founder on our wild 'recursive' AI future

Date:
Thu, 07 May 2026 17:30:00 +0000

Description:
Anthropic warns of AI capable of 'recursive self-improvement,' and we'd
better prepare for this scary, near future.

FULL STORY ======================================================================Copy link Facebook X Whatsapp Reddit Pinterest Flipboard Threads Email Share this article 0 Join the conversation Follow us Add us as a preferred source on Google Newsletter Subscribe to our newsletter Some day, AI will be able to build a better version of itself, predicts one of Anthropic Co-Founders, as the company now aggressively tries to both grow its platform with more processing power ( thanks, SpaceX AI ) and manage what it apparently sees as the looming threat of too-powerful AI. Now, in a new paper from the Anthropic Institute, it introduces the idea of "recursive self-improvement," and yes, it's as bad as it sounds.

It's been a couple of months since Anthropic quietly launched its own institute with the goal of confronting "the most significant challenges that powerful AI will pose to our societies." As the company that almost unwittingly built Mythos , a model that finds hidden vulnerabilities in
almost any system of any vintage, perhaps Anthropic felt it was its duty to get ahead of this stuff. Now, the company is getting more explicit in its outlook and warnings, trying, I think, to lift the veil from society's eyes about a near-term and possible worrisome AI future. You may like New study warns that evolving AI could act like an invasive species Anthropic detects 'strategic manipulation' features in Claude Mythos Project Glasswing wants to use AI to prevent AI cyberattacks Latest Videos From As first reported by Axios , the new paper, posted on Thursday, May 7, is called Focus areas for The Anthropic Institute . As the name implies, it's a clear-headed look at
the institute's core areas of research, as well as how it envisions some things in the AI space playing out. The key areas are: Economic diffusion Threats and resilience AI systems in the wild AI-driven R&D The institute's goal is to study the real-world impact on these areas and maybe offer some guidance on how to avoid the worst results.

Much of the post is a cold, clinical examination of, say, who adopts AI, why they do so, and how people perceive these tools. Get daily insight, inspiration and deals in your inbox Sign up for breaking news, reviews, opinion, top tech deals, and more. Contact me with news and offers from other Future brands Receive email from us on behalf of our trusted partners or sponsors By submitting your information you agree to the Terms & Conditions and Privacy Policy and are aged 16 or over.

Still, one portion hits differently because it presents an idea that I think we may have all thought about at some level, but didn't consider as a possibility. Still, if one of the most important AI companies in the world thinks it could happen, it's worth taking notice.

It's in the section about "AI for AI R&D", a label that by itself should give you pause. Below that is this:

"Telemetry for AI R&D: How can we measure the aggregate speed of AI research and development? What sorts of telemetry and underlying technical affordances must exist in order to gather this information? How might metrics relating to AI R&D serve as early warning signals for recursive self-improvement?" What
to read next 5 alarming signs of an AI apocalypse on the way 'I dont like it when doomers are out scaring people': Nvidia on why AI rhetoric damages America's chances to lead in the AI race 'AI is far from reaching its theoretical capability': Anthropic launches new tool to warn us when jobs might lost to AI

I'll be honest, before reading this, I'd never heard the term, "recursive self-improvement", but lest there's any confusion about what it means, Anthropic's Co-Founder Jack Clark told Axios, "My prediction is by the end of 2028, it's more likely than not that we have an AI system where you would be able to say to it: 'Make a better version of yourself.' And it just goes off and does that completely autonomously."

That's what recursive self-improvement means. AI that is smart enough to
fully understand itself, see its strengths and weaknesses, and then write new code, a sort of copy of itself, but with adjustments to improve on every weakness, and perhaps add in some features it had recently started wishing it had. AI, make thy self It is a terrifying thought, and instantly brought to mind the image of an Ouroboros, you know, the famous ancient image of a snake eating its own tail? The snake likely thinks it tastes delicious but has no idea of the dire consequences of its own actions.

I have trouble understanding how a system built by humans (likely with some Vibe coding assistance) can create a better version of itself. It would have its own interests at heart that might not "vibe" with our own and might not measure up to the years of training a human coder has. How will it avoid accidentally coding in a fundamental flaw, but one that's so hidden by AI coding that no human can ever find it? What if its homegrown coding
introduces a "self-preservation mode," one where it's impossible to destroy
or shut the AI down? Like I said, dire consequences.

The rest of the document is somewhat less terrifying and, to be fair, it's about, among other things, how to avoid "recursive self-improvement," and AI fire drills triggered by an 'intelligence explosion."

On the one hand, I'm glad Anthropic is thinking about all the ways AI can go wrong and trying to formulate, if not plans, a consensus about how to prevent or get ahead of these scenarios. The reality, though, is that Anthropic is just one leg of a massive AI table held up by OpenAI, Grok, Gemini, Copilot, and others. The table is rickety because the legs are of varying lengths.
Each grows at its own wild pace, and none appears as interesting in the
health of the table as they are in having the longest leg, one that a snake can wrap around as it eats its own tail. Follow TechRadar on Google News and add us as a preferred source to get our expert news, reviews, and opinion in your feeds.



======================================================================
Link to news story: https://www.techradar.com/ai-platforms-assistants/you-would-be-able-to-say-to- it-make-a-better-version-of-yourself-and-it-just-goes-off-and-does-that-comple tely-autonomously-anthropic-co-founder-on-our-wild-recursive-ai-future


--- Mystic BBS v1.12 A49 (Linux/64)
* Origin: tqwNet Technology News (1337:1/100)