How AI Got Better at Building Itself (Economist)

Summary

AI labs are edging toward Recursive Self-Improvement (RSI), the closed loop in which one model builds a more capable successor with no human in the loop. Anthropic co-founder Jack Clark puts a 60% chance on an AI being able to create its own successor unaided by end-2028, and Anthropic — whose own code is now >80% Claude-written — has called for an option to pause frontier development. Early signals already exist: Google DeepMind's AlphaEvolve designs novel algorithms, and Andrej Karpathy's Auto Research Loop (Karpathy) agent autonomously shaved Nanochat's training time by 18%. The piece weighs RSI's promise against physical limits (compute, data) and the safety fears of researchers like Max Tegmark.

Key claims

• More than four-fifths of the code Anthropic published in May was written by Claude, up from "low single-digits" before Claude Code launched in February 2025; a METR (AI Evaluation Think-Tank) benchmark shows its models went from ~1-hour human-engineer tasks (early 2025) to >1 working day.
• Jack Clark estimates a 60% chance that by end-2028 an AI can build its own successor with zero human involvement — the trigger for Recursive Self-Improvement (a "fast take-off" or "going foom").
• Google DeepMind's AlphaEvolve (May 2025) proposed a data-centre workload change saving 0.7% of Google's worldwide compute and improved matrix multiplication, speeding Gemini training by 1%.
• Andrej Karpathy trained a GPT-2-class model (Nanochat) on 8 GPUs in ~3 hours vs GPT-2's original 168 hours on 32 chips; his Auto Research Loop (Karpathy) agent then cut training to 1h39m unaided — an 18% gain — via prosaic fixes (better init values, wider attention window, fixing wandering focus).
• A January CSET (Georgetown) report warns AI-performed R&D could boost productivity 10x, 100x, then 1,000x over human-only R&D, overcoming bottlenecks faster than they appear, and that RSI systems "pose extreme risks."
• Physical brakes remain: compute access (progress paces data-centre build-out), consumer demand competing for capacity, and the limits of "verifiable rewards" — synthetic data works for code/maths but not creative writing or legal judgment.

Cross-vault relevance

tip Strong connection to a tracked AI theme This article is a direct, high-value hit on Recursive Self-Improvement and Self-Evolving Agents — themes already tracked in this brain. The Auto Research Loop (Karpathy) case study (an agent autonomously improving its own training pipeline by 18%) is a concrete, recent data point for RSI, and pairs naturally with MLEvolve (Self-Evolving ML Algorithm Discovery) and AlphaEvolve as examples of AI discovering its own improvements. The piece's framing — "models trained by models, to achieve goals set by models, whose safety is verified only by models" — is a sharp articulation of the Agentic Loop taken to its endpoint, and worth weaving into any synthesis on self-improving AI or AI-safety governance.

Wikilinks

Recursive Self-Improvement · Self-Evolving Agents · MLEvolve (Self-Evolving ML Algorithm Discovery) · Auto Research Loop (Karpathy) · Agentic Loop · Anthropic · OpenAI · Claude Code · Andrej Karpathy · AlphaEvolve · METR (AI Evaluation Think-Tank) · CSET (Center for Security and Emerging Technology) · Max Tegmark · Reflection AI

The Economist — Science & technology.