Survey paper

Towards AI That Improves Itself

A Survey of Recursive Self-Improvement

Hanjing Li, Qiguang Chen, Chenyuan Zhang, Qionglin Qiu, Fanqing Meng, Mengkang Hu, Libo Qin, Min Zhang

Harbin Institute of Technology (Shenzhen) University of Hong Kong National University of Singapore Shanghai Innovation Institute

Read PDF

Overview diagram of recursive self-improvement scope and boundaries — RSI moves from local revision toward persistent improvement loops across behavior, agent systems, models, data, and research processes.

3-stage loop Proposal -> Feedback -> Optimization

4 evolution targets Behavior, agents, model/data, research

7 frontier dimensions Reliability through generalization

Abstract

From single-step revision to verifiable improvement loops

Recent advances in agentic systems allow AI systems to conduct experiments, interpret feedback, and modify their own workflows, transforming Recursive Self-Improvement from a theoretical concept into an engineering problem. Yet the field still lacks a unified account of what constitutes RSI and how its improvement loop should be evaluated.

This survey defines RSI boundaries, distinguishes four stages (evolution, self-evolution, meta-evolution, and recursive self-improvement) and anchors recursion in state inheritance: an accepted improvement from one cycle becomes the starting state of a later cycle, while “recursive progress” is reserved for the stronger claim that the inherited state also improves the improver. It introduces a common Proposal → Feedback → Optimization loop, organizes methods across four evolution targets, and identifies research gaps around evidence-based acceptance, reuse across cycles, and transfer beyond the tasks or evaluators that produced the gains.

Definition

A common loop for modern RSI systems

RSI is anchored in state inheritance: an improvement accepted in one cycle is retained as the starting state of a later cycle. Recursive progress (the stronger claim that the inherited state also improves the improver) is treated separately.

Proposal

Select an editable target and generate a candidate change through prompting, search, evolution, program synthesis, training, or experimentation.

Target selection identifies what the loop is allowed to change.
Candidate generation creates concrete artifacts such as prompts, tools, memory, code, data, or checkpoints.

Feedback

Expose consequences in a repository, simulator, benchmark, training run, or live environment, then convert outcomes into evidence.

Execution environments reveal behavioral effects of a candidate.
Verification signals judge the evidence through tests, rewards, model judges, formal checks, or review.

Optimization

Retain, revise, reject, promote, or roll back the candidate while preserving memory, provenance, versioning, and validation evidence.

Update policies decide whether an apparent gain should persist.
Memory and lineage determine whether later cycles can reuse or audit the change.

Definition and characterization framework for recursive self-improvement

System Structure

What changes, how it is tested, and what persists

Across the systems we survey, the same anatomy recurs: a target is edited, the edit is exposed to an environment that produces evidence, and a decision layer retains or discards it. Six components organize these concerns into three stages: what can be modified, how it is tested, and what persists.

Proposal

Choose what to change, then generate a candidate through prompting, search, evolution, synthesis, or training.

Feedback

Run the change and turn its consequences into evidence via reference, rubric, and judge.

Optimization

Decide what persists and what is saved, tracked, and reused across cycles.

System structure of modern recursive self-improvement

Taxonomy

Four targets of RSI-related evolution

The four targets form a progression by persistence: behavior-level gains last for a single trajectory, agent-level changes outlive the current run, model-and-data changes transfer across tasks, and research-process evolution aims to improve the loop that produces improvements itself.

Behavior Evolution

Output self-refine, test-time evolution, and state optimization improve behavior around the current task or trajectory.

Output self-refine
Test-time evolution
State optimization

Agent-System Evolution

Workflow, memory, code rewriting, and harness evolution change the machinery that produces future agent behavior.

Workflow evolution
Memory evolution
Code rewriting and harness evolution

Model and Data Evolution

Synthetic data, meta-rewarding, post-training, and self-play alter the learning signal or model parameters themselves.

Synthetic data
Meta-rewarding and post-training
Self-play evolution

Research-Process Evolution

Scientific operation agents, AI4AI, MLE, and end-to-end AI scientist systems automate the research loop around AI improvement.

Scientific operation agents
AI4AI and MLE
End-to-end AI scientist systems

An evidence ladder: from local repair to compounding

Self-improvement claims differ in strength, and the ladder orders them by the evidence each one needs. Local repair fixes the current output and stops there; persistent transfer makes a change outlive the current run and help later instances; recursive inheritance (the minimal RSI claim) makes an accepted improvement the starting state of a later cycle; and recursive progress further shows that the inherited state improves the improver itself. Causal attribution supplies the supporting evidence (a counterfactual control against a frozen or no-retention baseline) that a retained intervention, not an unrelated confound, produced the gain; compounding is the claim that recursive-progress gains repeat across cycles.

Evidence ladder for distinguishing local repair from recursive self-improvement

Future Directions

An evidence-oriented evaluation profile

The seven properties below form a single evidence-oriented evaluation profile rather than seven independent capabilities. The organizing test is acceptance, reuse, and transfer: an accepted change must be credible, usable in later cycles, and helpful beyond the evaluator that admitted it.

ReliabilityDoes the change improve the intended capability, not just the measured score?

False positives can be written into memory, skills, or release pipelines; evaluations should report exploit resistance, calibration, trajectory integrity, and held-out transfer.

EfficiencyCan the loop improve within finite compute, time, and review budgets?

RSI needs cost frontiers: gains should be measured against compute, wall-clock time, failed attempts, deployment cost, and review burden.

DiversityDo proposals cover meaningfully different hypotheses and execution paths?

Diversity is about causal independence, not just different wording; systems should preserve disagreement, branch distance, and failure coverage.

PersistenceDoes a successful intervention remain available and useful later?

Retained artifacts need provenance, scope conditions, retirement criteria, and evidence that they contribute downstream rather than drifting into noise.

AdaptabilityCan the system respond to changing tasks, tools, evaluators, and environments?

The key split is frozen evidence for comparison versus adaptive evidence for discovering new weaknesses as the environment changes.

ControllabilityAre authority, review, monitoring, and rollback coupled to risk?

Update authority should track target scope, verifier strength, deployment context, and failure severity, with intervention points preserved across the loop.

GeneralizationDoes the improvement transfer beyond its original task and evaluator?

The strongest claim is transfer of the improvement procedure itself: later generations should search, evaluate, or optimize better because of what was retained.

Seven future directions and bottlenecks for recursive self-improvement

Recursive self-improvement has moved from theory to engineering practice; the remaining challenge is to make its gains credible, reusable, and transferable beyond the tasks and evaluators that produced them.