AI agents are quickly moving beyond systems that simply receive an instruction, call a tool, and return an answer. A growing research direction asks a more ambitious question: can an AI agent improve itself through experience? These systems are known as self-evolving (or self-improving / recursively self-improving) AI agents. Rather than keeping the same behavior after deployment, they may learn from previous failures, accumulate memory, build reusable skills, improve prompts, adapt tool-use strategies, or modify parts of their own reasoning pipeline. In this article, we will cover the 7 best resources to learn about self-evolving AI agents.
1. Hugging Face Agents Course
I would recommend this as the first resource before you study agents that modify themselves. It will help you understand how ordinary agents work. It teaches the basic Think/Act/Observe loop, tool use, reasoning, agent frameworks such as smolagents, LangGraph and LlamaIndex, agentic retrieval-augmented generation (RAG), function-calling fine-tuning, observability, and evaluation. The course is free and includes hands-on assignments and a final benchmark-based project.
2. Stanford CS329A: Self-Improving AI Agents
Stanford’s CS329A: Self-Improving AI Agents is probably the best structured academic starting point for this topic. It covers self-improvement techniques for large language models (LLMs) — including Constitutional AI, verifiers, test-time compute, and reinforcement learning — as well as tool use, memory, multi-step reasoning and planning, evaluation frameworks, and applications such as coding agents and research assistants. The best part is that the syllabus is organized around research papers rather than just agentic frameworks, so it gives you a deeper understanding of how agents improve themselves. I would highly recommend following the lecture sequence, reading the papers, and trying to reproduce a few of the ideas.
3. A Comprehensive Survey of Self-Evolving AI Agents
This survey is one of the most useful resources for understanding what researchers actually mean by a self-evolving agent. It frames evolution as a feedback loop connecting the agent, its environment, system inputs, and an optimizer. The survey discusses which parts of an agent can evolve — including the model, memory, prompts, tools, and multi-agent organization — and covers domain-specific systems in programming, finance, and biomedicine.
4. Self-Improvements in Modern Agentic Systems: A Survey
This is a useful complement to the previous survey. It tells you what the field looks like in 2026. It separates improvement of the foundation model itself from improvement of the agent’s scaffolding — such as prompts, memory, tools, skills, and control logic. An agent that rewrites its prompt, one that builds a skill library, and one that fine-tunes its own model are doing very different things. This survey provides a system-level taxonomy for understanding those differences and discusses evaluation and open research problems.
5. Awesome Self-Improving Modern Agentic Systems
You can use this repo as a bibliography after reading one of the surveys above. The repository organizes papers by what is being improved: model parameters, prompts, memory, tools, skills, and complete agent scaffolds. It also collects benchmarks, courses, talks, workshops, code, and newer 2026 work such as OpenSkill and evolving skill systems. Given how quickly agent research is developing, a continuously updated bibliography like this is often more useful than repeatedly searching arXiv from scratch.
6. Awesome RSI (Recursive Self-Improvement)
This is a broad research map that covers model-level self-improvement, harness and scaffold evolution, memory, embodied systems, automated AI R&D, benchmarks, and safety. It includes papers, frameworks, tools, and evaluation resources across these different directions. It’s extremely useful for seeing how self-evolving agents fit into the larger recursive self-improvement (RSI) landscape.
7. Awesome Harness Engineering for Self-Improvement
This list is narrower and more operational. It focuses on the “harness” — the surrounding system of tools, memory, control loops, evaluation, and so on. It includes foundational essays, papers, and practical engineering perspectives, making it useful for understanding how agents can improve not just their outputs, but the harness and processes they use to produce those outputs.
Wrapping Up
In my opinion, a sensible learning order for this topic would be Hugging Face Agents Course → Stanford CS329A → the two surveys → Awesome Self-Improving Agents → Awesome RSI → Awesome Harness Self-Improvement. The field is progressing rapidly, so don’t worry about trying to read everything. Pick a direction that interests you, follow a few key papers, and build something yourself.
Kanwal Mehreen is a machine learning engineer and a technical writer with a profound passion for data science and the intersection of AI with medicine. She co-authored the ebook “Maximizing Productivity with ChatGPT”. As a Google Generation Scholar 2022 for APAC, she champions diversity and academic excellence. She’s also recognized as a Teradata Diversity in Tech Scholar, Mitacs Globalink Research Scholar, and Harvard WeCode Scholar. Kanwal is an ardent advocate for change, having founded FEMCodes to empower women in STEM fields.
