Topline
AI is already helping build better AI. The window for oversight is shrinking.
Artificial intelligence (AI) systems are already automating meaningful components of AI research and development. If those systems begin contributing to their own improvement—a phenomenon known as recursive self-improvement (RSI)—capability gains could arrive faster than institutions and governance can absorb.
In this paper, the authors examine the empirical state of AI systems contributing to their own improvement. The authors analyze current trends, introduce a taxonomy for categorizing RSI-relevant activity, and propose policy options that perform well across a range of plausible development trajectories.
Key Takeaways
AI systems are already automating significant parts of AI research.
Multiple frontier labs report that a majority of new code is AI-generated. However, no disclosed system sets its own research agenda or designs its own successors.
It is unclear how much AI is accelerating this.
No systematic reporting exists from frontier labs to leading hardware vendors on automation share, compute allocated to AI-directed experimentation, or the extent to which model improvement traces to AI-generated ideas.
AI conducting research inside labs is already a security problem.
These systems hold insider-level access to a lab’s most sensitive code, data, and models. Compromising an internally deployed research agent could subtly influence the development of future AI systems.
Topics
This publication is part of the RAND expert insights series. The expert insights series presents perspectives on timely policy issues.
This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.
RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.