AGI Rideout: Managing Risk on the Road to Advanced AI

Deanna Lee

You're listening to Policy Minded, a podcast by RAND. I'm Deanna Lee. In Washington and Silicon Valley alike, achieving artificial general intelligence, or AGI, is increasingly seen as a geopolitical race, a high stakes contest between the United States and China. The idea is simple. The country that reaches AGI first could gain a major and lasting edge.

As the United States and China jockey for position on the road to AGI, RAND researchers have posed a critical question. What if the most important priority isn't winning the race at any cost, but making sure the U.S. navigates that race strategically? A new RAND paper argues that, as the U.S. pursues technological leadership, it needs to adopt an AGI Rideout strategy. This approach would focus on resilience, stability, and reducing the risk of conflict during the transition to advanced AI. Think of it as a playbook for navigating uncertainty.

Our guests today are the authors of the AGI Rideout paper, senior policy researcher David Frelinger and senior political scientist Karl Mueller.

David, welcome.

David Frelinger

Glad to be here.

Deanna Lee

And Karl, thanks for joining us.

Karl Mueller

Happy to talk about it.

Deanna Lee

Okay, before we dive into the premise of your paper, I want to ask you a basic question. We've addressed this topic on the show before, but I think it's worth revisiting for listeners who may need a refresher, maybe for a host that needs a refresher. What exactly is artificial general intelligence or AGI?

Karl Mueller

In the paper we use AGI as a shorthand term to refer to a broad range of possible artificial intelligence that would be considerably more advanced than what we have now and would be as good or better than humans at performing a fairly wide range of functions. It would also be AI that would be capable of doing a lot of self-improvement with minimal human involvement.

We're not particularly wedded to a specific definition of AGI. Most of the things that we have to say in this argument apply to future advanced AI capabilities regardless of whether it's a single unified artificial general intelligence or something else that has similar capabilities.

Deanna Lee

Okay, thank you. And, you know, it's worth pointing out that there isn't one single agreed-upon definition of AGI, so that certainly applies in our discussion today. You talked about some of the scenarios you were looking at with AGI. Broadly, why does the way the United States thinks about AGI matter so much for national security and global stability?

Karl Mueller

It matters because there are a number of bad consequences that potentially result from seeing the AI competition as a winner-takes-all race in which one state is going to dominate the world inevitably as a result of a sort of an intelligence explosion after they cross some AGI threshold and focusing on winning the race at the expense of thinking about questions that we're interested in here regarding how to stabilize the race along the way.

David Frelinger

Yeah, it also creates a situation where countries operate in a rather unpredictable way. Because this is all about expectations for the future, there are many possibilities of misunderstanding during these transition periods. And there are sort of self-induced fears about the consequences of AGI. And so there is a problem that creates sort of instability in the system writ large—either the U.S. perceiving China in a particular way or China perceiving the United States—which leads to a very unstable dynamic in some situations.

Deanna Lee

Okay, thank you. And I think you've kind of demonstrated why that transition period, why the road to AGI is just as important as, you know, what happens when you get to the finish line—that idea of instability. So let's get to your proposal, which is the AGI Rideout strategy. At a high level, what is it exactly?

Karl Mueller

The AGI Rideout strategy is a proposal for the approach that the United States we think should take to the AI race, focusing on building resilience and adaptation capabilities for coping with the changes that will be brought about by the arrival of AI. We're focused primarily on the geopolitical dimensions of this, trying to avoid international conflict related to the AGI competition.

David Frelinger

Right, and we're dealing, particularly in this transition period, with the sort of uneven future as things are unfolding. And the AGI Rideout strategy is dealing with this period, which is potentially much more extended than some people imagine, given an uneven development of artificial intelligence and applications in different domains. And so, it's important to actually have to navigate this environment to even get to the world that people are imagining with AGI, because you can run into many challenges along the way.

Deanna Lee

Right, and you describe it in the paper as shooting the rapids, not competing in a regatta, which is a metaphor that I appreciated. What does that kind of capture, what does that metaphor capture that maybe we miss if we just think about it as a race to the finish line?

Karl Mueller

We use the "shooting the rapids" metaphor because we thought it captured the nature of a race in which surviving the race is a big part of what you're trying to do. And it's a race that's not entirely under your control, right? Very often discussions about the AGI competition tend to assume that states and the firms within the states get to decide how fast the technology advances and what direction it will go.

Our assessment of it is that there's a great propulsive force in the technological progress, and the government has only limited abilities to influence it, has difficulty seeing very far down the path ahead, and needs to be prepared to sort of keep the boat right side up in the course of the instability that it's going to be enduring in the course of AGI development.

David Frelinger

And, indeed, like the river metaphor, in the rapids, is that while you've had a plan to go down the rapid a particular way, this is an unmapped rapid. So there are many possibilities that you went down a path you thought was safe and you discover big problems around that next boulder. So the idea of like lack of control in the environment is critical because it requires we have developed sort of a prudent approach to dealing with these uncertainties. Not just technological future, but in all those application domains Karl touched on.

Deanna Lee

Excellent. I also want to talk a little bit about where the term "rideout" comes from. It's borrowed from nuclear strategy, right? How do lessons from that arena apply?

Karl Mueller

Yeah, "rideout" is a term that we picked up from the nuclear strategy, the nuclear history arena. We're not using it to mean quite the same thing that it did in the nuclear context. Nuclear rideout was about being resilient enough that, in a crisis, you didn't have to respond right away if somebody launched a nuclear weapon in your direction.

In the AGI competition, we're talking about a longer-term competition between the United States and China and potentially other actors. And so, what you're trying to ride out is more along the lines of the transition that happened in nuclear history between the 1950s—when there were a lot of fears that the enemy might strike you in a surprise attack—and the latter part of the 1960s when nuclear arsenals were much more survivable and both the United States and the Soviet Union could have reasonable confidence that the adversary would not be able to disarm them by striking first. And so they had more options, crises became less intense. And the superpowers were not involved in such a, such a hair-trigger standoff.

David Frelinger

And I think, internally—and certainly the rideout from the U.S. perspective—it allowed them to have more confidence as decisionmakers that they could make a decision along the way and wouldn't be a completely life-or-death one. The system became more forgiving, right, to errors in the system or misunderstanding because it created a degree of robustness.

And it removed that sense of urgency at times Karl was talking about, where you had to do an all or nothing sort of operation because you thought you could deal with the next problem. You know, maybe "tomorrow would be the good day to start a nuclear war and not right this second" sort of routine. And for a nuclear analyst, which I was when I started out at RAND, this is a way of kind of thinking about problems that's helpful where you have a lot of uncertainty and perceived risk, right? It's that you want to turn the temperature down, you want to get time—time is an asset. And the Rideout strategy allows you to like, use the time effectively.

Deanna Lee

Right. Thanks for making that distinction. It's very helpful. And let's talk about the goal of Rideout, which you've just touched on a little bit. It is not to win the race at all costs. It's to do the best possible job with this uncertain transition period, right? Is there a singular sort of aim? Is it to prevent conflict? Are there multiple aims? What's the primary goal here?

Karl Mueller

I think, for our purpose, is thinking about it from the geopolitical perspective—and there's other concerns that one would also have about AI development—the overarching goal is really to avoid disasters, in a sense, right? It's to avoid a ruinous war with the Chinese. You would also want to avoid losing the race in some sort of catastrophic way that would open the door to Chinese subjugation of the United States, for example.

But that's not the only risk that you're concerned about, right? There are, along the way, as you're approaching more and more advanced AI, and even after you achieve it, there's a variety of ways where things could go off the rails. And Rideout says that you want to anticipate those and be as prepared as you can to navigate them successfully.

Deanna Lee

Karl, I'm glad you brought up the point of the U.S. losing the race as being an issue here. Because I want to be clear that, in the paper, you're not suggesting that the United States abandon this approach of being the first to develop and exploit AGI. You're just recommending strategic hedging that will prevent a catastrophe, as you say. But let's talk about some of those risks in more detail that you hinted at, Karl. You mentioned a ruinous war with China. What are the primary risks that could come up during this transition period to AGI, and how specifically does Rideout address them?

Karl Mueller

The way we break it down is that I think that it's useful to think about sort of three ways that you could get to a major conflict that you would regret associated with the AI race in particular, right? One is the possibility that somebody develops a capability that they can just use to try to destroy your country or inflict vast harm on you and worry about that. I think particularly with other sorts of adversaries where we worry about them getting biological wonder weapon through their use of AI or something to launch a terrible attack against the United States.

Another possibility is you get a conflict that's launched because somebody who's losing the race doesn't want to lose the race and I think it's going to be terrible for them if they do, right, and fear that an American victory in the AI competition will result in the United States taking over the world, if you want to oversimplify.

Karl Mueller

And then the third possibility is that AI capabilities create perceptions that there are attractive opportunities for regional aggression that could then spiral into a bigger conflict. The obvious possibility here would be China developing AI-enabled capabilities that could make leaders in Beijing think they could inexpensively and successfully attack Taiwan. That starts a war that the United States participates in, and then this escalates into a larger conflict. And that's a good example of the sort of AI developments that don't depend on somebody getting to AGI first that could happen along the way that could result in dire consequences if we're not prepared to cope with them.

Deanna Lee

I also want to ask how the U.S. can balance its pursuit of innovation and its continued pursuit of realizing AGI while it's still doing all of the things that Rideout sort of suggests that it does in terms of protecting against these catastrophes. Are there specific recommendations you offer in the paper to achieve that balance?

Karl Mueller

To a significant extent, we don't see that as a fundamental trade-off between hedging and pursuing success in the competition. The propulsive force for AI progress mostly comes from the private sector, as the AI firms are pursuing profit and through technological progress, it's not really the government's job to make that happen. And indeed, our ability to control it is somewhat limited. The hedging is something that we argue you should pay a lot of attention to doing along the way, regardless of how fast you're going in the race and regardless of how much you are leading the competition by in the competition.

David Frelinger

And I think the key here is imagining this kind of hedging strategy coming from a completely sort of different bucket of money, like Karl was talking about. These are things to adjust, for instance, the Department of War's perspective on these issues. How to think about reassuring allies, how to approach things about the business of dealing with international security regimes.

So, they're essentially complementary to each other because, if rideout is well done, it creates a more stable environment for progress in these other areas because any of the things Karl discussed leading to a war, even a limited war over Taiwan, could lead to loss of a lot of resources that are necessary like chips that are vital for our current approaches using LLMs and hyperscale server farms. So these really are things really to make that go better and to provide some insurance against incremental improvements destabilizing the situation.

Deanna Lee

Alright, so let's talk about putting this strategy into practice. You write that this would be a fundamental shift from what the U.S. is doing now, different from that win-the-race-at-all-costs approach. So, if U.S. leaders were to embrace this AGI Rideout strategy tomorrow, what should they do first?

Karl Mueller

I think the first order of business would basically be a shift in mindset rather than a specific policy action, which sounds on the face of it, sort of fluffy, but could be quite consequential. But in more concrete terms, the paper makes some recommendations about setting up new organizations within the defense and intelligence apparatuses to try to monitor and adapt to adversary AI innovations. That could be disruptive for U.S. national security.

David Frelinger

And that part is actually critical, because organizational changes like that take considerable amounts of time for the organizations to figure out how to absorb it. In our paper, we discussed creating two distinct types of organizations—one is an intelligence organization focused primarily on AI and AI applications that would provide targeted warnings to a defense organization whose job was to translated into actionable information that can be used to select programs for rapid development and go through different sort of department and service organizations that already exist to try to create things based on emerging situations. This allows us to respond effectively to early warnings of AI and to tailor them specifically to the most pressing questions. And you get that between the interaction between the intelligence community and the defense communities.

Deanna Lee

Okay, interesting. Any other keys to implementing this strategy in early days in particular that you want to highlight?

Karl Mueller

I think one would be the potential value of playing down the idea that this is a winner-take-all race, right? This is about perceptions, and stoking perceptions that this is an fight for control of the world is not helpful for international stability in the broad sense, and also risks making it more difficult to cooperate with China in particular on areas where there might be mutual interest. In the AI arena that might be things like cooperating on standards revolving safety of the AI models or applications of AI for mutually beneficial purposes in areas like public health or other sorts of areas where we expect to reap big benefits from the technology.

David Frelinger

Yeah, and I think the open-ended nature of this is kind of critical. Not having this like simple point that yo say, well, I've reached AGI and it's over, or I've reached this other threshold and it is over. Even if this is a very powerful technology in all likelihood, given the difficulties of diffusing it into economies and in the society, some of this will extend considerably longer than some advocates imagine. And trying to work cooperatively with China, for instance, on some of these issues, and avoiding kind of framing things in a sort of "twilight of the gods" type of battle, right, is really helpful. Because, as you're doing this, there are those opportunities for cooperation.

Also, as we're evolving and discovering in this uneven frontier—as these new AIs are being developed, they're being applied in different ways—there are unforeseen areas where cooperation may be necessary as we're going forward. We are, humanity in general, not very good at predicting the details of the future. We may have sort of broad brushstroke versions. But these details matter and can matter a lot in this international environment, especially considering this is affecting not only the U.S. and China but a whole host of other countries that can lead to problems as well, that blow over into the United States or to our partners around the world.

Deanna Lee

What are the odds of cooperation, specifically between the U.S. and China on this matter? I'm wondering to myself, is there a possibility that China embraces a strategy like this if the U.S. also does? Or is that off the table? What are your thoughts?

Karl Mueller

It's certainly possible that China would adopt such a strategy towards the AI competition. And indeed, that may be what they're following now. They've traditionally been in the position of followers rather than leaders in this race, like a lot of other technological competitions. And the idea that China would be looking at the competition as something where the goal was not to leap into the forefront of the competition right away but rather to stay close to the United States and stay in the game and find areas in which it could excel and gain advantages is quite consistent with a lot of the Chinese behavior that we've seen. And when we play games simulating the AI competition, we hear a lot that from our China expert players.

David Frelinger

And many of their strategic culture tends to favor this, not only in this kind of economic and technological competition. They haven't been in a place where they've had decisive advantages in many areas and have ended up being relatively cautious. So, while they may not think of it in the exactly same framing we're discussing in practice, it may amount to kind of the same thing at times—though, as always with unique Chinese characteristics, right? And so, those are the details, for instance, our China experts have to spend a lot of time on is trying to understand what that means to them. And it's always dangerous for us to project ourselves onto China. They like to point out they're a very old and serious civilization. And so they have their own approaches to these problems.

The thing is to find levels of compatibility. There isn't necessarily ones where these are in complete opposition to one another all the time. A sort of cautious strategy of hedging against the other side will have aspects that are looking like kind of classical arms competition, but the subtle differences matter, right, on exactly how offensively oriented it seems to be, what the mix of things are. That is back to knowledge of what your adversary is doing and thinking about and how they're putting it into their broader political, economic, and military doctrines. And so they're putting it into a different system than the United States. And so I really tend to think there are a lot of opportunities for us going forward, as we're both developing things, to see points of cooperation as well as points of competition.

Karl Mueller

This is not to say that cooperation with China will be easy to establish, as Dave likes to point out. This is not a technological competition, it's a geopolitical competition with a big technological component to it. And so, in a situation in which the relationship is quite fraught, even when there are areas of common interest on the technological front, it doesn't mean that the competition will be easy to establish. But if you look at the AI competition as a long-run thing, not a game that's going to end right away, then playing the long game makes a lot of sense for both sides.

Deanna Lee

David, I want to go back to something you said earlier, which was that the pursuit of innovation and strategic hedging, it's not necessarily one or the other. I'm wondering what the trade-offs are, then, with pursuing this AGI Rideout strategy. Are there costs that leaders need to be aware of or any downsides in particular?

David Frelinger

Well, in a resource-constrained world, there is a choice. The marginal dollar is going one way or another. The main thing we're kind of thinking is, as we're identifying the sort of dangers, a intelligence-driven approach will let us use the funds most efficiently. We will target the most destabilizing elements in order to preserve the system. So it actually does take a little bit away on the margin from other, you know, sort of plans of record.

But you might actually start thinking about this as a way of stabilizing the situation, so you can proceed with more natural transitions in other areas. So otherwise, you wait until too late, you have a crash program that's even worse, right, and as a higher prospect of not working. Whereas, if you're doing this all the time, you're getting both more knowledge about it, you're signaling to the other side what you're intending to do. And in the process, that creates a system, I think, that works better overall.

And well, the money has to come from somewhere, right? Even inside a large trillion dollar sort of budget, you can make some trade-offs, right, on this stuff, and kind of view this as a way of making other parts of the apparatus work well. But this is one of the reasons—this is back to Karl's idea—that you don't want to fall into the position that it is a, I get to AGI, everything's over, I win, victory is mine, because you won't do these hedging sorts of things along the way, and your business as usual won't necessarily leave you in the best position. Karl can probably expand more eloquently on this than I can.

Karl Mueller

One of the things that the paper is fundamentally arguing against is the perspective that, because large payoffs are expected if we win the competition to achieve AGI, that will solve all of our problems, right? That losing the race would be dangerous, and therefore winning the race will make us safe. And what we're arguing here is that there are a lot of dangers along the way that precede the end of the race to the extent that you can envision the race sort of having a culminating point, and that, even after the winner of the race is determined, being the most advanced state in the AGI race doesn't make you invulnerable to the things that your less advanced rival might be able to do with AI to impede or to injure or to threaten you.

Deanna Lee

Absolutely, it makes sense. Now, a theme of our conversation today has been uncertainty, for sure. One of the biggest uncertainties is that no one really knows when AGI might be realized. So, what would success look like if, say, five, ten years from now, the U.S. has been executing this strategy and has been executing it well? What does that look like exactly?

Karl Mueller

I think for the most part, it would look relatively uneventful. It would mean that major conflict had been avoided, that deterrence had not failed, and that the United States felt like it was in a position where it was well positioned to counter the surprises that might be thrown at it, either by the technology itself developing, or by the things that China or other states might do with the technology. The United States would feel relatively secure in spite of the changes coming down the road, which are often difficult to predict even if you put a lot of effort into trying to anticipate.

David Frelinger

My glib answer is that a well-executed Rideout strategy should be almost invisible, that in fact, nobody thinks about it. It is just like electricity, it just works. You throw the switch on, stuff happens, we haven't been terribly surprised. Anything that has happened, we were able to contain well—just kind of another day at the office, right? That's what you're aiming for. And that provides an environment where everybody else can go about their business and either try to develop advanced technology, pursue geopolitical interests. Everything just works.

Deanna Lee

So what's the single most important thing the U.S. can do right now? We talked about that mindset shift. It seems like that's probably the first and maybe the biggest hurdle right now. Are there actions that the U.S. can take to get there?

David Frelinger

Well, I'll lead off. I'll say one of the main things it can do is it puts money against it. It starts acting like that's the future it's aiming for. And not just rhetorically, it actually says, I'm going to cope with this world. I'm going to reorganize some things. I'm going to focus forward. I'm going do that openly to tell people this is what I am doing and why am I doing it, and proceeding in a way that makes everybody think that, well, there might be something there, right…if they're doing it? And that's actually an important action in this space is to signal that not only to other parties in the world but other parts of the government, to the country, that we're taking this seriously, we're going to really try to protect against these uncertainties, and we're not aiming to do something that is completely reckless. We're trying to be the responsible party, showing people what a responsible party does.

We did that in the nuclear arena, you know, over the period that Karl was talking about. And in that world, it was like every day, SAC in particular, they have not fired a nuclear weapon, deterrence held. In retrospect, people think that was like a pretty stable environment, but you know, it did its job. And in our environment here, we want to be able to make sure we can catch some things before the problems pop up. And start signaling early, that's the way we're thinking about it. And I think that's valuable.

Deanna Lee

Absolutely. Okay, before we wrap up, anything else that either of you want to underscore about AGI Rideout and how important it is to take this approach or move away from the current race that the U.S. is in?

Karl Mueller

I think the one thing I would add, in the Rideout paper, we focused very heavily on the geopolitical dimension—avoiding international conflict, trying to prevent the AI race from leading to instability and harm that could be avoided. But these are obviously not the only concerns with the development of advanced AI, right? There's also concerns about economic disruption, disruptive sides, societal changes, AI agents getting loose and causing harm on their own and in autonomous ways. And although the Rideout strategy, as we've framed it, doesn't address these sorts of things, the same sort of approach also applies in those cases where we would suggest that the general approach of making concerted efforts to anticipate these sorts of problems and acting early to try to build the capabilities to make adjustments, to adapt to the changes that are necessary and to try to provide offsets and countermeasures for the things that you can is very useful in those areas. And in many ways, some of those threats may be even more dire than the sort of worst-case scenarios that we're thinking about in the context of the geopolitical competition.

David Frelinger

Yeah, and oddly, looking at other countries, it's easier for us to see those problems and start suggesting solutions and thinking them through. So the more we pursue down this line, you start seeing what's working and not working internationally, you have a better idea of how people have coped with it. Some of those you can adopt, some of them you can't. So this actually creates a shift in mindset that's important. But Karl and I play and design games, and one of the things we have in a game is that they see the end of the game track or the end number of turns, and people start doing crazy things, right? And they play the game like it's gonna end. If you have an open-ended game, everything changes. And rideout, in fact, is a way that says, this is gonna be an open-ended problem. If AGI arrives and it's an uneven one, the future continues in ways we don't really understand, but it continues, and we're gonna be dealing with it for some time. And as policy analysts and researchers, a lot of what we have to do is help people to kind of think about how you deal with tomorrow, right? And the day after tomorrow, and putting yourself in the best position and not just trying to dive for a line.

Deanna Lee

Okay, thank you both. I think that's all that we have time for today. David, we appreciate you being here.

David Frelinger

Happy to be here.

Deanna Lee

And Karl, thanks for being on the show as well.

Karl Mueller

Enjoyed it.

Deanna Lee

And as always, thank you to our listeners. If you want to learn more about the AGI Rideout strategy, you can find a link to David and Karl's paper at rand.org/podcast.

Today's episode was recorded and edited by Emily Ashenfelter. It was produced by me, Deanna Lee. RAND's director of digital communications is Pete Wilmoth.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis.